From canvassedk50 at lipsbags.com Sat Mar 1 19:22:33 2008 From: canvassedk50 at lipsbags.com (Pamela Earl) Date: Sat, 2 Mar 2008 11:22:33 +0800 Subject: [ofa-general] HealthCertifiedWorldwideShipping Message-ID: <01c87c57$b58d6280$bcc4003c@canvassedk50> InternetPricesMedsInternetPrices http://diannkregerhu.blogspot.com From dwsavvyconsultingm at savvyconsulting.com Sun Mar 2 00:32:39 2008 From: dwsavvyconsultingm at savvyconsulting.com (Christian Haines) Date: Sat, 2 Mar 2008 09:32:39 +0100 Subject: [ofa-general] Medications that you need. Message-ID: <01c87c48$5b38d580$40c4ec4d@dwsavvyconsultingm> Buy Must Have medications at Canada based pharmacy. No prescription at all! Same quality! Save your money, buy pills immediately! http://geocities.com/brendanbeard83 We provide confidential and secure purchase! From a-and-c-russell at adison.com Sun Mar 2 01:59:25 2008 From: a-and-c-russell at adison.com (Bethany Barber) Date: Sat, 2 Mar 2008 17:59:25 +0800 Subject: [ofa-general] What are you up to? Message-ID: <01c87c8f$269c3c80$74f2f4de@a-and-c-russell> Hello! I am tired this evening. I am nice girl that would like to chat with you. Email me at Inger at GreatGolovaSite.com only, because I am using my friend's email to write this. Would you mind if I share some of my pictures with you? From vlad at lists.openfabrics.org Sat Mar 1 03:07:30 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sat, 1 Mar 2008 03:07:30 -0800 (PST) Subject: [ofa-general] ofa_1_3_kernel 20080301-0200 daily build status Message-ID: <20080301110730.82CC3E60289@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From kweber at uruguay-natural.com Sun Mar 2 05:45:38 2008 From: kweber at uruguay-natural.com (Lindsey Oakes) Date: Sat, 2 Mar 2008 05:45:38 -0800 Subject: [ofa-general] Qualität hoch, Preis niedrig, Software prima! Message-ID: <145692881.98887691342551@uruguay-natural.com> Die Software, legal und billig, ist möglichWir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren. http://geocities.com/wilderabdul/ Unser Preis: * Microsoft Office Enterprise 2007: $79.95 * Adobe Acrobat 8.0 Professional: $69.95 * Office 2003 Professional (including Publisher 2003): $59.95 * Adobe Creative Suite 3 Master Collection: $299.95 * Adobe Creative Suite 3 Design Premium: $229.95 * AutoCAD 2008: $129.95 http://geocities.com/wilderabdul/ Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-annvik at adeptia.com Sun Mar 2 07:15:15 2008 From: a-annvik at adeptia.com (Cedric Barber) Date: Sat, 2 Mar 2008 23:15:15 +0800 Subject: [ofa-general] Chatting online Message-ID: <906478918.64881307313814@adeptia.com> Hello! I am tired today. I am nice girl that would like to chat with you. Email me at Birgit at GreatGolovaSite.com only, because I am using my friend's email to write this. I would like to share some of my pics. From sashak at voltaire.com Sat Mar 1 07:52:03 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 1 Mar 2008 15:52:03 +0000 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <20080301014635.GF1485@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> Message-ID: <20080301155203.GK1485@sashak.voltaire.com> On 01:46 Sat 01 Mar , Sasha Khapyorsky wrote: > On 09:58 Fri 29 Feb , Albert Chu wrote: > > > > What you're trying to do is calculate "ignore_existing_lfts" when the port > > trap is received rather than during routing later on? > > Not trap but when PortInfo is received during subnet discovery phase of > sweep (before routing configuration). I started to test this I saw that it solves the problem only partially. For example in such topology: sw3 / \ - sw1 sw2 -> \ / sw4 Let's disconnect link between switches 2 and 3, paths to arrow direction at switch 1 will be rerouted via sw4. Now let's restore the link - switches 2 and 3 will be enforced to rebalance routing paths, but sw1 will not and all traffic will go via sw4. Looks we need to set ignore_existing_lfts globally when any switch was reconnected in a fabric. Sasha From anewton at anytownnotary.com Sun Mar 2 07:46:35 2008 From: anewton at anytownnotary.com (Mason Lay) Date: Sat, 2 Mar 2008 16:46:35 +0100 Subject: [ofa-general] Industriestandard der Software fü r weniger als billig Message-ID: <01c87c84$f9e32780$e4e5114f@anewton> Suchen Sie nach der Software? Mochten sie momentan bekommen? Das ist das! Nur bezahlen und auslasten. Die Programmen sind auf allen europaischen Sprachen uberlassen und fur Windows und Macintosh vorherbestimmt http://geocities.com/forrest.faulkner/Haben Sie Haben Sie Schwierigkeiten bei der Aufstellung des Programms? Sie bekommen die Hilfe der professionellen Konsultation des Anwenderdienstes. Haben Sie Fragen? Wir antworten schnell. Die Ruckzahlung ist moglich. Sie kaufen nur die ausgezeichnet funktionierende Software http://geocities.com/forrest.faulkner/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Sat Mar 1 08:08:25 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 1 Mar 2008 16:08:25 +0000 Subject: [ofa-general] [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <20080301155203.GK1485@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> Message-ID: <20080301160825.GL1485@sashak.voltaire.com> When switch ports were reconnected we need to recalculate routing paths balancing. Reconnection is detected by port state examination - when it becomes INIT routing paths rebalancing (ignore_existing_lfts flag) is enforced. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_port_info_rcv.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_port_info_rcv.c b/opensm/opensm/osm_port_info_rcv.c index ecac2a8..4c51829 100644 --- a/opensm/opensm/osm_port_info_rcv.c +++ b/opensm/opensm/osm_port_info_rcv.c @@ -316,6 +316,9 @@ __osm_pi_rcv_process_switch_port(IN osm_sm_t * sm, if (ib_port_info_get_port_state(p_pi) > IB_LINK_INIT && p_node->sw) p_node->sw->need_update = 0; + + if (p_physp->need_update) + sm->p_subn->ignore_existing_lfts = 1; if (port_num == 0) pi_rcv_check_and_fix_lid(sm->p_log, p_pi, p_physp); -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Sat Mar 1 08:10:23 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 1 Mar 2008 16:10:23 +0000 Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <20080301160825.GL1485@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> Message-ID: <20080301161023.GM1485@sashak.voltaire.com> Hi Al, On 16:08 Sat 01 Mar , Sasha Khapyorsky wrote: > > When switch ports were reconnected we need to recalculate routing paths > balancing. Reconnection is detected by port state examination - when it > becomes INIT routing paths rebalancing (ignore_existing_lfts flag) is > enforced. > > Signed-off-by: Sasha Khapyorsky This patch is simpler than all previous ones. I tested it with ibsim already. Could you test in your environment? Sasha From chu11 at llnl.gov Sat Mar 1 09:11:21 2008 From: chu11 at llnl.gov (Albert Chu) Date: Sat, 1 Mar 2008 09:11:21 -0800 (PST) Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <20080301161023.GM1485@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> <20080301161023.GM1485@sashak.voltaire.com> Message-ID: <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> Hey Sasha, This patch should definitely work. I'll let you know after I get a chance to try it. Al > Hi Al, > > On 16:08 Sat 01 Mar , Sasha Khapyorsky wrote: >> >> When switch ports were reconnected we need to recalculate routing paths >> balancing. Reconnection is detected by port state examination - when it >> becomes INIT routing paths rebalancing (ignore_existing_lfts flag) is >> enforced. >> >> Signed-off-by: Sasha Khapyorsky > > This patch is simpler than all previous ones. I tested it with ibsim > already. Could you test in your environment? > > Sasha > -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From darylgrunau at gmail.com Sat Mar 1 09:29:42 2008 From: darylgrunau at gmail.com (Daryl W. Grunau) Date: Sat, 1 Mar 2008 10:29:42 -0700 Subject: [ofa-general] OFED 1.3 GA fails to build ofa_kernel RPM on FC8/ppc64 Message-ID: <20080301172942.GA3283@gmail.com> Anybody else running into RPM build problems of 1.3 GA? All other packages build fine but when I get to the kernel-ib RPMS the rpmbuild check-buildroot puts a halt to everything: + /usr/lib/rpm/check-buildroot Binary file /var/tmp/OFED/lib/modules/2.6.23.1-42.7.cell/updates/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko matches Binary file /var/tmp/OFED/lib/modules/2.6.23.1-42.7.cell/updates/kernel/drivers/infiniband/hw/ipath/ib_ipath.ko matches Binary file /var/tmp/OFED/lib/modules/2.6.23.1-42.7.cell/updates/kernel/drivers/infiniband/hw/nes/iw_nes.ko matches . . . /BUILD/ofa_kernel-1.3/drivers/infiniband/core/ib_core EXPORT_SYMBOL /var/tmp/OFED/opt/ofed/src/ofa_kernel/configure.mk.kernel:CWD=/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3 /var/tmp/OFED/opt/ofed/src/ofa_kernel/configure.mk.kernel:AUTOCONF_H=/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3/include/linux/autoconf.h /var/tmp/OFED/opt/ofed/src/ofa_kernel-1.3/ofed_scripts/ofa_kernel.spec:BuildRoot: %{?build_root:%{build_root}}%{!?build_root:/var/tmp/OFED} Found '/var/tmp/OFED' in installed files; aborting Here's the relevant code in the check-buildroot script: find "$RPM_BUILD_ROOT" \! \( \ -name '*.pyo' -o -name '*.pyc' -o -name '*.elc' -o -name '.packlist' \ -o -name '*.jar.so.debug' \) -type f -print0 | \ LANG=C xargs -0r grep -F "$RPM_BUILD_ROOT" >$tmp test -s "$tmp" && { cat "$tmp" echo $"Found '$RPM_BUILD_ROOT' in installed files; aborting" exit 1 } || : Executing strings on one of the files it calls out shows: strings /var/tmp/OFED/lib/modules/2.6.23.1-42.7.cell/updates/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko | grep /var/tmp/OFED /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3/drivers/infiniband/hw/mthca/mthca_cmd.c /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3/drivers/infiniband/hw/mthca/mthca_mr.c /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3/drivers/infiniband/hw/mthca/mthca_mcg.c /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.3/drivers/infiniband/hw/mthca/mthca_memfree.c i.e. the grep is failing since the TOPDIR contains the $RPM_BUILD_ROOT path (here, "/var/tmp/OFED"). Ensuring that /var/tmp/OFED is not in the TOPDIR string would work around this problem, however I wouldn't want to speculate which is the correct solution (fix check-buildroot?). I opened bug 968 re: this problem. Anyone else seeing this issue? Daryl From linazqmet at azq.de Sat Mar 1 14:10:15 2008 From: linazqmet at azq.de (Klement BUTTRICK) Date: Sat, 1 Mar 2008 18:10:15 -0400 Subject: [ofa-general] All Mens Need This Message-ID: <331627043.74811550707805@azq.de> Hi there OpenibThe guys get jealous now when they see me in the bathroom.Klement BUTTRICKhttp://www.llfk.com/UDHME/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Sat Mar 1 13:01:32 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 1 Mar 2008 21:01:32 +0000 Subject: [ofa-general] Re: QoS management in OpenSM - doc In-Reply-To: <47C576E0.8050405@dev.mellanox.co.il> References: <47C576E0.8050405@dev.mellanox.co.il> Message-ID: <20080301210132.GA11788@sashak.voltaire.com> Hi Yevgeny, On 16:42 Wed 27 Feb , Yevgeny Kliteynik wrote: > > The following doc describes QoS management in OpenSM. > This doc (named QoS_management_in_OpenSM.txt) has been added to > the OFED docs, along with the QoS_in_OFED.txt. > > I'd like to add this info to OpenSM man pages as well. Yes, I think that it could be useful to have it under opensm/doc too. > I'm including the text here as is, so it will be easier to follow > possible changes. When those will be done, I'll fix the format to > match the OpenSM man pages and post a patch. > > The only problem is that the whole OpenSM man has ~850 lines, > while this QoS management file has ~500 lines... :) I would suggest to have some basic part (50-100 lines) included in the man page and reference an entire document (under opensm/doc) for more details. > Please review. Looks fine, few tiny nits are below. [snip...] > ============================================================================== > 4. Policy File Syntax Guidelines > ============================================================================== > > - Empty lines are ignored. It is mentioned on the next line too. > - Leading and trailing blanks, as well as empty lines, are ignored, so > the indentation in the example is just for better readability. > - Comments are started with the pound sign (#) and terminated by EOL. > - Any keyword should be the first non-blank in the line, unless it's a > comment. > - Keywords that denote section/subsection start have matching closing > keywords. > - Having a QoS Level named "DEFAULT" is a must - it is applied to PR/MPR > requests that didn't match any of the matching rules. > - Any section/subsection of the policy file is optional. [snip...] > ============================================================================== > 6. Simplified QoS Policy - Details and Examples > ============================================================================== > > Simplified QoS policy match rules are tailored for matching ULPs (or some > application on top of a ULP) PR/MPR requests. This section has a list of > per-ULP (or per-application) match rules and the SL that should be enforced > on the matched PR/MPR query. > > Match rules include: > - Default match rule that is applied to PR/MPR query that didn't match any > of the other match rules > - SDP > - SDP application with a specific target TCP/IP port range > - SRP with a specific target IB port GUID > - RDS > - iSER > - iSER application with a specific target TCP/IP port range > - IPoIB with a default PKey > - IPoIB with a specific PKey > - any ULP/application with a specific Service ID in the PR/MPR query > - any ULP/application with a specific PKey in the PR/MPR query > - any ULP/application with a specific target IB port GUID in the PR/MPR > query > > Since any section of the policy file is optional, as long as basic rules of > the file are kept (such as no referring to nonexisting port group, having > default QoS Level, etc), the simplified policy section (qos-ulps) can serve > as a complete QoS policy file. > The shortest policy file in this case would be as follows: > > qos-ulps > default : 0 #default SL > end-qos-ulps > > It is equivalent to the previous example of the shortest policy file, and > it > is also equivalent to not having policy file at all. > > Below is an example of simplified QoS policy with all the possible > keywords: > > qos-ulps > default : 0 # default SL > sdp, port-num 30000 : 0 # SL for application running on > top > # of SDP when a destination > # TCP/IPport is 30000 > sdp, port-num 10000-20000 : 0 > sdp : 1 # default SL for any other > # application running on top of > SDP > rds : 2 # SL for RDS traffic > iser, port-num 900 : 0 # SL for iSER with a specific > target > # port > iser : 3 # default SL for iSER > ipoib, pkey 0x0001 : 0 # SL for IPoIB on partition with > # pkey 0x0001 > ipoib : 4 # default IPoIB partition, > # pkey=0x7FFF > any, service-id 0x6234 : 6 # match any PR/MPR query with a > # specific Service ID > any, pkey 0x0ABC : 6 # match any PR/MPR query with a > # specific PKey > srp, target-port-guid 0x1234 : 5 # SRP when SRP Target is located > on > # a specified IB port GUID > any, target-port-guid 0x0ABC-0xFFFFF : 6 # match any PR/MPR query > with > # a specific target port GUID > end-qos-ulps Likely I missed this in implementation phase. But isn't it better to have ULPs to match QoS level rather than SL? Or probably both? > Similar to the full policy definition, matching of PR/MPR queries is done > in > order of appearance in the QoS policy file such as the first match takes > precedence, except for the "default" rule, which is applied only if the > query > didn't match any other rule. > > All other sections of the QoS policy file take precedence over the qos-ulps > section. That is, if a policy file has both qos-match-rules and qos-ulps > sections, then any query is matched first against the rules in the > qos-match-rules section, and only if there was no match, the query is > matched > against the rules in qos-ulps section. > > Note that some of these match rules may overlap, so in order to use the > simplified QoS definition effectively, it is important to understand how > each > of the ULPs is matched: > > 6.1 IPoIB > IPoIB query is matched by PKey. Default PKey for IPoIB partition is 0x7fff, > so > the following three match rules are equivalent: > > ipoib : > ipoib, 0x7fff : > any, pkey 0x7fff : > > 6.2 SDP > SDP PR query is matched by Service ID. The Service-ID for SDP is > 0x000000000001PPPP, where PPPP are 4 hex digits holding the remote TCP/IP > Port > Number to connect to. The following two match rules are equivalent: > > sdp : > any, service-id 0x0000000000010000-0x000000000001ffff : > > 6.3 RDS > Similar to SDP, RDS PR query is matched by Service ID. The Service ID for > RDS > is 0x000000000106PPPP, where PPPP are 4 hex digits holding the remote > TCP/IP > Port Number to connect to. Default port number for RDS is 0x48CA, which > makes > a default Service-ID 0x00000000010648CA. The following two match rules are > equivalent: > > rds : > any, service-id 0x00000000010648CA : > > 6.4 iSER > Similar to RDS, iSER query is matched by Service ID, where the the Service > ID > is also 0x000000000106PPPP. Default port number for iSER is 0x035C, which > makes > a default Service-ID 0x000000000106035C. The following two match rules are > equivalent: > > iser : > any, service-id 0x000000000106035C : > > 6.5 SRP > Service ID for SRP varies from storage vendor to vendor, thus SRP query is > matched by the target IB port GUID. The following two match rules are > equivalent: > > srp, target-port-guid 0x1234 : > any, target-port-guid 0x1234 : > > Note that any of the above ULPs might contain target port GUID in the PR > query, so in order for these queries not to be recognized by the QoS > manager > as SRP, the SRP match rule (or any match rule that refers to the target > port > guid only) should be placed at the end of the qos-ulps match rules. > > 6.6 MPI > SL for MPI is manually configured by MPI admin. OpenSM is not forcing any > SL > on the MPI traffic, and that's why it is the only ULP that did not appear > in > the qos-ulps section. > > > ============================================================================== > 7. SL2VL Mapping and VL Arbitration > ============================================================================== I think here should be stated that both policies should be merged at some point. [snip...] Sasha From sashak at voltaire.com Sat Mar 1 13:05:09 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 1 Mar 2008 21:05:09 +0000 Subject: [ofa-general] [PATCH] opensm: consolidate osm_sa_vendor_send() status check Message-ID: <20080301210509.GC11788@sashak.voltaire.com> Consolidate osm_sa_vendor_send() status check and error message log in case of failure. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_inform.c | 10 ++-------- opensm/opensm/osm_sa.c | 6 +++++- opensm/opensm/osm_sa_class_port_info.c | 10 +--------- opensm/opensm/osm_sa_guidinfo_record.c | 10 +--------- opensm/opensm/osm_sa_informinfo.c | 21 ++------------------- opensm/opensm/osm_sa_lft_record.c | 10 +--------- opensm/opensm/osm_sa_link_record.c | 10 +--------- opensm/opensm/osm_sa_mcmember_record.c | 20 ++------------------ opensm/opensm/osm_sa_mft_record.c | 10 +--------- opensm/opensm/osm_sa_multipath_record.c | 10 +--------- opensm/opensm/osm_sa_node_record.c | 10 +--------- opensm/opensm/osm_sa_path_record.c | 10 +--------- opensm/opensm/osm_sa_pkey_record.c | 9 +-------- opensm/opensm/osm_sa_portinfo_record.c | 9 +-------- opensm/opensm/osm_sa_response.c | 12 ++---------- opensm/opensm/osm_sa_service_record.c | 11 +---------- opensm/opensm/osm_sa_slvl_record.c | 9 +-------- opensm/opensm/osm_sa_sminfo_record.c | 8 +------- opensm/opensm/osm_sa_sw_info_record.c | 10 +--------- opensm/opensm/osm_sa_vlarb_record.c | 9 +-------- 20 files changed, 28 insertions(+), 186 deletions(-) diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index 9409a04..bbd573c 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -365,14 +365,8 @@ static ib_api_status_t __osm_send_report(IN osm_infr_t * p_infr_rec, /* the info *p_report_ntc = *p_ntc; /* The TRUE is for: response is expected */ - status = osm_sa_vendor_send(p_report_madw->h_bind, p_report_madw, TRUE, - p_infr_rec->sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(p_log, OSM_LOG_ERROR, "ERR 0204: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_report_madw->h_bind, p_report_madw, TRUE, + p_infr_rec->sa->p_subn); Exit: OSM_LOG_EXIT(p_log); diff --git a/opensm/opensm/osm_sa.c b/opensm/opensm/osm_sa.c index 9dbab9d..498bae8 100644 --- a/opensm/opensm/osm_sa.c +++ b/opensm/opensm/osm_sa.c @@ -327,8 +327,12 @@ osm_sa_vendor_send(IN osm_bind_handle_t h_bind, cl_atomic_inc(&p_subn->p_osm->stats.sa_mads_sent); status = osm_vendor_send(h_bind, p_madw, resp_expected); - if (status != IB_SUCCESS) + if (status != IB_SUCCESS) { cl_atomic_dec(&p_subn->p_osm->stats.sa_mads_sent); + OSM_LOG(&p_subn->p_osm->log, OSM_LOG_ERROR, "ERR 4C04: " + "osm_vendor_send failed, status = %s\n", + ib_get_err_str(status)); + } return status; } diff --git a/opensm/opensm/osm_sa_class_port_info.c b/opensm/opensm/osm_sa_class_port_info.c index 744c97d..4232fce 100644 --- a/opensm/opensm/osm_sa_class_port_info.c +++ b/opensm/opensm/osm_sa_class_port_info.c @@ -80,7 +80,6 @@ __osm_cpi_rcv_respond(IN osm_sa_t * sa, const ib_sa_mad_t *p_sa_mad; ib_sa_mad_t *p_resp_sa_mad; ib_class_port_info_t *p_resp_cpi; - ib_api_status_t status; ib_gid_t zero_gid; uint8_t rtv; @@ -176,14 +175,7 @@ __osm_cpi_rcv_respond(IN osm_sa_t * sa, if (osm_log_is_active(sa->p_log, OSM_LOG_FRAMES)) osm_dump_sa_mad(sa->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1409: " - "Unable to send MAD (%s)\n", ib_get_err_str(status)); - /* osm_mad_pool_put( sa->p_mad_pool, p_resp_madw ); */ - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_guidinfo_record.c b/opensm/opensm/osm_sa_guidinfo_record.c index c6baadd..8bdc136 100644 --- a/opensm/opensm/osm_sa_guidinfo_record.c +++ b/opensm/opensm/osm_sa_guidinfo_record.c @@ -324,7 +324,6 @@ void osm_gir_rcv_process(IN void *ctx, IN void *data) uint32_t i; osm_gir_search_ctxt_t context; osm_gir_item_t *p_rec_item; - ib_api_status_t status; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -502,14 +501,7 @@ void osm_gir_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5107: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_informinfo.c b/opensm/opensm/osm_sa_informinfo.c index 286a348..79e6e54 100644 --- a/opensm/opensm/osm_sa_informinfo.c +++ b/opensm/opensm/osm_sa_informinfo.c @@ -224,7 +224,6 @@ __osm_infr_rcv_respond(IN osm_sa_t * sa, const ib_sa_mad_t *p_sa_mad; ib_sa_mad_t *p_resp_sa_mad; ib_inform_info_t *p_resp_infr; - ib_api_status_t status; OSM_LOG_ENTER(sa->p_log); @@ -257,15 +256,7 @@ __osm_infr_rcv_respond(IN osm_sa_t * sa, p_resp_infr = (ib_inform_info_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4304: " - "Unable to send MAD (%s)\n", ib_get_err_str(status)); - /* osm_mad_pool_put( sa->p_mad_pool, p_resp_madw ); */ - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); @@ -375,7 +366,6 @@ osm_infr_rcv_process_get_method(IN osm_sa_t * sa, uint32_t i, j; osm_iir_search_ctxt_t context; osm_iir_item_t *p_rec_item; - ib_api_status_t status = IB_SUCCESS; osm_physp_t *p_req_physp; OSM_LOG_ENTER(sa->p_log); @@ -551,14 +541,7 @@ osm_infr_rcv_process_get_method(IN osm_sa_t * sa, CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 430C: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_lft_record.c b/opensm/opensm/osm_sa_lft_record.c index 10c0e71..b3d118d 100644 --- a/opensm/opensm/osm_sa_lft_record.c +++ b/opensm/opensm/osm_sa_lft_record.c @@ -230,7 +230,6 @@ void osm_lftr_rcv_process(IN void *ctx, IN void *data) uint32_t i; osm_lftr_search_ctxt_t context; osm_lftr_item_t *p_rec_item; - ib_api_status_t status = IB_SUCCESS; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -406,14 +405,7 @@ void osm_lftr_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4411: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_link_record.c b/opensm/opensm/osm_sa_link_record.c index 1f88d0e..d3b0a73 100644 --- a/opensm/opensm/osm_sa_link_record.c +++ b/opensm/opensm/osm_sa_link_record.c @@ -456,7 +456,6 @@ __osm_lr_rcv_respond(IN osm_sa_t * sa, size_t trim_num_rec; #endif ib_link_record_t *p_resp_lr; - ib_api_status_t status; osm_lr_item_t *p_lr_item; const ib_sa_mad_t *p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); @@ -573,14 +572,7 @@ __osm_lr_rcv_respond(IN osm_sa_t * sa, } } - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1803: " - "Unable to send MAD (%s)\n", ib_get_err_str(status)); - /* osm_mad_pool_put( sa->p_mad_pool, p_resp_madw ); */ - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_mcmember_record.c b/opensm/opensm/osm_sa_mcmember_record.c index 63202e8..50d281f 100644 --- a/opensm/opensm/osm_sa_mcmember_record.c +++ b/opensm/opensm/osm_sa_mcmember_record.c @@ -347,7 +347,6 @@ __osm_mcmr_rcv_respond(IN osm_sa_t * sa, osm_madw_t *p_resp_madw; ib_sa_mad_t *p_sa_mad, *p_resp_sa_mad; ib_member_rec_t *p_resp_mcmember_rec; - ib_api_status_t status; OSM_LOG_ENTER(sa->p_log); @@ -395,14 +394,7 @@ __osm_mcmr_rcv_respond(IN osm_sa_t * sa, p_resp_mcmember_rec->pkt_life &= 0x3f; p_resp_mcmember_rec->pkt_life |= 2 << 6; /* exactly */ - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B07: " - "Unable to send MAD (%s) for TID <0x%" PRIx64 ">\n", - ib_get_err_str(status), p_resp_sa_mad->trans_id); - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); @@ -1861,7 +1853,6 @@ __osm_mcmr_query_mgrp(IN osm_sa_t * sa, uint32_t i; osm_sa_mcmr_search_ctxt_t context; osm_mcmr_item_t *p_rec_item; - ib_api_status_t status; ib_net64_t comp_mask; osm_physp_t *p_req_physp; boolean_t trusted_req; @@ -2038,14 +2029,7 @@ __osm_mcmr_query_mgrp(IN osm_sa_t * sa, CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B17: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_mft_record.c b/opensm/opensm/osm_sa_mft_record.c index a5d1292..c1990e6 100644 --- a/opensm/opensm/osm_sa_mft_record.c +++ b/opensm/opensm/osm_sa_mft_record.c @@ -261,7 +261,6 @@ void osm_mftr_rcv_process(IN void *ctx, IN void *data) uint32_t i; osm_mftr_search_ctxt_t context; osm_mftr_item_t *p_rec_item; - ib_api_status_t status = IB_SUCCESS; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -437,14 +436,7 @@ void osm_mftr_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4A11: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_multipath_record.c b/opensm/opensm/osm_sa_multipath_record.c index 98a2da1..7e0ff53 100644 --- a/opensm/opensm/osm_sa_multipath_record.c +++ b/opensm/opensm/osm_sa_multipath_record.c @@ -1465,7 +1465,6 @@ __osm_mpr_rcv_respond(IN osm_sa_t * sa, size_t mad_size; ib_path_rec_t *p_resp_pr; ib_multipath_rec_t *p_mpr; - ib_api_status_t status; osm_mpr_item_t *p_mpr_item; uint32_t i; @@ -1537,14 +1536,7 @@ __osm_mpr_rcv_respond(IN osm_sa_t * sa, osm_dump_sa_mad(sa->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4507: " - "Unable to send MAD (%s)\n", ib_get_err_str(status)); - /* osm_mad_pool_put( sa->p_mad_pool, p_resp_madw ); */ - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_node_record.c b/opensm/opensm/osm_sa_node_record.c index ab5e3a6..21d439f 100644 --- a/opensm/opensm/osm_sa_node_record.c +++ b/opensm/opensm/osm_sa_node_record.c @@ -326,7 +326,6 @@ void osm_nr_rcv_process(IN void *ctx, IN void *data) uint32_t i; osm_nr_search_ctxt_t context; osm_nr_item_t *p_rec_item; - ib_api_status_t status; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -493,14 +492,7 @@ void osm_nr_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1D07: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c index f94145b..a570c92 100644 --- a/opensm/opensm/osm_sa_path_record.c +++ b/opensm/opensm/osm_sa_path_record.c @@ -1658,7 +1658,6 @@ __osm_pr_rcv_respond(IN osm_sa_t * sa, size_t trim_num_rec; #endif ib_path_rec_t *p_resp_pr; - ib_api_status_t status; const ib_sa_mad_t *sad_mad = osm_madw_get_sa_mad_ptr(p_madw); osm_pr_item_t *p_pr_item; uint32_t i; @@ -1777,14 +1776,7 @@ __osm_pr_rcv_respond(IN osm_sa_t * sa, CL_ASSERT(cl_is_qlist_empty(p_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F15: " - "Unable to send MAD (%s)\n", ib_get_err_str(status)); - /* osm_mad_pool_put( sa->p_mad_pool, p_resp_madw ); */ - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_pkey_record.c b/opensm/opensm/osm_sa_pkey_record.c index adfd5c8..769d3df 100644 --- a/opensm/opensm/osm_sa_pkey_record.c +++ b/opensm/opensm/osm_sa_pkey_record.c @@ -481,14 +481,7 @@ void osm_pkey_rec_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4607: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_portinfo_record.c b/opensm/opensm/osm_sa_portinfo_record.c index 3e866f3..df46139 100644 --- a/opensm/opensm/osm_sa_portinfo_record.c +++ b/opensm/opensm/osm_sa_portinfo_record.c @@ -726,14 +726,7 @@ void osm_pir_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2107: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_response.c b/opensm/opensm/osm_sa_response.c index 7db3927..6477aca 100644 --- a/opensm/opensm/osm_sa_response.c +++ b/opensm/opensm/osm_sa_response.c @@ -67,7 +67,6 @@ osm_sa_send_error(IN osm_sa_t * sa, osm_madw_t *p_resp_madw; ib_sa_mad_t *p_resp_sa_mad; ib_sa_mad_t *p_sa_mad; - ib_api_status_t status; OSM_LOG_ENTER(sa->p_log); @@ -115,15 +114,8 @@ osm_sa_send_error(IN osm_sa_t * sa, if (osm_log_is_active(sa->p_log, OSM_LOG_FRAMES)) osm_dump_sa_mad(sa->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); - status = osm_sa_vendor_send(osm_madw_get_bind_handle(p_resp_madw), - p_resp_madw, FALSE, sa->p_subn); - - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2302: " - "Error sending MAD (%s)\n", ib_get_err_str(status)); - /* osm_mad_pool_put( sa->p_mad_pool, p_resp_madw ); */ - goto Exit; - } + osm_sa_vendor_send(osm_madw_get_bind_handle(p_resp_madw), + p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_service_record.c b/opensm/opensm/osm_sa_service_record.c index 1680ea1..434ccec 100644 --- a/opensm/opensm/osm_sa_service_record.c +++ b/opensm/opensm/osm_sa_service_record.c @@ -222,7 +222,6 @@ __osm_sr_rcv_respond(IN osm_sa_t * sa, uint32_t trim_num_rec; #endif ib_service_record_t *p_resp_sr; - ib_api_status_t status; osm_sr_item_t *p_sr_item; const ib_sa_mad_t *p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); boolean_t trusted_req = TRUE; @@ -362,15 +361,7 @@ __osm_sr_rcv_respond(IN osm_sa_t * sa, } } - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2407: " - "Unable to send MAD (%s)\n", ib_get_err_str(status)); - /* osm_mad_pool_put( sa->p_mad_pool, p_resp_madw ); */ - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_slvl_record.c b/opensm/opensm/osm_sa_slvl_record.c index 9b587a5..9247fa6 100644 --- a/opensm/opensm/osm_sa_slvl_record.c +++ b/opensm/opensm/osm_sa_slvl_record.c @@ -455,14 +455,7 @@ void osm_slvl_rec_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2606: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_sminfo_record.c b/opensm/opensm/osm_sa_sminfo_record.c index 5527892..0127077 100644 --- a/opensm/opensm/osm_sa_sminfo_record.c +++ b/opensm/opensm/osm_sa_sminfo_record.c @@ -476,13 +476,7 @@ void osm_smir_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2802: " - "Error sending MAD (%s)\n", ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_sw_info_record.c b/opensm/opensm/osm_sa_sw_info_record.c index 79f7a19..3ed37ad 100644 --- a/opensm/opensm/osm_sa_sw_info_record.c +++ b/opensm/opensm/osm_sa_sw_info_record.c @@ -256,7 +256,6 @@ void osm_sir_rcv_process(IN void *ctx, IN void *data) uint32_t i; osm_sir_search_ctxt_t context; osm_sir_item_t *p_rec_item; - ib_api_status_t status; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -428,14 +427,7 @@ void osm_sir_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5307: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_vlarb_record.c b/opensm/opensm/osm_sa_vlarb_record.c index 7b022aa..29bbba2 100644 --- a/opensm/opensm/osm_sa_vlarb_record.c +++ b/opensm/opensm/osm_sa_vlarb_record.c @@ -472,14 +472,7 @@ void osm_vlarb_rec_rcv_process(IN void *ctx, IN void *data) CL_ASSERT(cl_is_qlist_empty(&rec_list)); - status = osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, - sa->p_subn); - if (status != IB_SUCCESS) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2A07: " - "osm_sa_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } + osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); Exit: OSM_LOG_EXIT(sa->p_log); -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Sat Mar 1 13:05:39 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 1 Mar 2008 21:05:39 +0000 Subject: [ofa-general] [PATCH] opensm: move osm_sa_send_error() to osm_sa.c file In-Reply-To: <20080301210509.GC11788@sashak.voltaire.com> References: <20080301210509.GC11788@sashak.voltaire.com> Message-ID: <20080301210539.GD11788@sashak.voltaire.com> Move osm_sa_send_error() tp osm_sa.c file. Remove empty osm_sa_response.c file. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/Makefile.am | 2 +- opensm/opensm/osm_sa.c | 63 ++++++++++++++++++++ opensm/opensm/osm_sa_response.c | 122 --------------------------------------- 3 files changed, 64 insertions(+), 123 deletions(-) delete mode 100644 opensm/opensm/osm_sa_response.c diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 50fdae1..d9ebe53 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -46,7 +46,7 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_sa_mcmember_record.c osm_sa_node_record.c \ osm_sa_path_record.c osm_sa_pkey_record.c \ osm_sa_portinfo_record.c osm_sa_guidinfo_record.c \ - osm_sa_multipath_record.c osm_sa_response.c \ + osm_sa_multipath_record.c \ osm_sa_service_record.c osm_sa_slvl_record.c \ osm_sa_sminfo_record.c osm_sa_vlarb_record.c \ osm_sa_sw_info_record.c osm_service.c \ diff --git a/opensm/opensm/osm_sa.c b/opensm/opensm/osm_sa.c index 498bae8..c557876 100644 --- a/opensm/opensm/osm_sa.c +++ b/opensm/opensm/osm_sa.c @@ -68,6 +68,7 @@ #include #include #include +#include #include #define OSM_SA_INITIAL_TID_VALUE 0xabc @@ -336,6 +337,68 @@ osm_sa_vendor_send(IN osm_bind_handle_t h_bind, return status; } +void +osm_sa_send_error(IN osm_sa_t * sa, + IN const osm_madw_t * const p_madw, + IN const ib_net16_t sa_status) +{ + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + ib_sa_mad_t *p_sa_mad; + + OSM_LOG_ENTER(sa->p_log); + + /* avoid races - if we are exiting - exit */ + if (osm_exit_flag) { + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Ignoring requested send after exit\n"); + goto Exit; + } + + p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, + p_madw->h_bind, MAD_BLOCK_SIZE, + &p_madw->mad_addr); + + if (p_resp_madw == NULL) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2301: " + "Unable to acquire response MAD\n"); + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + + /* Copy the MAD header back into the response mad */ + *p_resp_sa_mad = *p_sa_mad; + p_resp_sa_mad->status = sa_status; + + if (p_resp_sa_mad->method == IB_MAD_METHOD_SET) + p_resp_sa_mad->method = IB_MAD_METHOD_GET; + + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + + /* + * C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) + */ + p_resp_sa_mad->sm_key = 0; + + /* + * o15-0.2.7 - The PathRecord Attribute ID shall be used in + * the response (to a SubnAdmGetMulti(MultiPathRecord) + */ + if (p_resp_sa_mad->attr_id == IB_MAD_ATTR_MULTIPATH_RECORD) + p_resp_sa_mad->attr_id = IB_MAD_ATTR_PATH_RECORD; + + if (osm_log_is_active(sa->p_log, OSM_LOG_FRAMES)) + osm_dump_sa_mad(sa->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); + + osm_sa_vendor_send(osm_madw_get_bind_handle(p_resp_madw), + p_resp_madw, FALSE, sa->p_subn); + +Exit: + OSM_LOG_EXIT(sa->p_log); +} + /********************************************************************** **********************************************************************/ /* diff --git a/opensm/opensm/osm_sa_response.c b/opensm/opensm/osm_sa_response.c deleted file mode 100644 index 6477aca..0000000 --- a/opensm/opensm/osm_sa_response.c +++ /dev/null @@ -1,122 +0,0 @@ -/* - * Copyright (c) 2004-2007 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. - * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. - * - * This software is available to you under a choice of one of two - * licenses. You may choose to be licensed under the terms of the GNU - * General Public License (GPL) Version 2, available from the file - * COPYING in the main directory of this source tree, or the - * OpenIB.org BSD license below: - * - * Redistribution and use in source and binary forms, with or - * without modification, are permitted provided that the following - * conditions are met: - * - * - Redistributions of source code must retain the above - * copyright notice, this list of conditions and the following - * disclaimer. - * - * - Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following - * disclaimer in the documentation and/or other materials - * provided with the distribution. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND - * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS - * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN - * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN - * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE - * SOFTWARE. - * - */ - -/* - * Abstract: - * Implementation of osm_sa_resp_t. - * This object represents the SA query responder. - * This object is part of the opensm family of objects. - * - * Environment: - * Linux User Mode - * - * $Revision: 1.6 $ - */ - -#if HAVE_CONFIG_H -# include -#endif /* HAVE_CONFIG_H */ - -#include -#include -#include -#include -#include -#include -#include - -/********************************************************************** - **********************************************************************/ -void -osm_sa_send_error(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw, - IN const ib_net16_t sa_status) -{ - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - ib_sa_mad_t *p_sa_mad; - - OSM_LOG_ENTER(sa->p_log); - - /* avoid races - if we are exiting - exit */ - if (osm_exit_flag) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Ignoring requested send after exit\n"); - goto Exit; - } - - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, - p_madw->h_bind, MAD_BLOCK_SIZE, - &p_madw->mad_addr); - - if (p_resp_madw == NULL) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2301: " - "Unable to acquire response MAD\n"); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - - /* Copy the MAD header back into the response mad */ - *p_resp_sa_mad = *p_sa_mad; - p_resp_sa_mad->status = sa_status; - - if (p_resp_sa_mad->method == IB_MAD_METHOD_SET) - p_resp_sa_mad->method = IB_MAD_METHOD_GET; - - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - - /* - * C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) - */ - p_resp_sa_mad->sm_key = 0; - - /* - * o15-0.2.7 - The PathRecord Attribute ID shall be used in - * the response (to a SubnAdmGetMulti(MultiPathRecord) - */ - if (p_resp_sa_mad->attr_id == IB_MAD_ATTR_MULTIPATH_RECORD) - p_resp_sa_mad->attr_id = IB_MAD_ATTR_PATH_RECORD; - - if (osm_log_is_active(sa->p_log, OSM_LOG_FRAMES)) - osm_dump_sa_mad(sa->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); - - osm_sa_vendor_send(osm_madw_get_bind_handle(p_resp_madw), - p_resp_madw, FALSE, sa->p_subn); - -Exit: - OSM_LOG_EXIT(sa->p_log); -} -- 1.5.4.1.122.gaa8d From rdreier at cisco.com Sat Mar 1 13:40:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sat, 01 Mar 2008 13:40:05 -0800 Subject: [ofa-general] [RFC PATCH 15/14] RDMA/amso1100: Start of endianness annotation References: <20082292026.yXkK6fEAjkkUN7jE@cisco.com> Message-ID: Signed-off-by: Roland Dreier --- A bonus patch... cuts down on the sparse noise considerably, although there are still more related to the mqsp stuff, which I haven't understood sufficiently to annotate properly. drivers/infiniband/hw/amso1100/c2.c | 70 ++++++----- drivers/infiniband/hw/amso1100/c2.h | 10 +- drivers/infiniband/hw/amso1100/c2_mm.c | 2 +- drivers/infiniband/hw/amso1100/c2_mq.c | 4 +- drivers/infiniband/hw/amso1100/c2_mq.h | 2 +- drivers/infiniband/hw/amso1100/c2_rnic.c | 24 ++-- drivers/infiniband/hw/amso1100/c2_wr.h | 214 +++++++++++++++--------------- 7 files changed, 166 insertions(+), 160 deletions(-) diff --git a/drivers/infiniband/hw/amso1100/c2.c b/drivers/infiniband/hw/amso1100/c2.c index c50533b..113f3c0 100644 --- a/drivers/infiniband/hw/amso1100/c2.c +++ b/drivers/infiniband/hw/amso1100/c2.c @@ -130,10 +130,10 @@ static int c2_tx_ring_alloc(struct c2_ring *tx_ring, void *vaddr, tx_desc->status = 0; /* Set TXP_HTXD_UNINIT */ - __raw_writeq(cpu_to_be64(0x1122334455667788ULL), + __raw_writeq((__force u64) cpu_to_be64(0x1122334455667788ULL), (void __iomem *) txp_desc + C2_TXP_ADDR); __raw_writew(0, (void __iomem *) txp_desc + C2_TXP_LEN); - __raw_writew(cpu_to_be16(TXP_HTXD_UNINIT), + __raw_writew((__force u16) cpu_to_be16(TXP_HTXD_UNINIT), (void __iomem *) txp_desc + C2_TXP_FLAGS); elem->skb = NULL; @@ -179,13 +179,13 @@ static int c2_rx_ring_alloc(struct c2_ring *rx_ring, void *vaddr, rx_desc->status = 0; /* Set RXP_HRXD_UNINIT */ - __raw_writew(cpu_to_be16(RXP_HRXD_OK), + __raw_writew((__force u16) cpu_to_be16(RXP_HRXD_OK), (void __iomem *) rxp_desc + C2_RXP_STATUS); __raw_writew(0, (void __iomem *) rxp_desc + C2_RXP_COUNT); __raw_writew(0, (void __iomem *) rxp_desc + C2_RXP_LEN); - __raw_writeq(cpu_to_be64(0x99aabbccddeeffULL), + __raw_writeq((__force u64) cpu_to_be64(0x99aabbccddeeffULL), (void __iomem *) rxp_desc + C2_RXP_ADDR); - __raw_writew(cpu_to_be16(RXP_HRXD_UNINIT), + __raw_writew((__force u16) cpu_to_be16(RXP_HRXD_UNINIT), (void __iomem *) rxp_desc + C2_RXP_FLAGS); elem->skb = NULL; @@ -239,10 +239,11 @@ static inline int c2_rx_alloc(struct c2_port *c2_port, struct c2_element *elem) rxp_hdr->flags = RXP_HRXD_READY; __raw_writew(0, elem->hw_desc + C2_RXP_STATUS); - __raw_writew(cpu_to_be16((u16) maplen - sizeof(*rxp_hdr)), + __raw_writew((__force u16) cpu_to_be16((u16) maplen - sizeof(*rxp_hdr)), elem->hw_desc + C2_RXP_LEN); - __raw_writeq(cpu_to_be64(mapaddr), elem->hw_desc + C2_RXP_ADDR); - __raw_writew(cpu_to_be16(RXP_HRXD_READY), elem->hw_desc + C2_RXP_FLAGS); + __raw_writeq((__force u64) cpu_to_be64(mapaddr), elem->hw_desc + C2_RXP_ADDR); + __raw_writew((__force u16) cpu_to_be16(RXP_HRXD_READY), + elem->hw_desc + C2_RXP_FLAGS); elem->skb = skb; elem->mapaddr = mapaddr; @@ -290,9 +291,9 @@ static void c2_rx_clean(struct c2_port *c2_port) __raw_writew(0, elem->hw_desc + C2_RXP_STATUS); __raw_writew(0, elem->hw_desc + C2_RXP_COUNT); __raw_writew(0, elem->hw_desc + C2_RXP_LEN); - __raw_writeq(cpu_to_be64(0x99aabbccddeeffULL), + __raw_writeq((__force u64) cpu_to_be64(0x99aabbccddeeffULL), elem->hw_desc + C2_RXP_ADDR); - __raw_writew(cpu_to_be16(RXP_HRXD_UNINIT), + __raw_writew((__force u16) cpu_to_be16(RXP_HRXD_UNINIT), elem->hw_desc + C2_RXP_FLAGS); if (elem->skb) { @@ -346,16 +347,16 @@ static void c2_tx_clean(struct c2_port *c2_port) elem->hw_desc + C2_TXP_LEN); __raw_writeq(0, elem->hw_desc + C2_TXP_ADDR); - __raw_writew(cpu_to_be16(TXP_HTXD_DONE), + __raw_writew((__force u16) cpu_to_be16(TXP_HTXD_DONE), elem->hw_desc + C2_TXP_FLAGS); c2_port->netstats.tx_dropped++; break; } else { __raw_writew(0, elem->hw_desc + C2_TXP_LEN); - __raw_writeq(cpu_to_be64(0x1122334455667788ULL), + __raw_writeq((__force u64) cpu_to_be64(0x1122334455667788ULL), elem->hw_desc + C2_TXP_ADDR); - __raw_writew(cpu_to_be16(TXP_HTXD_UNINIT), + __raw_writew((__force u16) cpu_to_be16(TXP_HTXD_UNINIT), elem->hw_desc + C2_TXP_FLAGS); } @@ -390,7 +391,7 @@ static void c2_tx_interrupt(struct net_device *netdev) for (elem = tx_ring->to_clean; elem != tx_ring->to_use; elem = elem->next) { txp_htxd.flags = - be16_to_cpu(readw(elem->hw_desc + C2_TXP_FLAGS)); + be16_to_cpu((__force __be16) readw(elem->hw_desc + C2_TXP_FLAGS)); if (txp_htxd.flags != TXP_HTXD_DONE) break; @@ -398,7 +399,7 @@ static void c2_tx_interrupt(struct net_device *netdev) if (netif_msg_tx_done(c2_port)) { /* PCI reads are expensive in fast path */ txp_htxd.len = - be16_to_cpu(readw(elem->hw_desc + C2_TXP_LEN)); + be16_to_cpu((__force __be16) readw(elem->hw_desc + C2_TXP_LEN)); pr_debug("%s: tx done slot %3Zu status 0x%x len " "%5u bytes\n", netdev->name, elem - tx_ring->start, @@ -448,10 +449,12 @@ static void c2_rx_error(struct c2_port *c2_port, struct c2_element *elem) /* Write the descriptor to the adapter's rx ring */ __raw_writew(0, elem->hw_desc + C2_RXP_STATUS); __raw_writew(0, elem->hw_desc + C2_RXP_COUNT); - __raw_writew(cpu_to_be16((u16) elem->maplen - sizeof(*rxp_hdr)), + __raw_writew((__force u16) cpu_to_be16((u16) elem->maplen - sizeof(*rxp_hdr)), elem->hw_desc + C2_RXP_LEN); - __raw_writeq(cpu_to_be64(elem->mapaddr), elem->hw_desc + C2_RXP_ADDR); - __raw_writew(cpu_to_be16(RXP_HRXD_READY), elem->hw_desc + C2_RXP_FLAGS); + __raw_writeq((__force u64) cpu_to_be64(elem->mapaddr), + elem->hw_desc + C2_RXP_ADDR); + __raw_writew((__force u16) cpu_to_be16(RXP_HRXD_READY), + elem->hw_desc + C2_RXP_FLAGS); pr_debug("packet dropped\n"); c2_port->netstats.rx_dropped++; @@ -653,7 +656,7 @@ static int c2_up(struct net_device *netdev) i++, elem++) { rxp_hdr = (struct c2_rxp_hdr *) elem->skb->data; rxp_hdr->flags = 0; - __raw_writew(cpu_to_be16(RXP_HRXD_READY), + __raw_writew((__force u16) cpu_to_be16(RXP_HRXD_READY), elem->hw_desc + C2_RXP_FLAGS); } @@ -787,9 +790,12 @@ static int c2_xmit_frame(struct sk_buff *skb, struct net_device *netdev) elem->maplen = maplen; /* Tell HW to xmit */ - __raw_writeq(cpu_to_be64(mapaddr), elem->hw_desc + C2_TXP_ADDR); - __raw_writew(cpu_to_be16(maplen), elem->hw_desc + C2_TXP_LEN); - __raw_writew(cpu_to_be16(TXP_HTXD_READY), elem->hw_desc + C2_TXP_FLAGS); + __raw_writeq((__force u64) cpu_to_be64(mapaddr), + elem->hw_desc + C2_TXP_ADDR); + __raw_writew((__force u16) cpu_to_be16(maplen), + elem->hw_desc + C2_TXP_LEN); + __raw_writew((__force u16) cpu_to_be16(TXP_HTXD_READY), + elem->hw_desc + C2_TXP_FLAGS); c2_port->netstats.tx_packets++; c2_port->netstats.tx_bytes += maplen; @@ -810,11 +816,11 @@ static int c2_xmit_frame(struct sk_buff *skb, struct net_device *netdev) elem->maplen = maplen; /* Tell HW to xmit */ - __raw_writeq(cpu_to_be64(mapaddr), + __raw_writeq((__force u64) cpu_to_be64(mapaddr), elem->hw_desc + C2_TXP_ADDR); - __raw_writew(cpu_to_be16(maplen), + __raw_writew((__force u16) cpu_to_be16(maplen), elem->hw_desc + C2_TXP_LEN); - __raw_writew(cpu_to_be16(TXP_HTXD_READY), + __raw_writew((__force u16) cpu_to_be16(TXP_HTXD_READY), elem->hw_desc + C2_TXP_FLAGS); c2_port->netstats.tx_packets++; @@ -1029,10 +1035,10 @@ static int __devinit c2_probe(struct pci_dev *pcidev, } /* Validate the adapter version */ - if (be32_to_cpu(readl(mmio_regs + C2_REGS_VERS)) != C2_VERSION) { + if (be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_VERS)) != C2_VERSION) { printk(KERN_ERR PFX "Version mismatch " "[fw=%u, c2=%u], Adapter not claimed\n", - be32_to_cpu(readl(mmio_regs + C2_REGS_VERS)), + be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_VERS)), C2_VERSION); ret = -EINVAL; iounmap(mmio_regs); @@ -1040,12 +1046,12 @@ static int __devinit c2_probe(struct pci_dev *pcidev, } /* Validate the adapter IVN */ - if (be32_to_cpu(readl(mmio_regs + C2_REGS_IVN)) != C2_IVN) { + if (be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_IVN)) != C2_IVN) { printk(KERN_ERR PFX "Downlevel FIrmware level. You should be using " "the OpenIB device support kit. " "[fw=0x%x, c2=0x%x], Adapter not claimed\n", - be32_to_cpu(readl(mmio_regs + C2_REGS_IVN)), - C2_IVN); + be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_IVN)), + C2_IVN); ret = -EINVAL; iounmap(mmio_regs); goto bail2; @@ -1068,7 +1074,7 @@ static int __devinit c2_probe(struct pci_dev *pcidev, /* Get the last RX index */ c2dev->cur_rx = - (be32_to_cpu(readl(mmio_regs + C2_REGS_HRX_CUR)) - + (be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_HRX_CUR)) - 0xffffc000) / sizeof(struct c2_rxp_desc); /* Request an interrupt line for the driver */ @@ -1090,7 +1096,7 @@ static int __devinit c2_probe(struct pci_dev *pcidev, } /* Save off the actual size prior to unmapping mmio_regs */ - kva_map_size = be32_to_cpu(readl(mmio_regs + C2_REGS_PCI_WINSIZE)); + kva_map_size = be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_PCI_WINSIZE)); /* Unmap the adapter PCI registers in BAR4 */ iounmap(mmio_regs); diff --git a/drivers/infiniband/hw/amso1100/c2.h b/drivers/infiniband/hw/amso1100/c2.h index fa58200..21bcb3a 100644 --- a/drivers/infiniband/hw/amso1100/c2.h +++ b/drivers/infiniband/hw/amso1100/c2.h @@ -346,7 +346,7 @@ struct c2_dev { // spinlock_t aeq_lock; // spinlock_t rnic_lock; - u16 *hint_count; + __be16 *hint_count; dma_addr_t hint_count_dma; u16 hints_read; @@ -425,10 +425,10 @@ static inline void __raw_writeq(u64 val, void __iomem * addr) #endif #define C2_SET_CUR_RX(c2dev, cur_rx) \ - __raw_writel(cpu_to_be32(cur_rx), c2dev->mmio_txp_ring + 4092) + __raw_writel((__force u32) cpu_to_be32(cur_rx), c2dev->mmio_txp_ring + 4092) #define C2_GET_CUR_RX(c2dev) \ - be32_to_cpu(readl(c2dev->mmio_txp_ring + 4092)) + be32_to_cpu((__force __be32) readl(c2dev->mmio_txp_ring + 4092)) static inline struct c2_dev *to_c2dev(struct ib_device *ibdev) { @@ -485,8 +485,8 @@ extern void c2_unregister_device(struct c2_dev *c2dev); extern int c2_rnic_init(struct c2_dev *c2dev); extern void c2_rnic_term(struct c2_dev *c2dev); extern void c2_rnic_interrupt(struct c2_dev *c2dev); -extern int c2_del_addr(struct c2_dev *c2dev, u32 inaddr, u32 inmask); -extern int c2_add_addr(struct c2_dev *c2dev, u32 inaddr, u32 inmask); +extern int c2_del_addr(struct c2_dev *c2dev, __be32 inaddr, __be32 inmask); +extern int c2_add_addr(struct c2_dev *c2dev, __be32 inaddr, __be32 inmask); /* QPs */ extern int c2_alloc_qp(struct c2_dev *c2dev, struct c2_pd *pd, diff --git a/drivers/infiniband/hw/amso1100/c2_mm.c b/drivers/infiniband/hw/amso1100/c2_mm.c index 1e4f464..b506fe2 100644 --- a/drivers/infiniband/hw/amso1100/c2_mm.c +++ b/drivers/infiniband/hw/amso1100/c2_mm.c @@ -45,7 +45,7 @@ * Reply buffer _is_ freed by this function. */ static int -send_pbl_messages(struct c2_dev *c2dev, u32 stag_index, +send_pbl_messages(struct c2_dev *c2dev, __be32 stag_index, unsigned long va, u32 pbl_depth, struct c2_vq_req *vq_req, int pbl_type) { diff --git a/drivers/infiniband/hw/amso1100/c2_mq.c b/drivers/infiniband/hw/amso1100/c2_mq.c index b88a755..0cddc49 100644 --- a/drivers/infiniband/hw/amso1100/c2_mq.c +++ b/drivers/infiniband/hw/amso1100/c2_mq.c @@ -64,7 +64,7 @@ void c2_mq_produce(struct c2_mq *q) q->priv = (q->priv + 1) % q->q_size; q->hint_count++; /* Update peer's offset. */ - __raw_writew(cpu_to_be16(q->priv), &q->peer->shared); + __raw_writew((__force u16) cpu_to_be16(q->priv), &q->peer->shared); } } @@ -105,7 +105,7 @@ void c2_mq_free(struct c2_mq *q) #endif q->priv = (q->priv + 1) % q->q_size; /* Update peer's offset. */ - __raw_writew(cpu_to_be16(q->priv), &q->peer->shared); + __raw_writew((__force u16) cpu_to_be16(q->priv), &q->peer->shared); } } diff --git a/drivers/infiniband/hw/amso1100/c2_mq.h b/drivers/infiniband/hw/amso1100/c2_mq.h index 9185bbb..acede00 100644 --- a/drivers/infiniband/hw/amso1100/c2_mq.h +++ b/drivers/infiniband/hw/amso1100/c2_mq.h @@ -75,7 +75,7 @@ struct c2_mq { u16 hint_count; u16 priv; struct c2_mq_shared __iomem *peer; - u16 *shared; + __be16 *shared; dma_addr_t shared_dma; u32 q_size; u32 msg_size; diff --git a/drivers/infiniband/hw/amso1100/c2_rnic.c b/drivers/infiniband/hw/amso1100/c2_rnic.c index 1687c51..7be1f87 100644 --- a/drivers/infiniband/hw/amso1100/c2_rnic.c +++ b/drivers/infiniband/hw/amso1100/c2_rnic.c @@ -208,7 +208,7 @@ static int c2_rnic_query(struct c2_dev *c2dev, struct ib_device_attr *props) /* * Add an IP address to the RNIC interface */ -int c2_add_addr(struct c2_dev *c2dev, u32 inaddr, u32 inmask) +int c2_add_addr(struct c2_dev *c2dev, __be32 inaddr, __be32 inmask) { struct c2_vq_req *vq_req; struct c2wr_rnic_setconfig_req *wr; @@ -270,7 +270,7 @@ int c2_add_addr(struct c2_dev *c2dev, u32 inaddr, u32 inmask) /* * Delete an IP address from the RNIC interface */ -int c2_del_addr(struct c2_dev *c2dev, u32 inaddr, u32 inmask) +int c2_del_addr(struct c2_dev *c2dev, __be32 inaddr, __be32 inmask) { struct c2_vq_req *vq_req; struct c2wr_rnic_setconfig_req *wr; @@ -506,17 +506,17 @@ int __devinit c2_rnic_init(struct c2_dev *c2dev) mmio_regs = c2dev->kva; /* Initialize the Verbs Request Queue */ c2_mq_req_init(&c2dev->req_vq, 0, - be32_to_cpu(readl(mmio_regs + C2_REGS_Q0_QSIZE)), - be32_to_cpu(readl(mmio_regs + C2_REGS_Q0_MSGSIZE)), + be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q0_QSIZE)), + be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q0_MSGSIZE)), mmio_regs + - be32_to_cpu(readl(mmio_regs + C2_REGS_Q0_POOLSTART)), + be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q0_POOLSTART)), mmio_regs + - be32_to_cpu(readl(mmio_regs + C2_REGS_Q0_SHARED)), + be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q0_SHARED)), C2_MQ_ADAPTER_TARGET); /* Initialize the Verbs Reply Queue */ - qsize = be32_to_cpu(readl(mmio_regs + C2_REGS_Q1_QSIZE)); - msgsize = be32_to_cpu(readl(mmio_regs + C2_REGS_Q1_MSGSIZE)); + qsize = be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q1_QSIZE)); + msgsize = be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q1_MSGSIZE)); q1_pages = dma_alloc_coherent(&c2dev->pcidev->dev, qsize * msgsize, &c2dev->rep_vq.host_dma, GFP_KERNEL); if (!q1_pages) { @@ -532,12 +532,12 @@ int __devinit c2_rnic_init(struct c2_dev *c2dev) msgsize, q1_pages, mmio_regs + - be32_to_cpu(readl(mmio_regs + C2_REGS_Q1_SHARED)), + be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q1_SHARED)), C2_MQ_HOST_TARGET); /* Initialize the Asynchronus Event Queue */ - qsize = be32_to_cpu(readl(mmio_regs + C2_REGS_Q2_QSIZE)); - msgsize = be32_to_cpu(readl(mmio_regs + C2_REGS_Q2_MSGSIZE)); + qsize = be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q2_QSIZE)); + msgsize = be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q2_MSGSIZE)); q2_pages = dma_alloc_coherent(&c2dev->pcidev->dev, qsize * msgsize, &c2dev->aeq.host_dma, GFP_KERNEL); if (!q2_pages) { @@ -553,7 +553,7 @@ int __devinit c2_rnic_init(struct c2_dev *c2dev) msgsize, q2_pages, mmio_regs + - be32_to_cpu(readl(mmio_regs + C2_REGS_Q2_SHARED)), + be32_to_cpu((__force __be32) readl(mmio_regs + C2_REGS_Q2_SHARED)), C2_MQ_HOST_TARGET); /* Initialize the verbs request allocator */ diff --git a/drivers/infiniband/hw/amso1100/c2_wr.h b/drivers/infiniband/hw/amso1100/c2_wr.h index 3ec6c43..186a5ea 100644 --- a/drivers/infiniband/hw/amso1100/c2_wr.h +++ b/drivers/infiniband/hw/amso1100/c2_wr.h @@ -180,8 +180,8 @@ enum c2_wr_type { }; struct c2_netaddr { - u32 ip_addr; - u32 netmask; + __be32 ip_addr; + __be32 netmask; u32 mtu; }; @@ -199,9 +199,9 @@ struct c2_route { * A Scatter Gather Entry. */ struct c2_data_addr { - u32 stag; - u32 length; - u64 to; + __be32 stag; + __be32 length; + __be64 to; }; /* @@ -274,7 +274,7 @@ struct c2wr_hdr { * from the host to adapter by libccil, but we copy it anyway * to make the memcpy to the adapter better aligned. */ - u32 wqe_count; + __be32 wqe_count; /* Put these fields next so that later 32- and 64-bit * quantities are naturally aligned. @@ -316,8 +316,8 @@ enum c2_rnic_flags { struct c2wr_rnic_open_req { struct c2wr_hdr hdr; u64 user_context; - u16 flags; /* See enum c2_rnic_flags */ - u16 port_num; + __be16 flags; /* See enum c2_rnic_flags */ + __be16 port_num; } __attribute__((packed)); struct c2wr_rnic_open_rep { @@ -341,30 +341,30 @@ struct c2wr_rnic_query_req { struct c2wr_rnic_query_rep { struct c2wr_hdr hdr; u64 user_context; - u32 vendor_id; - u32 part_number; - u32 hw_version; - u32 fw_ver_major; - u32 fw_ver_minor; - u32 fw_ver_patch; + __be32 vendor_id; + __be32 part_number; + __be32 hw_version; + __be32 fw_ver_major; + __be32 fw_ver_minor; + __be32 fw_ver_patch; char fw_ver_build_str[WR_BUILD_STR_LEN]; - u32 max_qps; - u32 max_qp_depth; + __be32 max_qps; + __be32 max_qp_depth; u32 max_srq_depth; u32 max_send_sgl_depth; u32 max_rdma_sgl_depth; - u32 max_cqs; - u32 max_cq_depth; + __be32 max_cqs; + __be32 max_cq_depth; u32 max_cq_event_handlers; - u32 max_mrs; + __be32 max_mrs; u32 max_pbl_depth; - u32 max_pds; - u32 max_global_ird; + __be32 max_pds; + __be32 max_global_ird; u32 max_global_ord; - u32 max_qp_ird; - u32 max_qp_ord; + __be32 max_qp_ird; + __be32 max_qp_ord; u32 flags; - u32 max_mws; + __be32 max_mws; u32 pbe_range_low; u32 pbe_range_high; u32 max_srqs; @@ -405,7 +405,7 @@ union c2wr_rnic_getconfig { struct c2wr_rnic_setconfig_req { struct c2wr_hdr hdr; u32 rnic_handle; - u32 option; /* See c2_setconfig_cmd_t */ + __be32 option; /* See c2_setconfig_cmd_t */ /* variable data and pad. See c2_netaddr and c2_route */ u8 data[0]; } __attribute__((packed)) ; @@ -441,18 +441,18 @@ union c2wr_rnic_close { */ struct c2wr_cq_create_req { struct c2wr_hdr hdr; - u64 shared_ht; + __be64 shared_ht; u64 user_context; - u64 msg_pool; + __be64 msg_pool; u32 rnic_handle; - u32 msg_size; - u32 depth; + __be32 msg_size; + __be32 depth; } __attribute__((packed)) ; struct c2wr_cq_create_rep { struct c2wr_hdr hdr; - u32 mq_index; - u32 adapter_shared; + __be32 mq_index; + __be32 adapter_shared; u32 cq_handle; } __attribute__((packed)) ; @@ -585,40 +585,40 @@ enum c2wr_qp_flags { struct c2wr_qp_create_req { struct c2wr_hdr hdr; - u64 shared_sq_ht; - u64 shared_rq_ht; + __be64 shared_sq_ht; + __be64 shared_rq_ht; u64 user_context; u32 rnic_handle; u32 sq_cq_handle; u32 rq_cq_handle; - u32 sq_depth; - u32 rq_depth; + __be32 sq_depth; + __be32 rq_depth; u32 srq_handle; u32 srq_limit; - u32 flags; /* see enum c2wr_qp_flags */ - u32 send_sgl_depth; - u32 recv_sgl_depth; - u32 rdma_write_sgl_depth; - u32 ord; - u32 ird; + __be32 flags; /* see enum c2wr_qp_flags */ + __be32 send_sgl_depth; + __be32 recv_sgl_depth; + __be32 rdma_write_sgl_depth; + __be32 ord; + __be32 ird; u32 pd_id; } __attribute__((packed)) ; struct c2wr_qp_create_rep { struct c2wr_hdr hdr; - u32 sq_depth; - u32 rq_depth; + __be32 sq_depth; + __be32 rq_depth; u32 send_sgl_depth; u32 recv_sgl_depth; u32 rdma_write_sgl_depth; u32 ord; u32 ird; - u32 sq_msg_size; - u32 sq_mq_index; - u32 sq_mq_start; - u32 rq_msg_size; - u32 rq_mq_index; - u32 rq_mq_start; + __be32 sq_msg_size; + __be32 sq_mq_index; + __be32 sq_mq_start; + __be32 rq_msg_size; + __be32 rq_mq_index; + __be32 rq_mq_start; u32 qp_handle; } __attribute__((packed)) ; @@ -667,11 +667,11 @@ struct c2wr_qp_modify_req { u32 stream_msg_length; u32 rnic_handle; u32 qp_handle; - u32 next_qp_state; - u32 ord; - u32 ird; - u32 sq_depth; - u32 rq_depth; + __be32 next_qp_state; + __be32 ord; + __be32 ird; + __be32 sq_depth; + __be32 rq_depth; u32 llp_ep_handle; } __attribute__((packed)) ; @@ -721,10 +721,10 @@ struct c2wr_qp_connect_req { struct c2wr_hdr hdr; u32 rnic_handle; u32 qp_handle; - u32 remote_addr; - u16 remote_port; + __be32 remote_addr; + __be16 remote_port; u16 pad; - u32 private_data_length; + __be32 private_data_length; u8 private_data[0]; /* Private data in-line. */ } __attribute__((packed)) ; @@ -759,25 +759,25 @@ union c2wr_nsmr_stag_alloc { struct c2wr_nsmr_register_req { struct c2wr_hdr hdr; - u64 va; + __be64 va; u32 rnic_handle; - u16 flags; + __be16 flags; u8 stag_key; u8 pad; u32 pd_id; - u32 pbl_depth; - u32 pbe_size; - u32 fbo; - u32 length; - u32 addrs_length; + __be32 pbl_depth; + __be32 pbe_size; + __be32 fbo; + __be32 length; + __be32 addrs_length; /* array of paddrs (must be aligned on a 64bit boundary) */ - u64 paddrs[0]; + __be64 paddrs[0]; } __attribute__((packed)) ; struct c2wr_nsmr_register_rep { struct c2wr_hdr hdr; u32 pbl_depth; - u32 stag_index; + __be32 stag_index; } __attribute__((packed)) ; union c2wr_nsmr_register { @@ -788,11 +788,11 @@ union c2wr_nsmr_register { struct c2wr_nsmr_pbl_req { struct c2wr_hdr hdr; u32 rnic_handle; - u32 flags; - u32 stag_index; - u32 addrs_length; + __be32 flags; + __be32 stag_index; + __be32 addrs_length; /* array of paddrs (must be aligned on a 64bit boundary) */ - u64 paddrs[0]; + __be64 paddrs[0]; } __attribute__((packed)) ; struct c2wr_nsmr_pbl_rep { @@ -847,7 +847,7 @@ union c2wr_mw_query { struct c2wr_stag_dealloc_req { struct c2wr_hdr hdr; u32 rnic_handle; - u32 stag_index; + __be32 stag_index; } __attribute__((packed)) ; struct c2wr_stag_dealloc_rep { @@ -949,7 +949,7 @@ struct c2wr_ce { u64 qp_user_context; /* c2_user_qp_t * */ u32 qp_state; /* Current QP State */ u32 handle; /* QPID or EP Handle */ - u32 bytes_rcvd; /* valid for RECV WCs */ + __be32 bytes_rcvd; /* valid for RECV WCs */ u32 stag; } __attribute__((packed)) ; @@ -984,8 +984,8 @@ struct c2_rq_hdr { */ struct c2wr_send_req { struct c2_sq_hdr sq_hdr; - u32 sge_len; - u32 remote_stag; + __be32 sge_len; + __be32 remote_stag; u8 data[0]; /* SGE array */ } __attribute__((packed)); @@ -996,9 +996,9 @@ union c2wr_send { struct c2wr_rdma_write_req { struct c2_sq_hdr sq_hdr; - u64 remote_to; - u32 remote_stag; - u32 sge_len; + __be64 remote_to; + __be32 remote_stag; + __be32 sge_len; u8 data[0]; /* SGE array */ } __attribute__((packed)); @@ -1009,11 +1009,11 @@ union c2wr_rdma_write { struct c2wr_rdma_read_req { struct c2_sq_hdr sq_hdr; - u64 local_to; - u64 remote_to; - u32 local_stag; - u32 remote_stag; - u32 length; + __be64 local_to; + __be64 remote_to; + __be32 local_stag; + __be32 remote_stag; + __be32 length; } __attribute__((packed)); union c2wr_rdma_read { @@ -1112,10 +1112,10 @@ union c2wr_recv { */ struct c2wr_ae_hdr { struct c2wr_hdr hdr; - u64 user_context; /* user context for this res. */ - u32 resource_type; /* see enum c2_resource_indicator */ - u32 resource; /* handle for resource */ - u32 qp_state; /* current QP State */ + __be64 user_context; /* user context for this res. */ + __be32 resource_type; /* see enum c2_resource_indicator */ + __be32 resource; /* handle for resource */ + __be32 qp_state; /* current QP State */ } __attribute__((packed)); /* @@ -1124,11 +1124,11 @@ struct c2wr_ae_hdr { */ struct c2wr_ae_active_connect_results { struct c2wr_ae_hdr ae_hdr; - u32 laddr; - u32 raddr; - u16 lport; - u16 rport; - u32 private_data_length; + __be32 laddr; + __be32 raddr; + __be16 lport; + __be16 rport; + __be32 private_data_length; u8 private_data[0]; /* data is in-line in the msg. */ } __attribute__((packed)); @@ -1142,11 +1142,11 @@ struct c2wr_ae_active_connect_results { struct c2wr_ae_connection_request { struct c2wr_ae_hdr ae_hdr; u32 cr_handle; /* connreq handle (sock ptr) */ - u32 laddr; - u32 raddr; - u16 lport; - u16 rport; - u32 private_data_length; + __be32 laddr; + __be32 raddr; + __be16 lport; + __be16 rport; + __be32 private_data_length; u8 private_data[0]; /* data is in-line in the msg. */ } __attribute__((packed)); @@ -1158,12 +1158,12 @@ union c2wr_ae { struct c2wr_init_req { struct c2wr_hdr hdr; - u64 hint_count; - u64 q0_host_shared; - u64 q1_host_shared; - u64 q1_host_msg_pool; - u64 q2_host_shared; - u64 q2_host_msg_pool; + __be64 hint_count; + __be64 q0_host_shared; + __be64 q1_host_shared; + __be64 q1_host_msg_pool; + __be64 q2_host_shared; + __be64 q2_host_msg_pool; } __attribute__((packed)); struct c2wr_init_rep { @@ -1276,10 +1276,10 @@ struct c2wr_ep_listen_create_req { struct c2wr_hdr hdr; u64 user_context; /* returned in AEs. */ u32 rnic_handle; - u32 local_addr; /* local addr, or 0 */ - u16 local_port; /* 0 means "pick one" */ + __be32 local_addr; /* local addr, or 0 */ + __be16 local_port; /* 0 means "pick one" */ u16 pad; - u32 backlog; /* tradional tcp listen bl */ + __be32 backlog; /* tradional tcp listen bl */ } __attribute__((packed)); struct c2wr_ep_listen_create_rep { @@ -1340,7 +1340,7 @@ struct c2wr_cr_accept_req { u32 rnic_handle; u32 qp_handle; /* QP to bind to this LLP conn */ u32 ep_handle; /* LLP handle to accept */ - u32 private_data_length; + __be32 private_data_length; u8 private_data[0]; /* data in-line in msg. */ } __attribute__((packed)); @@ -1508,7 +1508,7 @@ static __inline__ void c2_wr_set_sge_count(void *wr, u8 sge_count) { ((struct c2wr_hdr *) wr)->sge_count = sge_count; } -static __inline__ u32 c2_wr_get_wqe_count(void *wr) +static __inline__ __be32 c2_wr_get_wqe_count(void *wr) { return ((struct c2wr_hdr *) wr)->wqe_count; } -- 1.5.4.2 From sashak at voltaire.com Sat Mar 1 14:53:47 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 1 Mar 2008 22:53:47 +0000 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080301225347.GE11788@sashak.voltaire.com> On 19:59 Fri 29 Feb , Hal Rosenstock wrote: > > If that makes sense, then also query commands on this "state" would > likely also. Not sure about this. It is dynamically updated flag, so it would be hard to catch a "valid" value by hand from the OpenSM console. Sasha From sashak at voltaire.com Sat Mar 1 22:25:29 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 2 Mar 2008 06:25:29 +0000 Subject: [ofa-general] Re: [PATCH] opensm: console split console into two modules In-Reply-To: <47BCA935.4030907@llnl.gov> References: <47BCA935.4030907@llnl.gov> Message-ID: <20080302062529.GF11788@sashak.voltaire.com> Hi Tim, On 14:27 Wed 20 Feb , Timothy A. Meier wrote: > > This is the first of two patches, which depend on each other. > > I split the osm_console.c code into two modules, one containing the console > commands, and the other (new osm_console_io.c) containing the console > setup, > tear down and connection specific code. > > The primary purpose of the separation is to provide an isolated and > decoupled > place to add the implementation of the new SSL connection. > > -- > Timothy A. Meier > Computer Scientist > ICCD/High Performance Computing > 925.422.3341 > meier3 at llnl.gov > From 453991ff76f01464cf90d9d1a1b6b38bbb96ef7e Mon Sep 17 00:00:00 2001 > From: Tim Meier > Date: Wed, 20 Feb 2008 13:56:04 -0800 > Subject: [PATCH] opensm: console split console into two modules > > Added osm_console_io.c module, to decouple connection and IO > code from the console functionality. Provides the ability > to extend (add) an implementation for an SSL connection. > > Signed-off-by: Tim Meier Both patches are applied. The patches were split so that it was not compilable. I merged this together. Please be sure that each patch is valid. Also small tip is below. [snip...] > diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > index fcc7a44..f8c091e 100644 > --- a/opensm/opensm/main.c > +++ b/opensm/opensm/main.c > @@ -60,6 +60,7 @@ > #include > #include > #include > +#include > #include > > volatile unsigned int osm_exit_flag = 0; > @@ -550,7 +551,8 @@ static int daemonize(osm_opensm_t * osm) > int osm_manager_loop(osm_subn_opt_t * p_opt, osm_opensm_t * p_osm) > { > if (is_console_enabled(p_opt)) > - osm_console_init(p_opt, p_osm); > + osm_console_init(p_opt, &p_osm->console, &p_osm->log); > +// osm_console_init(p_opt, p_osm); Please don't leave such commented code which is not needed anymore. Sasha From kliteyn at dev.mellanox.co.il Sun Mar 2 01:39:15 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Sun, 02 Mar 2008 11:39:15 +0200 Subject: [ofa-general] Re: QoS management in OpenSM - doc In-Reply-To: <20080301210132.GA11788@sashak.voltaire.com> References: <47C576E0.8050405@dev.mellanox.co.il> <20080301210132.GA11788@sashak.voltaire.com> Message-ID: <47CA75C3.8090000@dev.mellanox.co.il> Sasha Khapyorsky wrote: > Hi Yevgeny, > > On 16:42 Wed 27 Feb , Yevgeny Kliteynik wrote: >> The following doc describes QoS management in OpenSM. >> This doc (named QoS_management_in_OpenSM.txt) has been added to >> the OFED docs, along with the QoS_in_OFED.txt. >> >> I'd like to add this info to OpenSM man pages as well. > > Yes, I think that it could be useful to have it under opensm/doc too. > >> I'm including the text here as is, so it will be easier to follow >> possible changes. When those will be done, I'll fix the format to >> match the OpenSM man pages and post a patch. >> >> The only problem is that the whole OpenSM man has ~850 lines, >> while this QoS management file has ~500 lines... :) > > I would suggest to have some basic part (50-100 lines) included in the > man page and reference an entire document (under opensm/doc) for more > details. OK, I can prepare some kind of summary that would go into the man page. However, this means that a user would no be able to define a QoS policy just from reading an OpenSM man pages - he will HAVE to check the full doc under opensm/doc. >> Please review. > > Looks fine, few tiny nits are below. > > [snip...] > >> ============================================================================== >> 4. Policy File Syntax Guidelines >> ============================================================================== >> >> - Empty lines are ignored. > > It is mentioned on the next line too. Right > >> - Leading and trailing blanks, as well as empty lines, are ignored, so >> the indentation in the example is just for better readability. >> - Comments are started with the pound sign (#) and terminated by EOL. >> - Any keyword should be the first non-blank in the line, unless it's a >> comment. >> - Keywords that denote section/subsection start have matching closing >> keywords. >> - Having a QoS Level named "DEFAULT" is a must - it is applied to PR/MPR >> requests that didn't match any of the matching rules. >> - Any section/subsection of the policy file is optional. > > [snip...] > >> ============================================================================== >> 6. Simplified QoS Policy - Details and Examples >> ============================================================================== >> >> Simplified QoS policy match rules are tailored for matching ULPs (or some >> application on top of a ULP) PR/MPR requests. This section has a list of >> per-ULP (or per-application) match rules and the SL that should be enforced >> on the matched PR/MPR query. >> >> Match rules include: >> - Default match rule that is applied to PR/MPR query that didn't match any >> of the other match rules >> - SDP >> - SDP application with a specific target TCP/IP port range >> - SRP with a specific target IB port GUID >> - RDS >> - iSER >> - iSER application with a specific target TCP/IP port range >> - IPoIB with a default PKey >> - IPoIB with a specific PKey >> - any ULP/application with a specific Service ID in the PR/MPR query >> - any ULP/application with a specific PKey in the PR/MPR query >> - any ULP/application with a specific target IB port GUID in the PR/MPR >> query >> >> Since any section of the policy file is optional, as long as basic rules of >> the file are kept (such as no referring to nonexisting port group, having >> default QoS Level, etc), the simplified policy section (qos-ulps) can serve >> as a complete QoS policy file. >> The shortest policy file in this case would be as follows: >> >> qos-ulps >> default : 0 #default SL >> end-qos-ulps >> >> It is equivalent to the previous example of the shortest policy file, and >> it >> is also equivalent to not having policy file at all. >> >> Below is an example of simplified QoS policy with all the possible >> keywords: >> >> qos-ulps >> default : 0 # default SL >> sdp, port-num 30000 : 0 # SL for application running on >> top >> # of SDP when a destination >> # TCP/IPport is 30000 >> sdp, port-num 10000-20000 : 0 >> sdp : 1 # default SL for any other >> # application running on top of >> SDP >> rds : 2 # SL for RDS traffic >> iser, port-num 900 : 0 # SL for iSER with a specific >> target >> # port >> iser : 3 # default SL for iSER >> ipoib, pkey 0x0001 : 0 # SL for IPoIB on partition with >> # pkey 0x0001 >> ipoib : 4 # default IPoIB partition, >> # pkey=0x7FFF >> any, service-id 0x6234 : 6 # match any PR/MPR query with a >> # specific Service ID >> any, pkey 0x0ABC : 6 # match any PR/MPR query with a >> # specific PKey >> srp, target-port-guid 0x1234 : 5 # SRP when SRP Target is located >> on >> # a specified IB port GUID >> any, target-port-guid 0x0ABC-0xFFFFF : 6 # match any PR/MPR query >> with >> # a specific target port GUID >> end-qos-ulps > > Likely I missed this in implementation phase. But isn't it better to > have ULPs to match QoS level rather than SL? Or probably both? The goal of this part is to give up some flexibility in order to get maximum simplicity. If an administrator wants to define a qos-level with more than just SL (and I think that in most cases he wouldn't want to do that), there's another way to refer to such qos-level - trough match rules. >> Similar to the full policy definition, matching of PR/MPR queries is done >> in >> order of appearance in the QoS policy file such as the first match takes >> precedence, except for the "default" rule, which is applied only if the >> query >> didn't match any other rule. >> >> All other sections of the QoS policy file take precedence over the qos-ulps >> section. That is, if a policy file has both qos-match-rules and qos-ulps >> sections, then any query is matched first against the rules in the >> qos-match-rules section, and only if there was no match, the query is >> matched >> against the rules in qos-ulps section. >> >> Note that some of these match rules may overlap, so in order to use the >> simplified QoS definition effectively, it is important to understand how >> each >> of the ULPs is matched: >> >> 6.1 IPoIB >> IPoIB query is matched by PKey. Default PKey for IPoIB partition is 0x7fff, >> so >> the following three match rules are equivalent: >> >> ipoib : >> ipoib, 0x7fff : >> any, pkey 0x7fff : >> >> 6.2 SDP >> SDP PR query is matched by Service ID. The Service-ID for SDP is >> 0x000000000001PPPP, where PPPP are 4 hex digits holding the remote TCP/IP >> Port >> Number to connect to. The following two match rules are equivalent: >> >> sdp : >> any, service-id 0x0000000000010000-0x000000000001ffff : >> >> 6.3 RDS >> Similar to SDP, RDS PR query is matched by Service ID. The Service ID for >> RDS >> is 0x000000000106PPPP, where PPPP are 4 hex digits holding the remote >> TCP/IP >> Port Number to connect to. Default port number for RDS is 0x48CA, which >> makes >> a default Service-ID 0x00000000010648CA. The following two match rules are >> equivalent: >> >> rds : >> any, service-id 0x00000000010648CA : >> >> 6.4 iSER >> Similar to RDS, iSER query is matched by Service ID, where the the Service >> ID >> is also 0x000000000106PPPP. Default port number for iSER is 0x035C, which >> makes >> a default Service-ID 0x000000000106035C. The following two match rules are >> equivalent: >> >> iser : >> any, service-id 0x000000000106035C : >> >> 6.5 SRP >> Service ID for SRP varies from storage vendor to vendor, thus SRP query is >> matched by the target IB port GUID. The following two match rules are >> equivalent: >> >> srp, target-port-guid 0x1234 : >> any, target-port-guid 0x1234 : >> >> Note that any of the above ULPs might contain target port GUID in the PR >> query, so in order for these queries not to be recognized by the QoS >> manager >> as SRP, the SRP match rule (or any match rule that refers to the target >> port >> guid only) should be placed at the end of the qos-ulps match rules. >> >> 6.6 MPI >> SL for MPI is manually configured by MPI admin. OpenSM is not forcing any >> SL >> on the MPI traffic, and that's why it is the only ULP that did not appear >> in >> the qos-ulps section. >> >> >> ============================================================================== >> 7. SL2VL Mapping and VL Arbitration >> ============================================================================== > > I think here should be stated that both policies should be merged at > some point. Agree -- Yevgeny > > [snip...] > > Sasha > From ll.ss22 at aliceadsl.fr Sun Mar 2 02:09:03 2008 From: ll.ss22 at aliceadsl.fr (=?iso-8859-1?Q?Gordon_Anthony_?=) Date: Sun, 2 Mar 2008 11:09:03 +0100 Subject: [ofa-general] MUTUAL TRUST Message-ID: Attn: It is indeed my pleasure to write to you this letter, which I believe will be a surprise, met on he net we are both complete strangers. As you read this, I don't want you to feel sorry for me, because I believe everyone will die someday. My name is Gordon Anthony, a former oil merchant in the middle east. I have been diagnosed with Esophageal cancer which was discovered very late, due to my laxity in caring for my health. It has defiled all forms of medicine, and right now I have only about a few months to live, according to medical experts. I have not particularly lived my life so well, as I never really cared for anyone not even myself but my business. Though I am very rich, I was never generous, I was always hostile to people and only focus on my business as that was the only thing I cared for, but now I regret all this as I now know that there is more to life than just wanting to have or make all the money in the world. I believe when I have a second chance to come to this world I would live my life a different way from how I had lived it, now that it is dark for me, I have willed and given most of my properties and assets to my immediate and extended family members and as well as a few close friends. To correct my wrong past life, I have decided to give alms to charity organizations, as I want this to be one of the last good deeds I do on earth. So far, I have distributed money to some charity organizations in the U.A.E, Algeria and Malaysia. Now that my health has deteriorated so badly, I cannot do this my self anymore. I once asked members of my family to close one of my accounts and distribute the money which I have there to charity organization in Bulgaria, India and Pakistan, they refused and kept the money to themselves. Hence, I do not trust them anymore, as they seem not to be contended with what I have left for them. The last of my money which no one knows of is the huge cash deposit of Six million dollars that I have with a Fiducially Company. I will want you to help me collect this deposit and dispatched it to charity organizations and you must be sending me information's of how it was disbursed by email. I have set aside 20% for you for your time and patience. Thanks. Eng. Gordon Anthony. ---------------------- ALICE C'EST ENCORE MIEUX AVEC LA MUSIQUE ! -------------------- Découvrez vite l'offre exclusive ALICE BOX avec ALICE MUSIC, le téléchargement légal et illimité de plus de 300 000 titres ! En cliquant ici http://alicemusic.aliceadsl.fr Offre soumise à conditions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Sun Mar 2 03:07:11 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sun, 2 Mar 2008 03:07:11 -0800 (PST) Subject: [ofa-general] ofa_1_3_kernel 20080302-0200 daily build status Message-ID: <20080302110712.16312E6085D@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From hrosenstock at xsigo.com Sun Mar 2 05:04:10 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Sun, 02 Mar 2008 05:04:10 -0800 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <20080301225347.GE11788@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> Message-ID: <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> On Sat, 2008-03-01 at 22:53 +0000, Sasha Khapyorsky wrote: > On 19:59 Fri 29 Feb , Hal Rosenstock wrote: > > > > If that makes sense, then also query commands on this "state" would > > likely also. > > Not sure about this. It is dynamically updated flag, so it would be hard > to catch a "valid" value by hand from the OpenSM console. I was referring to the "balance" state not that flag. Does that make more sense ? -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sashak at voltaire.com Sun Mar 2 06:17:13 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 2 Mar 2008 14:17:13 +0000 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080302141713.GC30723@sashak.voltaire.com> On 05:04 Sun 02 Mar , Hal Rosenstock wrote: > On Sat, 2008-03-01 at 22:53 +0000, Sasha Khapyorsky wrote: > > On 19:59 Fri 29 Feb , Hal Rosenstock wrote: > > > > > > If that makes sense, then also query commands on this "state" would > > > likely also. > > > > Not sure about this. It is dynamically updated flag, so it would be hard > > to catch a "valid" value by hand from the OpenSM console. > > I was referring to the "balance" state not that flag. Does that make > more sense ? What do you mean? Routing dumps? Sasha From eli at dev.mellanox.co.il Sun Mar 2 06:16:57 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Sun, 02 Mar 2008 16:16:57 +0200 Subject: [ofa-general] page allocation failure In-Reply-To: <200802281842.19303.bs@q-leap.de> References: <200802281842.19303.bs@q-leap.de> Message-ID: <1204467417.3358.123.camel@mtls03> Look like your system is low on memory. Maybe you have to add memory or maybe something eats your memory (a memory leak?). On Thu, 2008-02-28 at 18:42 +0100, Bernd Schubert wrote: > Hello, > > on several on our Lustre Servers we can see page allocation failures. > > This is with 2.6.22 + kernel modules from ofed 1.2.5 > > > [44464.764559] Lustre: 24052:0:(ldlm_lib.c:698:target_handle_connect()) Skipped 16 previous similar messages > [54132.351263] ib_cm/2: page allocation failure. order:0, mode:0x10d0 > [54132.360738] > [54132.360741] Call Trace: > [54132.367803] [] show_trace+0x34/0x47 > [54132.373235] [] dump_stack+0x12/0x17 > [54132.378937] [] __alloc_pages+0x2a3/0x2bc > [54132.386180] [] dma_alloc_pages+0x9b/0xbf > [54132.395120] [] dma_alloc_coherent+0x76/0x1cc > [54132.401651] [] :ib_mthca:mthca_buf_alloc+0x1bd/0x2a3 > [54132.408897] [] :ib_mthca:mthca_alloc_qp_common+0x246/0x4e5 > [54132.418884] [] :ib_mthca:mthca_alloc_qp+0xab/0x102 > [54132.425774] [] :ib_mthca:mthca_create_qp+0x126/0x281 > [54132.432716] [] :ib_core:ib_create_qp+0x17/0x91 > [54132.439102] [] :rdma_cm:rdma_create_qp+0x2d/0x153 > [54132.446301] [] :ko2iblnd:kiblnd_create_conn+0x81c/0x1250 > [54132.456992] [] :ko2iblnd:kiblnd_passive_connect+0x605/0xdd0 > [54132.469847] [] :ko2iblnd:kiblnd_cm_callback+0x255/0xeb0 > [54132.478821] [] :rdma_cm:cma_req_handler+0x322/0x389 > [54132.485637] [] :ib_cm:cm_process_work+0x17/0xad > [54132.492182] [] :ib_cm:cm_req_handler+0x7ae/0x81b > [54132.499236] [] :ib_cm:cm_work_handler+0x2d/0xbaa > [54132.506690] [] run_workqueue+0x7f/0x10b > [54132.512652] [] worker_thread+0xda/0xe4 > [54132.520136] [] kthread+0x47/0x75 > [54132.525570] [] child_rip+0xa/0x12 > [54132.532975] > [54132.535527] Mem-info: > [54132.538157] Node 0 DMA per-cpu: > [54132.542303] CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54132.551752] CPU 1: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54132.561661] CPU 2: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54132.571154] CPU 3: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54132.580597] CPU 4: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54132.592354] CPU 5: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54132.601794] CPU 6: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54132.610719] CPU 7: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54132.619630] Node 0 DMA32 per-cpu: > [54132.623551] CPU 0: Hot: hi: 186, btch: 31 usd: 49 Cold: hi: 62, btch: 15 usd: 49 > [54132.632691] CPU 1: Hot: hi: 186, btch: 31 usd: 26 Cold: hi: 62, btch: 15 usd: 3 > [54132.642680] CPU 2: Hot: hi: 186, btch: 31 usd: 30 Cold: hi: 62, btch: 15 usd: 54 > [54132.651897] CPU 3: Hot: hi: 186, btch: 31 usd: 1 Cold: hi: 62, btch: 15 usd: 13 > [54132.663321] CPU 4: Hot: hi: 186, btch: 31 usd: 43 Cold: hi: 62, btch: 15 usd: 55 > [54132.673282] CPU 5: Hot: hi: 186, btch: 31 usd: 30 Cold: hi: 62, btch: 15 usd: 49 > [54132.683636] CPU 6: Hot: hi: 186, btch: 31 usd: 25 Cold: hi: 62, btch: 15 usd: 1 > [54132.693156] CPU 7: Hot: hi: 186, btch: 31 usd: 13 Cold: hi: 62, btch: 15 usd: 56 > [54132.703412] Node 0 Normal per-cpu: > [54132.707024] CPU 0: Hot: hi: 186, btch: 31 usd: 130 Cold: hi: 62, btch: 15 usd: 14 > [54132.719317] CPU 1: Hot: hi: 186, btch: 31 usd: 81 Cold: hi: 62, btch: 15 usd: 1 > [54132.729276] CPU 2: Hot: hi: 186, btch: 31 usd: 134 Cold: hi: 62, btch: 15 usd: 2 > [54132.738819] CPU 3: Hot: hi: 186, btch: 31 usd: 124 Cold: hi: 62, btch: 15 usd: 8 > [54132.748078] CPU 4: Hot: hi: 186, btch: 31 usd: 21 Cold: hi: 62, btch: 15 usd: 4 > [54132.758029] CPU 5: Hot: hi: 186, btch: 31 usd: 30 Cold: hi: 62, btch: 15 usd: 9 > [54132.766855] CPU 6: Hot: hi: 186, btch: 31 usd: 120 Cold: hi: 62, btch: 15 usd: 13 > [54132.776462] CPU 7: Hot: hi: 186, btch: 31 usd: 166 Cold: hi: 62, btch: 15 usd: 12 > [54132.786009] Active:28507 inactive:62701 dirty:8386 writeback:27 unstable:0 > [54132.786010] free:5586 slab:273528 mapped:2136 pagetables:699 bounce:0 > [54132.803082] Node 0 DMA free:11192kB min:20kB low:24kB high:28kB active:0kB inactive:0kB present:10660kB pages_scanned:0 all_unreclaimable? yes > [54132.816507] lowmem_reserve[]: 0 3255 4013 > [54132.820811] Node 0 DMA32 free:9812kB min:6564kB low:8204kB high:9844kB active:52536kB inactive:134508kB present:3333728kB pages_scanned:0 all_unreclaimable? no > [54132.839252] lowmem_reserve[]: 0 0 757 > [54132.843205] Node 0 Normal free:1340kB min:1524kB low:1904kB high:2284kB active:61492kB inactive:116296kB present:775680kB pages_scanned:800 all_unreclaimable? no > [54132.859932] lowmem_reserve[]: 0 0 0 > [54132.863784] Node 0 DMA: 6*4kB 4*8kB 4*16kB 4*32kB 3*64kB 0*128kB 2*256kB 0*512kB 2*1024kB 0*2048kB 2*4096kB = 11192kB > [54132.876957] Node 0 DMA32: 48*4kB 33*8kB 26*16kB 3*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 2*4096kB = 9608kB > [54132.891138] Node 0 Normal: 0*4kB 0*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1456kB > [54132.903195] Swap cache: add 0, delete 0, find 0/0, race 0+0 > [54132.909967] Free swap = 4200888kB > [54132.913677] Total swap = 4200888kB > [54132.917229] Free swap: 4200888kB > [54132.967201] 1245184 pages of RAM > [54132.971121] 231685 reserved pages > [54132.974973] 58033 pages shared > [54132.978329] 0 pages swap cached > [54132.982267] LustreError: 4103:0:(o2iblnd.c:791:kiblnd_create_conn()) Can't create QP: -12 > [54177.640441] ib_cm/5: page allocation failure. order:0, mode:0x10d0 > [54177.648631] > [54177.648632] Call Trace: > [54177.653908] [] show_trace+0x34/0x47 > [54177.660073] [] dump_stack+0x12/0x17 > [54177.667176] [] __alloc_pages+0x2a3/0x2bc > [54177.682952] [] dma_alloc_pages+0x9b/0xbf > [54177.688811] [] dma_alloc_coherent+0x76/0x1cc > [54177.695277] [] :ib_mthca:mthca_buf_alloc+0x1bd/0x2a3 > [54177.702683] [] :ib_mthca:mthca_alloc_cq_buf+0x38/0x86 > [54177.711034] [] :ib_mthca:mthca_init_cq+0x12a/0x397 > [54177.718478] [] :ib_mthca:mthca_create_cq+0xf0/0x1be > [54177.725601] [] :ib_core:ib_create_cq+0x27/0x56 > [54177.732384] [] :ko2iblnd:kiblnd_create_conn+0x3b0/0x1250 > [54177.739683] [] :ko2iblnd:kiblnd_passive_connect+0x605/0xdd0 > [54177.748451] [] :ko2iblnd:kiblnd_cm_callback+0x255/0xeb0 > [54177.757088] [] :rdma_cm:cma_req_handler+0x322/0x389 > [54177.763985] [] :ib_cm:cm_process_work+0x17/0xad > [54177.770664] [] :ib_cm:cm_req_handler+0x7ae/0x81b > [54177.777248] [] :ib_cm:cm_work_handler+0x2d/0xbaa > [54177.784045] [] run_workqueue+0x7f/0x10b > [54177.790439] [] worker_thread+0xda/0xe4 > [54177.799862] [] kthread+0x47/0x75 > [54177.805672] [] child_rip+0xa/0x12 > [54177.811717] > [54177.813851] Mem-info: > [54177.816666] Node 0 DMA per-cpu: > [54177.820479] CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54177.829621] CPU 1: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54177.839216] CPU 2: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54177.849488] CPU 3: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54177.859625] CPU 4: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54177.871977] CPU 5: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54177.881930] CPU 6: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54177.891980] CPU 7: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0 > [54177.902800] Node 0 DMA32 per-cpu: > [54177.906462] CPU 0: Hot: hi: 186, btch: 31 usd: 10 Cold: hi: 62, btch: 15 usd: 58 > [54177.916162] CPU 1: Hot: hi: 186, btch: 31 usd: 26 Cold: hi: 62, btch: 15 usd: 3 > [54177.926049] CPU 2: Hot: hi: 186, btch: 31 usd: 139 Cold: hi: 62, btch: 15 usd: 54 > [54177.936948] CPU 3: Hot: hi: 186, btch: 31 usd: 1 Cold: hi: 62, btch: 15 usd: 13 > [54177.946968] CPU 4: Hot: hi: 186, btch: 31 usd: 56 Cold: hi: 62, btch: 15 usd: 55 > [54177.956868] CPU 5: Hot: hi: 186, btch: 31 usd: 30 Cold: hi: 62, btch: 15 usd: 57 > [54177.965685] CPU 6: Hot: hi: 186, btch: 31 usd: 25 Cold: hi: 62, btch: 15 usd: 1 > [54177.975412] CPU 7: Hot: hi: 186, btch: 31 usd: 13 Cold: hi: 62, btch: 15 usd: 56 > [54177.986045] Node 0 Normal per-cpu: > [54177.990527] CPU 0: Hot: hi: 186, btch: 31 usd: 128 Cold: hi: 62, btch: 15 usd: 14 > [54178.002993] CPU 1: Hot: hi: 186, btch: 31 usd: 81 Cold: hi: 62, btch: 15 usd: 1 > [54178.012136] CPU 2: Hot: hi: 186, btch: 31 usd: 113 Cold: hi: 62, btch: 15 usd: 2 > [54178.022533] CPU 3: Hot: hi: 186, btch: 31 usd: 124 Cold: hi: 62, btch: 15 usd: 8 > [54178.032316] CPU 4: Hot: hi: 186, btch: 31 usd: 27 Cold: hi: 62, btch: 15 usd: 4 > [54178.041380] CPU 5: Hot: hi: 186, btch: 31 usd: 24 Cold: hi: 62, btch: 15 usd: 9 > [54178.050941] CPU 6: Hot: hi: 186, btch: 31 usd: 120 Cold: hi: 62, btch: 15 usd: 13 > [54178.061180] CPU 7: Hot: hi: 186, btch: 31 usd: 166 Cold: hi: 62, btch: 15 usd: 12 > [54178.072162] Active:28319 inactive:62389 dirty:8381 writeback:27 unstable:0 > [54178.072163] free:5581 slab:273603 mapped:2117 pagetables:690 bounce:0 > [54178.087805] Node 0 DMA free:11192kB min:20kB low:24kB high:28kB active:0kB inactive:0kB present:10660kB pages_scanned:0 all_unreclaimable? yes > [54178.103794] lowmem_reserve[]: 0 3255 4013 > [54178.108294] Node 0 DMA32 free:9784kB min:6564kB low:8204kB high:9844kB active:51792kB inactive:133260kB present:3333728kB pages_scanned:0 all_unreclaimable? no > [54178.129648] lowmem_reserve[]: 0 0 757 > [54178.133756] Node 0 Normal free:1348kB min:1524kB low:1904kB high:2284kB active:61484kB inactive:116296kB present:775680kB pages_scanned:728 all_unreclaimable? no > [54178.154399] lowmem_reserve[]: 0 0 0 > [54178.158450] Node 0 DMA: 6*4kB 4*8kB 4*16kB 4*32kB 3*64kB 0*128kB 2*256kB 0*512kB 2*1024kB 0*2048kB 2*4096kB = 11192kB > [54178.172214] Node 0 DMA32: 65*4kB 17*8kB 37*16kB 6*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 2*4096kB = 9628kB > [54178.188210] Node 0 Normal: 0*4kB 1*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1464kB > [54178.202288] Swap cache: add 0, delete 0, find 0/0, race 0+0 > [54178.208654] Free swap = 4200888kB > [54178.212390] Total swap = 4200888kB > [54178.218597] Free swap: 4200888kB > [54178.264623] 1245184 pages of RAM > [54178.268302] 231685 reserved pages > [54178.271793] 57602 pages shared > [54178.275306] 0 pages swap cached > [54178.278778] LustreError: 4106:0:(o2iblnd.c:732:kiblnd_create_conn()) Can't create CQ: -12 > [54277.772930] ib_cm/2: page allocation failure. order:0, mode:0x10d0 > [54277.781944] > [54277.781945] Call Trace: > [54277.788321] [] show_trace+0x34/0x47 > [54277.793761] [] dump_stack+0x12/0x17 > [54277.799744] [] __alloc_pages+0x2a3/0x2bc > [54277.806044] [] dma_alloc_pages+0x9b/0xbf > [54277.814225] [] dma_alloc_coherent+0x76/0x1cc > [54277.821449] [] :ib_mthca:mthca_buf_alloc+0x1bd/0x2a3 > [54277.831300] [] :ib_mthca:mthca_alloc_qp_common+0x246/0x4e5 > [54277.838934] [] :ib_mthca:mthca_alloc_qp+0xab/0x102 > [54277.846467] [] :ib_mthca:mthca_create_qp+0x126/0x281 > [54277.854289] [] :ib_core:ib_create_qp+0x17/0x91 > [54277.862274] [] :rdma_cm:rdma_create_qp+0x2d/0x153 > [54277.870048] [] :ko2iblnd:kiblnd_create_conn+0x81c/0x1250 > [54277.877973] [] :ko2iblnd:kiblnd_passive_connect+0x605/0xdd0 > [54277.886679] [] :ko2iblnd:kiblnd_cm_callback+0x255/0xeb0 > [54277.895646] [] :rdma_cm:cma_req_handler+0x322/0x389 > [54277.903470] [] :ib_cm:cm_process_work+0x17/0xad > [54277.910567] [] :ib_cm:cm_req_handler+0x7ae/0x81b > [54277.918121] [] :ib_cm:cm_work_handler+0x2d/0xbaa > [54277.926378] [] run_workqueue+0x7f/0x10b > [54277.932202] [] worker_thread+0xda/0xe4 > [54277.938003] [] kthread+0x47/0x75 > [54277.944032] [] child_rip+0xa/0x12 > [54277.950581] > > > Any ideas? > > Thanks, > Bernd > From chu11 at llnl.gov Sun Mar 2 07:16:26 2008 From: chu11 at llnl.gov (Albert Chu) Date: Sun, 2 Mar 2008 07:16:26 -0800 (PST) Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <20080302141713.GC30723@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> <20080302141713.GC30723@sashak.voltaire.com> Message-ID: <45087.128.15.244.18.1204470986.squirrel@127.0.0.1> Hey Hal, Are you saying a flag inside each osm_switch_t to indicate if that specific switch is balanced? The script I wrote for the balance check did have difficulty determining a lot of corner cases (is port connected to a CA? is it active? what ports are up vs. down links, etc.). At the end of the day you just output a lot of extra info and have to look through it manually. Although probably not easy as a whole, these calculations would be easier in opensm since that information is available. Al > On 05:04 Sun 02 Mar , Hal Rosenstock wrote: >> On Sat, 2008-03-01 at 22:53 +0000, Sasha Khapyorsky wrote: >> > On 19:59 Fri 29 Feb , Hal Rosenstock wrote: >> > > >> > > If that makes sense, then also query commands on this "state" would >> > > likely also. >> > >> > Not sure about this. It is dynamically updated flag, so it would be >> hard >> > to catch a "valid" value by hand from the OpenSM console. >> >> I was referring to the "balance" state not that flag. Does that make >> more sense ? > > What do you mean? Routing dumps? > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From chu11 at llnl.gov Sun Mar 2 07:46:59 2008 From: chu11 at llnl.gov (Albert Chu) Date: Sun, 2 Mar 2008 07:46:59 -0800 (PST) Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> <20080301161023.GM1485@sashak.voltaire.com> <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> Message-ID: <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> Hey Sasha, In order to make things work, I also had to add this patch. Seems like a corner case that needs to be handled since we never fall into __osm_pi_rcv_process_switch_port(). (BTW, I am working off a 3.1.10 branch for the test cluster, so this patch is forward ported and technically untested.) --- a/opensm/opensm/osm_port_info_rcv.c +++ b/opensm/opensm/osm_port_info_rcv.c @@ -564,6 +564,7 @@ void osm_pi_rcv_process(IN void *context, IN void *data) ", Commencing heavy sweep\n", cl_ntoh64(node_guid), cl_ntoh64(port_guid)); sm->p_subn->force_heavy_sweep = 1; + sm->p_subn->ignore_existing_lfts = 1; goto Exit; } Al > Hey Sasha, > > This patch should definitely work. I'll let you know after I get a chance > to try it. > > Al > >> Hi Al, >> >> On 16:08 Sat 01 Mar , Sasha Khapyorsky wrote: >>> >>> When switch ports were reconnected we need to recalculate routing paths >>> balancing. Reconnection is detected by port state examination - when it >>> becomes INIT routing paths rebalancing (ignore_existing_lfts flag) is >>> enforced. >>> >>> Signed-off-by: Sasha Khapyorsky >> >> This patch is simpler than all previous ones. I tested it with ibsim >> already. Could you test in your environment? >> >> Sasha >> > > > -- > Albert Chu > chu11 at llnl.gov > 925-422-5311 > Computer Scientist > High Performance Systems Division > Lawrence Livermore National Laboratory > > -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From andrea at qumranet.com Sun Mar 2 07:54:57 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Sun, 2 Mar 2008 16:54:57 +0100 Subject: [ofa-general] [PATCH] mmu notifiers #v8 In-Reply-To: <20080227192610.GF28483@v2.random> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> Message-ID: <20080302155457.GK8091@v2.random> Difference between #v7 and #v8: 1) s/age_page/clear_flush_young/ (Nick's suggestion) 2) macro fix (Andrew) 3) move release before final unmap_vmas (for GRU, Jack/Christoph) 4) microoptimize mmu_notifier_unregister (Christoph) 5) use mmap_sem for registration serialization (Christoph) The (void)xxx in macros doesn't work with "args". Christoph's solution look best in avoiding warnings, even if it forces to make the mmu notifier operation structure visible even if MMU_NOTIFIER=n (that's the only downside). I didn't drop invalidate_page, because invalidate_range_begin/end would be slower for usages like KVM/GRU (we don't need a begin/end there because where invalidate_page is called, the VM holds a reference on the page). do_wp_page should also use invalidate_page since it can free the page after dropping the PT lock without losing any performance (that's not true for the places where invalidate_range is called). It'd be nice if everyone involved can agree to converge on this API for .25. KVM/GRU (and perhaps Quadrics) and similar usages will be fully covered in .25. This is a kernel internal API so there's no problem if all the methods will become sleep capable only starting only in .26. The brainer part of the VM work to do to make it sleep capable is pretty much orthogonal with this patch. Signed-off-by: Andrea Arcangeli Signed-off-by: Christoph Lameter diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -228,6 +229,8 @@ struct mm_struct { #ifdef CONFIG_CGROUP_MEM_CONT struct mem_cgroup *mem_cgroup; #endif + + struct mmu_notifier_head mmu_notifier; /* MMU notifier list */ }; #endif /* _LINUX_MM_TYPES_H */ diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h new file mode 100644 --- /dev/null +++ b/include/linux/mmu_notifier.h @@ -0,0 +1,161 @@ +#ifndef _LINUX_MMU_NOTIFIER_H +#define _LINUX_MMU_NOTIFIER_H + +#include +#include + +struct mmu_notifier; + +struct mmu_notifier_ops { + /* + * Called when nobody can register any more notifier in the mm + * and after the "mn" notifier has been disarmed already. + */ + void (*release)(struct mmu_notifier *mn, + struct mm_struct *mm); + + /* + * clear_flush_young is called after the VM is + * test-and-clearing the young/accessed bitflag in the + * pte. This way the VM will provide proper aging to the + * accesses to the page through the secondary MMUs and not + * only to the ones through the Linux pte. + */ + int (*clear_flush_young)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long address); + + /* + * Before this is invoked any secondary MMU is still ok to + * read/write to the page previously pointed by the Linux pte + * because the old page hasn't been freed yet. If required + * set_page_dirty has to be called internally to this method. + */ + void (*invalidate_page)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long address); + + /* + * invalidate_range_begin() and invalidate_range_end() must be + * paired. Multiple invalidate_range_begin/ends may be nested + * or called concurrently. + */ + void (*invalidate_range_begin)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long start, unsigned long end); + void (*invalidate_range_end)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long start, unsigned long end); +}; + +struct mmu_notifier { + struct hlist_node hlist; + const struct mmu_notifier_ops *ops; +}; + +#ifdef CONFIG_MMU_NOTIFIER + +struct mmu_notifier_head { + struct hlist_head head; +}; + +#include + +/* + * Must hold the mmap_sem for write. + * + * RCU is used to traverse the list. A quiescent period needs to pass + * before the notifier is guaranteed to be visible to all threads. + */ +extern void mmu_notifier_register(struct mmu_notifier *mn, + struct mm_struct *mm); +/* + * Must hold the mmap_sem for write. + * + * RCU is used to traverse the list. A quiescent period needs to pass + * before the "struct mmu_notifier" can be freed. Alternatively it + * can be synchronously freed inside ->release when the list can't + * change anymore and nobody could possibly walk it. + */ +extern void mmu_notifier_unregister(struct mmu_notifier *mn, + struct mm_struct *mm); +extern void mmu_notifier_release(struct mm_struct *mm); +extern int mmu_notifier_clear_flush_young(struct mm_struct *mm, + unsigned long address); + +static inline void mmu_notifier_head_init(struct mmu_notifier_head *mnh) +{ + INIT_HLIST_HEAD(&mnh->head); +} + +#define mmu_notifier(function, mm, args...) \ + do { \ + struct mmu_notifier *__mn; \ + struct hlist_node *__n; \ + struct mm_struct * __mm = mm; \ + \ + if (unlikely(!hlist_empty(&__mm->mmu_notifier.head))) { \ + rcu_read_lock(); \ + hlist_for_each_entry_rcu(__mn, __n, \ + &__mm->mmu_notifier.head, \ + hlist) \ + if (__mn->ops->function) \ + __mn->ops->function(__mn, \ + __mm, \ + args); \ + rcu_read_unlock(); \ + } \ + } while (0) + +#define ptep_clear_flush_notify(__vma, __address, __ptep) \ +({ \ + pte_t __pte; \ + struct vm_area_struct * ___vma = __vma; \ + unsigned long ___address = __address; \ + __pte = ptep_clear_flush(___vma, ___address, __ptep); \ + mmu_notifier(invalidate_page, ___vma->vm_mm, ___address); \ + __pte; \ +}) + +#define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ +({ \ + int __young; \ + struct vm_area_struct * ___vma = __vma; \ + unsigned long ___address = __address; \ + __young = ptep_clear_flush_young(___vma, ___address, __ptep); \ + __young |= mmu_notifier_clear_flush_young(___vma->vm_mm, \ + ___address); \ + __young; \ +}) + +#else /* CONFIG_MMU_NOTIFIER */ + +struct mmu_notifier_head {}; + +#define mmu_notifier_register(mn, mm) do {} while(0) +#define mmu_notifier_unregister(mn, mm) do {} while (0) +#define mmu_notifier_release(mm) do {} while (0) +#define mmu_notifier_head_init(mmh) do {} while (0) + +/* + * Notifiers that use the parameters that they were passed so that the + * compiler does not complain about unused variables but does proper + * parameter checks even if !CONFIG_MMU_NOTIFIER. + * Macros generate no code. + */ +#define mmu_notifier(function, mm, args...) \ + do { \ + if (0) { \ + struct mmu_notifier *__mn; \ + \ + __mn = (struct mmu_notifier *)(0x00ff); \ + __mn->ops->function(__mn, mm, args); \ + }; \ + } while (0) + +#define ptep_clear_flush_young_notify ptep_clear_flush_young +#define ptep_clear_flush_notify ptep_clear_flush + +#endif /* CONFIG_MMU_NOTIFIER */ + +#endif /* _LINUX_MMU_NOTIFIER_H */ diff --git a/kernel/fork.c b/kernel/fork.c --- a/kernel/fork.c +++ b/kernel/fork.c @@ -362,6 +362,7 @@ static struct mm_struct * mm_init(struct if (likely(!mm_alloc_pgd(mm))) { mm->def_flags = 0; + mmu_notifier_head_init(&mm->mmu_notifier); return mm; } diff --git a/mm/Kconfig b/mm/Kconfig --- a/mm/Kconfig +++ b/mm/Kconfig @@ -193,3 +193,7 @@ config VIRT_TO_BUS config VIRT_TO_BUS def_bool y depends on !ARCH_NO_VIRT_TO_BUS + +config MMU_NOTIFIER + def_bool y + bool "MMU notifier, for paging KVM/RDMA" diff --git a/mm/Makefile b/mm/Makefile --- a/mm/Makefile +++ b/mm/Makefile @@ -33,4 +33,4 @@ obj-$(CONFIG_SMP) += allocpercpu.o obj-$(CONFIG_SMP) += allocpercpu.o obj-$(CONFIG_QUICKLIST) += quicklist.o obj-$(CONFIG_CGROUP_MEM_CONT) += memcontrol.o - +obj-$(CONFIG_MMU_NOTIFIER) += mmu_notifier.o diff --git a/mm/filemap_xip.c b/mm/filemap_xip.c --- a/mm/filemap_xip.c +++ b/mm/filemap_xip.c @@ -194,7 +194,7 @@ __xip_unmap (struct address_space * mapp if (pte) { /* Nuke the page table entry. */ flush_cache_page(vma, address, pte_pfn(*pte)); - pteval = ptep_clear_flush(vma, address, pte); + pteval = ptep_clear_flush_notify(vma, address, pte); page_remove_rmap(page, vma); dec_mm_counter(mm, file_rss); BUG_ON(pte_dirty(pteval)); diff --git a/mm/fremap.c b/mm/fremap.c --- a/mm/fremap.c +++ b/mm/fremap.c @@ -214,7 +214,9 @@ asmlinkage long sys_remap_file_pages(uns spin_unlock(&mapping->i_mmap_lock); } + mmu_notifier(invalidate_range_begin, mm, start, start + size); err = populate_range(mm, vma, start, size, pgoff); + mmu_notifier(invalidate_range_end, mm, start, start + size); if (!err && !(flags & MAP_NONBLOCK)) { if (unlikely(has_write_lock)) { downgrade_write(&mm->mmap_sem); diff --git a/mm/hugetlb.c b/mm/hugetlb.c --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -755,6 +755,7 @@ void __unmap_hugepage_range(struct vm_ar BUG_ON(start & ~HPAGE_MASK); BUG_ON(end & ~HPAGE_MASK); + mmu_notifier(invalidate_range_begin, mm, start, end); spin_lock(&mm->page_table_lock); for (address = start; address < end; address += HPAGE_SIZE) { ptep = huge_pte_offset(mm, address); @@ -775,6 +776,7 @@ void __unmap_hugepage_range(struct vm_ar } spin_unlock(&mm->page_table_lock); flush_tlb_range(vma, start, end); + mmu_notifier(invalidate_range_end, mm, start, end); list_for_each_entry_safe(page, tmp, &page_list, lru) { list_del(&page->lru); put_page(page); diff --git a/mm/memory.c b/mm/memory.c --- a/mm/memory.c +++ b/mm/memory.c @@ -611,6 +611,9 @@ int copy_page_range(struct mm_struct *ds if (is_vm_hugetlb_page(vma)) return copy_hugetlb_page_range(dst_mm, src_mm, vma); + if (is_cow_mapping(vma->vm_flags)) + mmu_notifier(invalidate_range_begin, src_mm, addr, end); + dst_pgd = pgd_offset(dst_mm, addr); src_pgd = pgd_offset(src_mm, addr); do { @@ -621,6 +624,11 @@ int copy_page_range(struct mm_struct *ds vma, addr, next)) return -ENOMEM; } while (dst_pgd++, src_pgd++, addr = next, addr != end); + + if (is_cow_mapping(vma->vm_flags)) + mmu_notifier(invalidate_range_end, src_mm, + vma->vm_start, end); + return 0; } @@ -897,7 +905,9 @@ unsigned long zap_page_range(struct vm_a lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); + mmu_notifier(invalidate_range_begin, mm, address, end); end = unmap_vmas(&tlb, vma, address, end, &nr_accounted, details); + mmu_notifier(invalidate_range_end, mm, address, end); if (tlb) tlb_finish_mmu(tlb, address, end); return end; @@ -1463,10 +1473,11 @@ int apply_to_page_range(struct mm_struct { pgd_t *pgd; unsigned long next; - unsigned long end = addr + size; + unsigned long start = addr, end = addr + size; int err; BUG_ON(addr >= end); + mmu_notifier(invalidate_range_begin, mm, start, end); pgd = pgd_offset(mm, addr); do { next = pgd_addr_end(addr, end); @@ -1474,6 +1485,7 @@ int apply_to_page_range(struct mm_struct if (err) break; } while (pgd++, addr = next, addr != end); + mmu_notifier(invalidate_range_end, mm, start, end); return err; } EXPORT_SYMBOL_GPL(apply_to_page_range); @@ -1675,7 +1687,7 @@ gotten: * seen in the presence of one thread doing SMC and another * thread doing COW. */ - ptep_clear_flush(vma, address, page_table); + ptep_clear_flush_notify(vma, address, page_table); set_pte_at(mm, address, page_table, entry); update_mmu_cache(vma, address, entry); lru_cache_add_active(new_page); diff --git a/mm/mmap.c b/mm/mmap.c --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1747,11 +1747,13 @@ static void unmap_region(struct mm_struc lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); + mmu_notifier(invalidate_range_begin, mm, start, end); unmap_vmas(&tlb, vma, start, end, &nr_accounted, NULL); vm_unacct_memory(nr_accounted); free_pgtables(&tlb, vma, prev? prev->vm_end: FIRST_USER_ADDRESS, next? next->vm_start: 0); tlb_finish_mmu(tlb, start, end); + mmu_notifier(invalidate_range_end, mm, start, end); } /* @@ -2037,6 +2039,7 @@ void exit_mmap(struct mm_struct *mm) unsigned long end; /* mm's last user has gone, and its about to be pulled down */ + mmu_notifier_release(mm); arch_exit_mmap(mm); lru_add_drain(); diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c new file mode 100644 --- /dev/null +++ b/mm/mmu_notifier.c @@ -0,0 +1,75 @@ +/* + * linux/mm/mmu_notifier.c + * + * Copyright (C) 2008 Qumranet, Inc. + * Copyright (C) 2008 SGI + * Christoph Lameter + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + */ + +#include +#include +#include + +/* + * No synchronization. This function can only be called when only a single + * process remains that performs teardown. + */ +void mmu_notifier_release(struct mm_struct *mm) +{ + struct mmu_notifier *mn; + + while (unlikely(!hlist_empty(&mm->mmu_notifier.head))) { + mn = hlist_entry(mm->mmu_notifier.head.first, + struct mmu_notifier, + hlist); + hlist_del(&mn->hlist); + if (mn->ops->release) + mn->ops->release(mn, mm); + } +} + +/* + * If no young bitflag is supported by the hardware, ->age_page can + * unmap the address and return 1 or 0 depending if the mapping previously + * existed or not. + */ +int mmu_notifier_clear_flush_young(struct mm_struct *mm, unsigned long address) +{ + struct mmu_notifier *mn; + struct hlist_node *n; + int young = 0; + + if (unlikely(!hlist_empty(&mm->mmu_notifier.head))) { + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, + &mm->mmu_notifier.head, hlist) { + if (mn->ops->clear_flush_young) + young |= mn->ops->clear_flush_young(mn, mm, + address); + } + rcu_read_unlock(); + } + + return young; +} + +/* + * Note that all notifiers use RCU. The updates are only guaranteed to + * be visible to other processes after a RCU quiescent period! + * + * Must hold mmap_sem writably when calling registration functions. + */ +void mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm) +{ + hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier.head); +} +EXPORT_SYMBOL_GPL(mmu_notifier_register); + +void mmu_notifier_unregister(struct mmu_notifier *mn, struct mm_struct *mm) +{ + hlist_del_rcu(&mn->hlist); +} +EXPORT_SYMBOL_GPL(mmu_notifier_unregister); diff --git a/mm/mprotect.c b/mm/mprotect.c --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -198,10 +198,12 @@ success: dirty_accountable = 1; } + mmu_notifier(invalidate_range_begin, mm, start, end); if (is_vm_hugetlb_page(vma)) hugetlb_change_protection(vma, start, end, vma->vm_page_prot); else change_protection(vma, start, end, vma->vm_page_prot, dirty_accountable); + mmu_notifier(invalidate_range_end, mm, start, end); vm_stat_account(mm, oldflags, vma->vm_file, -nrpages); vm_stat_account(mm, newflags, vma->vm_file, nrpages); return 0; diff --git a/mm/mremap.c b/mm/mremap.c --- a/mm/mremap.c +++ b/mm/mremap.c @@ -74,6 +74,7 @@ static void move_ptes(struct vm_area_str struct mm_struct *mm = vma->vm_mm; pte_t *old_pte, *new_pte, pte; spinlock_t *old_ptl, *new_ptl; + unsigned long old_start; if (vma->vm_file) { /* @@ -100,6 +101,9 @@ static void move_ptes(struct vm_area_str spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); arch_enter_lazy_mmu_mode(); + old_start = old_addr; + mmu_notifier(invalidate_range_begin, vma->vm_mm, + old_start, old_end); for (; old_addr < old_end; old_pte++, old_addr += PAGE_SIZE, new_pte++, new_addr += PAGE_SIZE) { if (pte_none(*old_pte)) @@ -108,6 +112,7 @@ static void move_ptes(struct vm_area_str pte = move_pte(pte, new_vma->vm_page_prot, old_addr, new_addr); set_pte_at(mm, new_addr, new_pte, pte); } + mmu_notifier(invalidate_range_end, vma->vm_mm, old_start, old_end); arch_leave_lazy_mmu_mode(); if (new_ptl != old_ptl) diff --git a/mm/rmap.c b/mm/rmap.c --- a/mm/rmap.c +++ b/mm/rmap.c @@ -287,7 +287,7 @@ static int page_referenced_one(struct pa if (vma->vm_flags & VM_LOCKED) { referenced++; *mapcount = 1; /* break early from loop */ - } else if (ptep_clear_flush_young(vma, address, pte)) + } else if (ptep_clear_flush_young_notify(vma, address, pte)) referenced++; /* Pretend the page is referenced if the task has the @@ -454,7 +454,7 @@ static int page_mkclean_one(struct page pte_t entry; flush_cache_page(vma, address, pte_pfn(*pte)); - entry = ptep_clear_flush(vma, address, pte); + entry = ptep_clear_flush_notify(vma, address, pte); entry = pte_wrprotect(entry); entry = pte_mkclean(entry); set_pte_at(mm, address, pte, entry); @@ -712,14 +712,14 @@ static int try_to_unmap_one(struct page * skipped over this mm) then we should reactivate it. */ if (!migration && ((vma->vm_flags & VM_LOCKED) || - (ptep_clear_flush_young(vma, address, pte)))) { + (ptep_clear_flush_young_notify(vma, address, pte)))) { ret = SWAP_FAIL; goto out_unmap; } /* Nuke the page table entry. */ flush_cache_page(vma, address, page_to_pfn(page)); - pteval = ptep_clear_flush(vma, address, pte); + pteval = ptep_clear_flush_notify(vma, address, pte); /* Move the dirty bit to the physical page now the pte is gone. */ if (pte_dirty(pteval)) @@ -844,12 +844,12 @@ static void try_to_unmap_cluster(unsigne page = vm_normal_page(vma, address, *pte); BUG_ON(!page || PageAnon(page)); - if (ptep_clear_flush_young(vma, address, pte)) + if (ptep_clear_flush_young_notify(vma, address, pte)) continue; /* Nuke the page table entry. */ flush_cache_page(vma, address, pte_pfn(*pte)); - pteval = ptep_clear_flush(vma, address, pte); + pteval = ptep_clear_flush_notify(vma, address, pte); /* If nonlinear, store the file page offset in the pte. */ if (page->index != linear_page_index(vma, address)) From andrea at qumranet.com Sun Mar 2 08:03:51 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Sun, 2 Mar 2008 17:03:51 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 + xpmem In-Reply-To: <20080302155457.GK8091@v2.random> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> Message-ID: <20080302160351.GL8091@v2.random> Here an example of the futher orthogonal work to do on top of #v8 during .26-rc to make the whole mmu notifier API sleep capable. 1) Every single ptep_clear_flush_young_notify and ptep_clear_flush_notify must be converted like the below. The below is the conversion of a single one. do_wp_page has been converted by Christoph already but with invalidate_range (should be changed to invalidate_page by releasing the refcount on the page after calling invalidate_page). Hope it's clear why I'd rather not depend on these changes to be merged in .25 in order to have the mmu notifier included in .25. 2) Then after all this conversion work is finished, it's trivial to delete ptep_clear_flush_young_notify and ptep_clear_flush_notify from mmu_notifier.h (they will be unused macros once the conversion is complete). 3) After that the VM has to be changed to convert anon_vma lock and i_mmap_lock spinlocks to mutex/rwsemaphore. 4) Then finally the mmu_notifier_unregister must be dropped to make the mmu notifier sleep capable with RCU in the mmu_notifier() fast path. It's unclear at this point if 3/4 should be switchable and happening under a CONFIG_XPMEM or similar or if everyone will benefit from those spinlock becoming mutex (the only one that is certain to appreciate such a change is preempt-rt, the rest of the userbase I don't know for sure and I'd be more confortable with a TPC number comparison before doing such a chance by default, but I leave the commentary on such a change to linux-mm in a separate thread). Signed-off-by: Andrea Arcangeli diff --git a/mm/rmap.c b/mm/rmap.c --- a/mm/rmap.c +++ b/mm/rmap.c @@ -274,7 +274,7 @@ static int page_referenced_one(struct pa unsigned long address; pte_t *pte; spinlock_t *ptl; - int referenced = 0; + int referenced = 0, clear_flush_young = 0; address = vma_address(page, vma); if (address == -EFAULT) @@ -287,8 +287,11 @@ static int page_referenced_one(struct pa if (vma->vm_flags & VM_LOCKED) { referenced++; *mapcount = 1; /* break early from loop */ - } else if (ptep_clear_flush_young_notify(vma, address, pte)) - referenced++; + } else { + clear_flush_young = 1; + if (ptep_clear_flush_young(vma, address, pte)) + referenced++; + } /* Pretend the page is referenced if the task has the swap token and is in the middle of a page fault. */ @@ -298,6 +301,11 @@ static int page_referenced_one(struct pa (*mapcount)--; pte_unmap_unlock(pte, ptl); + + if (clear_flush_young) + referenced += mmu_notifier_clear_flush_young(vma->vm_mm, + address); + out: return referenced; } From chu11 at llnl.gov Sun Mar 2 08:10:50 2008 From: chu11 at llnl.gov (Albert Chu) Date: Sun, 2 Mar 2008 08:10:50 -0800 (PST) Subject: [ofa-general] [infiniband-diags] check_lft_balance script Message-ID: <52160.128.15.244.18.1204474250.squirrel@127.0.0.1> Hey Sasha, Here's the script I mentioned before that I used for the balance checking earlier. Its nothing fancy but probably could be useful to others. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-add-check_lft_balance-script.patch Type: text/x-patch Size: 11863 bytes Desc: not available URL: From RickyWhitney31 at yahoo.co.uk Sun Mar 2 09:10:31 2008 From: RickyWhitney31 at yahoo.co.uk (Canadian Pharmacy) Date: Sun, 2 Mar 2008 10:10:31 -0700 Subject: [ofa-general] Viagra Message-ID: <000b01c87c4d$a5819c80$56f88d3e@62.141.248.86> Viagra is an oral drug for male impotence, also known as erectile dysfunction. Having been around for a lot longer, Viagra has a great safety track record and proven effects that start acting in 30 minutes and last for about 5 hours. Please visit our site for more details. Type the URL below without spaces to visit us h t t p : / / b u r n h i t . c o m / From a.p.zijlstra at chello.nl Sun Mar 2 08:23:58 2008 From: a.p.zijlstra at chello.nl (Peter Zijlstra) Date: Sun, 02 Mar 2008 17:23:58 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 + xpmem In-Reply-To: <20080302160351.GL8091@v2.random> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080302160351.GL8091@v2.random> Message-ID: <1204475038.6240.47.camel@lappy> On Sun, 2008-03-02 at 17:03 +0100, Andrea Arcangeli wrote: > 4) Then finally the mmu_notifier_unregister must be dropped to make the > mmu notifier sleep capable with RCU in the mmu_notifier() fast path. Or require PREEMPTIBLE_RCU, that can handle sleeps.. From hrosenstock at xsigo.com Sun Mar 2 09:51:49 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Sun, 02 Mar 2008 09:51:49 -0800 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <20080302141713.GC30723@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> <20080302141713.GC30723@sashak.voltaire.com> Message-ID: <1204480309.6469.201.camel@hrosenstock-ws.xsigo.com> On Sun, 2008-03-02 at 14:17 +0000, Sasha Khapyorsky wrote: > On 05:04 Sun 02 Mar , Hal Rosenstock wrote: > > On Sat, 2008-03-01 at 22:53 +0000, Sasha Khapyorsky wrote: > > > On 19:59 Fri 29 Feb , Hal Rosenstock wrote: > > > > > > > > If that makes sense, then also query commands on this "state" would > > > > likely also. > > > > > > Not sure about this. It is dynamically updated flag, so it would be hard > > > to catch a "valid" value by hand from the OpenSM console. > > > > I was referring to the "balance" state not that flag. Does that make > > more sense ? > > What do you mean? Routing dumps? A different routing dump reflecting balance or not and how out of balance. -- Hal > > Sasha From hrosenstock at xsigo.com Sun Mar 2 09:54:18 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Sun, 02 Mar 2008 09:54:18 -0800 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <45087.128.15.244.18.1204470986.squirrel@127.0.0.1> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> <20080302141713.GC30723@sashak.voltaire.com> <45087.128.15.244.18.1204470986.squirrel@127.0.0.1> Message-ID: <1204480458.6469.205.camel@hrosenstock-ws.xsigo.com> Hi Al, On Sun, 2008-03-02 at 07:16 -0800, Albert Chu wrote: > Hey Hal, > > Are you saying a flag inside each osm_switch_t to indicate if that > specific switch is balanced? I wasn't saying anything about implementation. I was saying there could be OpenSM console commands to 1. rebalance, and 2. display relevant state regarding balance/imbalance. > The script I wrote for the balance check did have difficulty determining a > lot of corner cases (is port connected to a CA? is it active? what ports > are up vs. down links, etc.). At the end of the day you just output a lot > of extra info and have to look through it manually. > > Although probably not easy as a whole, these calculations would be easier > in opensm since that information is available. That's what I was suggesting rather than a separate diag script although the latter seems like it would be good too. -- Hal > Al > > > On 05:04 Sun 02 Mar , Hal Rosenstock wrote: > >> On Sat, 2008-03-01 at 22:53 +0000, Sasha Khapyorsky wrote: > >> > On 19:59 Fri 29 Feb , Hal Rosenstock wrote: > >> > > > >> > > If that makes sense, then also query commands on this "state" would > >> > > likely also. > >> > > >> > Not sure about this. It is dynamically updated flag, so it would be > >> hard > >> > to catch a "valid" value by hand from the OpenSM console. > >> > >> I was referring to the "balance" state not that flag. Does that make > >> more sense ? > > > > What do you mean? Routing dumps? > > > > Sasha > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > > From sashak at voltaire.com Sun Mar 2 11:05:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 2 Mar 2008 19:05:05 +0000 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <1204480309.6469.201.camel@hrosenstock-ws.xsigo.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> <20080302141713.GC30723@sashak.voltaire.com> <1204480309.6469.201.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080302190504.GE30723@sashak.voltaire.com> On 09:51 Sun 02 Mar , Hal Rosenstock wrote: > > A different routing dump reflecting balance or not and how out of > balance. This makes sense. Actually OpenSM has such sort of dump right now, but it is printed to stdout. Sasha From chu11 at llnl.gov Sun Mar 2 10:53:41 2008 From: chu11 at llnl.gov (Albert Chu) Date: Sun, 2 Mar 2008 10:53:41 -0800 (PST) Subject: [ofa-general] [infiniband-diags] check_lft_balance script In-Reply-To: <52160.128.15.244.18.1204474250.squirrel@127.0.0.1> References: <52160.128.15.244.18.1204474250.squirrel@127.0.0.1> Message-ID: <57972.128.15.244.53.1204484021.squirrel@127.0.0.1> Hey Sasha, Noticed one more thing I could clean up from the original script. here's a new one. Al > Hey Sasha, > > Here's the script I mentioned before that I used for the balance checking > earlier. Its nothing fancy but probably could be useful to others. > > Al > > -- > Albert Chu > chu11 at llnl.gov > 925-422-5311 > Computer Scientist > High Performance Systems Division > Lawrence Livermore National Laboratory > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-add-check_lft_balance-script.patch Type: text/x-patch Size: 11778 bytes Desc: not available URL: From Margarita at gs.com Sun Mar 2 10:54:47 2008 From: Margarita at gs.com (Margarita Sterling) Date: Sun, 02 Mar 2008 20:54:47 +0200 Subject: [ofa-general] Achieve all your dreams Message-ID: <008d01c87c96$e2ea6590$c0a80107@Margarita> Gain the greatest Schlong ever! >!ck enlargement becomes much easier! http://mutyouch.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Sun Mar 2 12:05:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 2 Mar 2008 20:05:05 +0000 Subject: [ofa-general] [PATCH] opensm: consolidate SA response sending code over SA processors Message-ID: <20080302200505.GF30723@sashak.voltaire.com> Consolidate SA response sending code over SA processors in single osm_sa_respond() function. Drop a lot of duplicated code. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_sa.h | 31 +++++ opensm/opensm/osm_sa.c | 116 +++++++++++++++++- opensm/opensm/osm_sa_guidinfo_record.c | 130 +-------------------- opensm/opensm/osm_sa_informinfo.c | 180 ++++------------------------ opensm/opensm/osm_sa_lft_record.c | 132 +-------------------- opensm/opensm/osm_sa_link_record.c | 145 +---------------------- opensm/opensm/osm_sa_mcmember_record.c | 203 +++++-------------------------- opensm/opensm/osm_sa_mft_record.c | 132 +-------------------- opensm/opensm/osm_sa_multipath_record.c | 104 ++-------------- opensm/opensm/osm_sa_node_record.c | 121 +------------------ opensm/opensm/osm_sa_path_record.c | 139 +--------------------- opensm/opensm/osm_sa_pkey_record.c | 133 +-------------------- opensm/opensm/osm_sa_portinfo_record.c | 142 +-------------------- opensm/opensm/osm_sa_service_record.c | 167 +++----------------------- opensm/opensm/osm_sa_slvl_record.c | 133 +-------------------- opensm/opensm/osm_sa_sminfo_record.c | 133 +-------------------- opensm/opensm/osm_sa_sw_info_record.c | 123 +------------------ opensm/opensm/osm_sa_vlarb_record.c | 133 +-------------------- 18 files changed, 244 insertions(+), 2153 deletions(-) diff --git a/opensm/include/opensm/osm_sa.h b/opensm/include/opensm/osm_sa.h index a150695..f4f751b 100644 --- a/opensm/include/opensm/osm_sa.h +++ b/opensm/include/opensm/osm_sa.h @@ -398,6 +398,37 @@ osm_sa_send_error(IN osm_sa_t * sa, * SA object *********/ +/****f* OpenSM: SA/osm_sa_respond +* NAME +* osm_sa_respond +* +* DESCRIPTION +* Sends SA MAD response +*/ +void osm_sa_respond(osm_sa_t *sa, osm_madw_t *madw, size_t attr_size, + cl_qlist_t *list); +/* +* PARAMETERS +* sa +* [in] Pointer to an osm_sa_t object. +* +* p_madw +* [in] Original MAD to which the response must be sent. +* +* attr_size +* [in] Size of this SA attribute. +* +* list +* [in] List of attribute to respond - it will be freed after +* sending. +* +* RETURN VALUES +* None. +* +* SEE ALSO +* SA object +*********/ + struct _osm_opensm_t; /****f* OpenSM: SA/osm_sa_db_file_dump * NAME diff --git a/opensm/opensm/osm_sa.c b/opensm/opensm/osm_sa.c index c557876..4edce47 100644 --- a/opensm/opensm/osm_sa.c +++ b/opensm/opensm/osm_sa.c @@ -360,7 +360,7 @@ osm_sa_send_error(IN osm_sa_t * sa, &p_madw->mad_addr); if (p_resp_madw == NULL) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2301: " + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4C07: " "Unable to acquire response MAD\n"); goto Exit; } @@ -399,6 +399,120 @@ Exit: OSM_LOG_EXIT(sa->p_log); } +void osm_sa_respond(osm_sa_t *sa, osm_madw_t *madw, size_t attr_size, + cl_qlist_t *list) +{ + struct item_data { + cl_list_item_t list; + char data[0]; + }; + cl_list_item_t *item; + osm_madw_t *resp_madw; + ib_sa_mad_t *sa_mad, *resp_sa_mad; + unsigned num_rec, i; +#ifndef VENDOR_RMPP_SUPPORT + unsigned trim_num_rec; +#endif + void *p; + + sa_mad = osm_madw_get_sa_mad_ptr(madw); + num_rec = cl_qlist_count(list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error! + */ + if (sa_mad->method == IB_MAD_METHOD_GET && num_rec > 1) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4C05: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(sa, madw, IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + goto Exit; + } + +#ifndef VENDOR_RMPP_SUPPORT + trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / attr_size; + if (trim_num_rec < num_rec) { + OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } +#endif + + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); + + if (sa_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { + osm_sa_send_error(sa, madw, IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + resp_madw = osm_mad_pool_get(sa->p_mad_pool, madw->h_bind, + num_rec * attr_size + IB_SA_MAD_HDR_SIZE, + &madw->mad_addr); + if (!resp_madw) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4C06: " + "osm_mad_pool_get failed\n"); + osm_sa_send_error(sa, madw, IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + resp_sa_mad = osm_madw_get_sa_mad_ptr(resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(resp_sa_mad, sa_mad, IB_SA_MAD_HDR_SIZE); + if (resp_sa_mad->method == IB_MAD_METHOD_SET) + resp_sa_mad->method = IB_MAD_METHOD_GET; + resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + resp_sa_mad->sm_key = 0; + + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + resp_sa_mad->attr_offset = ib_get_attr_offset(attr_size); + + p = ib_sa_mad_get_payload_ptr(resp_sa_mad); + +#ifndef VENDOR_RMPP_SUPPORT + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } +#else + /* forcefully define the packet as RMPP one */ + if (resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; +#endif + + for (i = 0; i < num_rec; i++) { + item = cl_qlist_remove_head(list); + memcpy(p, ((struct item_data *)item)->data, attr_size); + p += attr_size; + } + + osm_sa_vendor_send(resp_madw->h_bind, resp_madw, FALSE, sa->p_subn); + + osm_dump_sa_mad(sa->p_log, resp_sa_mad, OSM_LOG_FRAMES); +Exit: + /* need to set the mem free ... */ + item = cl_qlist_remove_head(list); + while (item != cl_qlist_end(list)) { + free(item); + item = cl_qlist_remove_head(list); + } +} + /********************************************************************** **********************************************************************/ /* diff --git a/opensm/opensm/osm_sa_guidinfo_record.c b/opensm/opensm/osm_sa_guidinfo_record.c index 76332bd..43f249a 100644 --- a/opensm/opensm/osm_sa_guidinfo_record.c +++ b/opensm/opensm/osm_sa_guidinfo_record.c @@ -314,16 +314,7 @@ void osm_gir_rcv_process(IN void *ctx, IN void *data) const ib_sa_mad_t *p_rcvd_mad; const ib_guidinfo_record_t *p_rcvd_rec; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - ib_guidinfo_record_t *p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_gir_search_ctxt_t context; - osm_gir_item_t *p_rec_item; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -376,126 +367,7 @@ void osm_gir_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5103: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_gir_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_gir_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_gir_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - - IB_SA_MAD_HDR_SIZE) / sizeof(ib_guidinfo_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_guidinfo_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5106: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_gir_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_guidinfo_record_t)); - - p_resp_rec = (ib_guidinfo_record_t *) - ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = (osm_gir_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_rec = p_rec_item->rec; - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_guidinfo_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_informinfo.c b/opensm/opensm/osm_sa_informinfo.c index d231102..f91d0ab 100644 --- a/opensm/opensm/osm_sa_informinfo.c +++ b/opensm/opensm/osm_sa_informinfo.c @@ -218,12 +218,10 @@ Set(InformInfo) request. **********************************************************************/ static void __osm_infr_rcv_respond(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { - osm_madw_t *p_resp_madw; - const ib_sa_mad_t *p_sa_mad; - ib_sa_mad_t *p_resp_sa_mad; - ib_inform_info_t *p_resp_infr; + cl_qlist_t rec_list; + osm_iir_item_t *item; OSM_LOG_ENTER(sa->p_log); @@ -232,31 +230,21 @@ __osm_infr_rcv_respond(IN osm_sa_t * sa, "Generating successful InformInfo response\n"); } - /* - Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, - p_madw->h_bind, - MAD_BLOCK_SIZE, &p_madw->mad_addr); - if (!p_resp_madw) { + item = malloc(sizeof(*item)); + if (!item) { OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4303: " - "Unable to allocate MAD\n"); + "rec_item alloc failed\n"); goto Exit; } - p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* copy the request InformInfo */ - memcpy(p_resp_sa_mad, p_sa_mad, MAD_BLOCK_SIZE); - p_resp_sa_mad->method = IB_MAD_METHOD_GET_RESP; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; + memcpy(&item->rec, + ib_sa_mad_get_payload_ptr(osm_madw_get_sa_mad_ptr(p_madw)), + sizeof(item->rec)); - p_resp_infr = - (ib_inform_info_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); + cl_qlist_init(&rec_list); + cl_qlist_insert_tail(&rec_list, &item->list_item); - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_inform_info_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); @@ -351,22 +339,14 @@ Received a Get(InformInfoRecord) or GetTable(InformInfoRecord) MAD **********************************************************************/ static void osm_infr_rcv_process_get_method(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { ib_sa_mad_t *p_rcvd_mad; const ib_inform_info_record_t *p_rcvd_rec; - ib_inform_info_record_t *p_resp_rec; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i, j; osm_iir_search_ctxt_t context; - osm_iir_item_t *p_rec_item; osm_physp_t *p_req_physp; + osm_iir_item_t *item; OSM_LOG_ENTER(sa->p_log); @@ -416,131 +396,15 @@ osm_infr_rcv_process_get_method(IN osm_sa_t * sa, cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 430A: " - "More than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_iir_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_iir_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_iir_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = - (MAD_BLOCK_SIZE - - IB_SA_MAD_HDR_SIZE) / sizeof(ib_inform_info_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; + /* clear reserved and pad fields in InformInfoRecord */ + for (item = (osm_iir_item_t *) cl_qlist_head(&rec_list); + item != (osm_iir_item_t *) cl_qlist_end(&rec_list); + item = (osm_iir_item_t *)cl_qlist_next(&item->list_item)) { + memset(item->rec.reserved, 0, sizeof(item->rec.reserved)); + memset(item->rec.pad, 0, sizeof(item->rec.pad)); } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, - p_madw->h_bind, - num_rec * - sizeof(ib_inform_info_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 430B: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_iir_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_inform_info_record_t)); - - p_resp_rec = (ib_inform_info_record_t *) - ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = (osm_iir_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) { - *p_resp_rec = p_rec_item->rec; - /* clear reserved and pad fields in InformInfoRecord */ - for (j = 0; j < 6; j++) - p_resp_rec->reserved[j] = 0; - for (j = 0; j < 4; j++) - p_resp_rec->pad[j] = 0; - } - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_inform_info_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); @@ -551,7 +415,7 @@ Received a Set(InformInfo) MAD **********************************************************************/ static void osm_infr_rcv_process_set_method(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { ib_sa_mad_t *p_sa_mad; ib_inform_info_t *p_recvd_inform_info; diff --git a/opensm/opensm/osm_sa_lft_record.c b/opensm/opensm/osm_sa_lft_record.c index c516231..82bb63c 100644 --- a/opensm/opensm/osm_sa_lft_record.c +++ b/opensm/opensm/osm_sa_lft_record.c @@ -219,17 +219,8 @@ void osm_lftr_rcv_process(IN void *ctx, IN void *data) osm_madw_t *p_madw = data; const ib_sa_mad_t *p_rcvd_mad; const ib_lft_record_t *p_rcvd_rec; - ib_lft_record_t *p_resp_rec; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_lftr_search_ctxt_t context; - osm_lftr_item_t *p_rec_item; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -279,128 +270,7 @@ void osm_lftr_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4409: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_lftr_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_lftr_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_lftr_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = - (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_lft_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_lft_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4410: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_lftr_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_lft_record_t)); - - p_resp_rec = - (ib_lft_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = - (osm_lftr_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_rec = p_rec_item->rec; - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_lft_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_link_record.c b/opensm/opensm/osm_sa_link_record.c index 2badc9d..31abf43 100644 --- a/opensm/opensm/osm_sa_link_record.c +++ b/opensm/opensm/osm_sa_link_record.c @@ -443,142 +443,6 @@ Exit: /********************************************************************** **********************************************************************/ -static void -__osm_lr_rcv_respond(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw, - IN cl_qlist_t * const p_list) -{ - osm_madw_t *p_resp_madw; - const ib_sa_mad_t *p_sa_mad; - ib_sa_mad_t *p_resp_sa_mad; - size_t num_rec, num_copied; -#ifndef VENDOR_RMPP_SUPPORT - size_t trim_num_rec; -#endif - ib_link_record_t *p_resp_lr; - osm_lr_item_t *p_lr_item; - const ib_sa_mad_t *p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); - - OSM_LOG_ENTER(sa->p_log); - - num_rec = cl_qlist_count(p_list); - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1806: " - "Got more than one record for SubnAdmGet (%zu)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_lr_item = (osm_lr_item_t *) cl_qlist_remove_head(p_list); - while (p_lr_item != (osm_lr_item_t *) cl_qlist_end(p_list)) { - free(p_lr_item); - p_lr_item = - (osm_lr_item_t *) cl_qlist_remove_head(p_list); - } - - goto Exit; - } -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_link_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Generating response with %zu records", num_rec); - } - - /* - Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_link_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1802: " - "Unable to allocate MAD\n"); - /* Release the quick pool items */ - p_lr_item = (osm_lr_item_t *) cl_qlist_remove_head(p_list); - while (p_lr_item != (osm_lr_item_t *) cl_qlist_end(p_list)) { - free(p_lr_item); - p_lr_item = - (osm_lr_item_t *) cl_qlist_remove_head(p_list); - } - - goto Exit; - } - - p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* Copy the header from the request to response */ - memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_link_record_t)); - /* C15-0.1.5 - always return SM_Key = 0 (table table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - p_resp_lr = - (ib_link_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - - if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { - p_resp_sa_mad->status = IB_SA_MAD_STATUS_NO_RECORDS; - memset(p_resp_lr, 0, sizeof(*p_resp_lr)); - } else { - p_lr_item = (osm_lr_item_t *) cl_qlist_remove_head(p_list); - /* we need to track the number of copied items so we can - * stop the copy - but clear them all - */ - num_copied = 0; - while (p_lr_item != (osm_lr_item_t *) cl_qlist_end(p_list)) { - /* Copy the Link Records from the list into the MAD */ - /* only if we did not go over the mad size (since we might trimmed it) */ - if (num_copied < num_rec) { - *p_resp_lr = p_lr_item->link_rec; - num_copied++; - } - free(p_lr_item); - p_resp_lr++; - p_lr_item = - (osm_lr_item_t *) cl_qlist_remove_head(p_list); - } - } - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); - -Exit: - OSM_LOG_EXIT(sa->p_log); -} - -/********************************************************************** - **********************************************************************/ void osm_lr_rcv_process(IN void *context, IN void *data) { osm_sa_t *sa = context; @@ -641,15 +505,8 @@ void osm_lr_rcv_process(IN void *context, IN void *data) cl_plock_release(sa->p_lock); - if (cl_qlist_count(&lr_list) == 0 && - p_sa_mad->method == IB_MAD_METHOD_GET) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - __osm_lr_rcv_respond(sa, p_madw, &lr_list); + osm_sa_respond(sa, p_madw, sizeof(ib_link_record_t), &lr_list); Exit: - OSM_LOG_EXIT(sa->p_log); } diff --git a/opensm/opensm/osm_sa_mcmember_record.c b/opensm/opensm/osm_sa_mcmember_record.c index 1de05e7..136ba77 100644 --- a/opensm/opensm/osm_sa_mcmember_record.c +++ b/opensm/opensm/osm_sa_mcmember_record.c @@ -334,59 +334,35 @@ Generate the response MAD **********************************************************************/ static void __osm_mcmr_rcv_respond(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw, + IN osm_madw_t * const p_madw, IN ib_member_rec_t * p_mcmember_rec) { - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_sa_mad, *p_resp_sa_mad; - ib_member_rec_t *p_resp_mcmember_rec; + cl_qlist_t rec_list; + osm_mcmr_item_t *item; OSM_LOG_ENTER(sa->p_log); - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, - p_madw->h_bind, - sizeof(ib_member_rec_t) + - IB_SA_MAD_HDR_SIZE, - osm_madw_get_mad_addr_ptr(p_madw)); - if (!p_resp_madw) + item = malloc(sizeof(*item)); + if (!item) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B16: " + "rec_item alloc failed\n"); goto Exit; - - p_resp_sa_mad = (ib_sa_mad_t *) p_resp_madw->p_mad; - p_sa_mad = (ib_sa_mad_t *) p_madw->p_mad; - /* Copy the MAD header back into the response mad */ - memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); - /* based on the current method decide about the response: */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GET || - p_resp_sa_mad->method == IB_MAD_METHOD_SET) - p_resp_sa_mad->method = IB_MAD_METHOD_GET_RESP; - else if (p_resp_sa_mad->method == IB_MAD_METHOD_DELETE) - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - else { - CL_ASSERT(p_resp_sa_mad->method == 0); } - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_member_rec_t)); - p_resp_mcmember_rec = (ib_member_rec_t *) & p_resp_sa_mad->data; - - *p_resp_mcmember_rec = *p_mcmember_rec; + item->rec = *p_mcmember_rec; /* Fill in the mtu, rate, and packet lifetime selectors */ - p_resp_mcmember_rec->mtu &= 0x3f; - p_resp_mcmember_rec->mtu |= 2 << 6; /* exactly */ - p_resp_mcmember_rec->rate &= 0x3f; - p_resp_mcmember_rec->rate |= 2 << 6; /* exactly */ - p_resp_mcmember_rec->pkt_life &= 0x3f; - p_resp_mcmember_rec->pkt_life |= 2 << 6; /* exactly */ + item->rec.mtu &= 0x3f; + item->rec.mtu |= 2 << 6; /* exactly */ + item->rec.rate &= 0x3f; + item->rec.rate |= 2 << 6; /* exactly */ + item->rec.pkt_life &= 0x3f; + item->rec.pkt_life |= 2 << 6; /* exactly */ + + cl_qlist_init(&rec_list); + cl_qlist_insert_tail(&rec_list, &item->list_item); - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_member_rec_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); @@ -1188,7 +1164,7 @@ Process a request for leaving the group **********************************************************************/ static void __osm_mcmr_rcv_leave_mgrp(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { boolean_t valid; osm_mgrp_t *p_mgrp; @@ -1307,7 +1283,7 @@ Exit: **********************************************************************/ static void __osm_mcmr_rcv_join_mgrp(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { boolean_t valid; osm_mgrp_t *p_mgrp = NULL; @@ -1804,21 +1780,12 @@ Exit: **********************************************************************/ static void __osm_mcmr_query_mgrp(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { const ib_sa_mad_t *p_rcvd_mad; const ib_member_rec_t *p_rcvd_rec; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - ib_member_rec_t *p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_sa_mcmr_search_ctxt_t context; - osm_mcmr_item_t *p_rec_item; ib_net64_t comp_mask; osm_physp_t *p_req_physp; boolean_t trusted_req; @@ -1862,108 +1829,6 @@ __osm_mcmr_query_mgrp(IN osm_sa_t * sa, CL_PLOCK_RELEASE(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B05: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_mcmr_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_mcmr_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = - (osm_mcmr_item_t *) cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_member_rec_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_member_rec_t) + - IB_SA_MAD_HDR_SIZE, - osm_madw_get_mad_addr_ptr(p_madw)); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1B16: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_mcmr_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_member_rec_t)); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - p_resp_rec = - (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - /* p923 - The PortGID, JoinState and ProxyJoin shall be zero, except in the case of a trusted request. @@ -1972,26 +1837,18 @@ __osm_mcmr_query_mgrp(IN osm_sa_t * sa, sm_key. */ - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = - (osm_mcmr_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) { - *p_resp_rec = p_rec_item->rec; - if (trusted_req == FALSE) { - memset(&p_resp_rec->port_gid, 0, - sizeof(ib_gid_t)); - ib_member_set_join_state(p_resp_rec, 0); - p_resp_rec->proxy_join = 0; - } + if (!p_rcvd_mad->sm_key) { + osm_mcmr_item_t *item; + for (item = (osm_mcmr_item_t *) cl_qlist_head(&rec_list); + item != (osm_mcmr_item_t *) cl_qlist_end(&rec_list); + item = (osm_mcmr_item_t *)cl_qlist_next(&item->list_item)) { + memset(&item->rec.port_gid, 0, sizeof(ib_gid_t)); + ib_member_set_join_state(&item->rec, 0); + item->rec.proxy_join = 0; } - free(p_rec_item); - p_resp_rec++; } - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_member_rec_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_mft_record.c b/opensm/opensm/osm_sa_mft_record.c index 2ad82bd..da786a0 100644 --- a/opensm/opensm/osm_sa_mft_record.c +++ b/opensm/opensm/osm_sa_mft_record.c @@ -250,17 +250,8 @@ void osm_mftr_rcv_process(IN void *ctx, IN void *data) osm_madw_t *p_madw = data; const ib_sa_mad_t *p_rcvd_mad; const ib_mft_record_t *p_rcvd_rec; - ib_mft_record_t *p_resp_rec; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_mftr_search_ctxt_t context; - osm_mftr_item_t *p_rec_item; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -311,128 +302,7 @@ void osm_mftr_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4A09: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_mftr_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_mftr_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_mftr_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = - (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_mft_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_mft_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4A10: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_mftr_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_mft_record_t)); - - p_resp_rec = - (ib_mft_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = - (osm_mftr_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_rec = p_rec_item->rec; - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_mft_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_multipath_record.c b/opensm/opensm/osm_sa_multipath_record.c index e9ddea5..4634ba9 100644 --- a/opensm/opensm/osm_sa_multipath_record.c +++ b/opensm/opensm/osm_sa_multipath_record.c @@ -69,10 +69,10 @@ typedef struct _osm_mpr_item { cl_list_item_t list_item; + ib_path_rec_t path_rec; const osm_port_t *p_src_port; const osm_port_t *p_dest_port; int hops; - ib_path_rec_t path_rec; } osm_mpr_item_t; typedef struct _osm_path_parms { @@ -1453,102 +1453,12 @@ Exit: /********************************************************************** **********************************************************************/ -static void -__osm_mpr_rcv_respond(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw, - IN cl_qlist_t * const p_list) -{ - osm_madw_t *p_resp_madw; - const ib_sa_mad_t *p_sa_mad; - ib_sa_mad_t *p_resp_sa_mad; - size_t num_rec; - size_t mad_size; - ib_path_rec_t *p_resp_pr; - ib_multipath_rec_t *p_mpr; - osm_mpr_item_t *p_mpr_item; - uint32_t i; - - OSM_LOG_ENTER(sa->p_log); - - p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - p_mpr = (ib_multipath_rec_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); - - num_rec = cl_qlist_count(p_list); - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Generating response with %zu records\n", num_rec); - - mad_size = IB_SA_MAD_HDR_SIZE + num_rec * sizeof(ib_path_rec_t); - - /* - Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - mad_size, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, - "ERR 4502: Unable to allocate MAD\n"); - - for (i = 0; i < num_rec; i++) { - p_mpr_item = - (osm_mpr_item_t *) cl_qlist_remove_head(p_list); - free(p_mpr_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* - o15-0.2.7: If MultiPath is supported, then SA shall respond to a - SubnAdmGetMulti() containing a valid MultiPathRecord attribute with - a set of zero or more PathRecords satisfying the constraints indicated - in the MultiPathRecord received. The PathRecord Attribute ID shall be - used in the response. - */ - p_resp_sa_mad->attr_id = IB_MAD_ATTR_PATH_RECORD; - p_resp_sa_mad->attr_offset = ib_get_attr_offset(sizeof(ib_path_rec_t)); - - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; - - p_resp_pr = (ib_path_rec_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - - for (i = 0; i < num_rec; i++) { - p_mpr_item = (osm_mpr_item_t *) cl_qlist_remove_head(p_list); - - /* Copy the Path Records from the list into the MAD */ - *p_resp_pr = p_mpr_item->path_rec; - - free(p_mpr_item); - p_resp_pr++; - } - - CL_ASSERT(cl_is_qlist_empty(p_list)); - - osm_dump_sa_mad(sa->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); - -Exit: - OSM_LOG_EXIT(sa->p_log); -} - -/********************************************************************** - **********************************************************************/ void osm_mpr_rcv_process(IN void *context, IN void *data) { osm_sa_t *sa = context; osm_madw_t *p_madw = data; const ib_multipath_rec_t *p_mpr; - const ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_sa_mad; osm_port_t *requester_port; osm_port_t *pp_ports[IB_MULTIPATH_MAX_GIDS]; cl_qlist_t pr_list; @@ -1629,7 +1539,15 @@ void osm_mpr_rcv_process(IN void *context, IN void *data) p_sa_mad->comp_mask, &pr_list); cl_plock_release(sa->p_lock); - __osm_mpr_rcv_respond(sa, p_madw, &pr_list); + + /* o15-0.2.7: If MultiPath is supported, then SA shall respond to a + SubnAdmGetMulti() containing a valid MultiPathRecord attribute with + a set of zero or more PathRecords satisfying the constraints + indicated in the MultiPathRecord received. The PathRecord Attribute + ID shall be used in the response. + */ + p_sa_mad->attr_id = IB_MAD_ATTR_PATH_RECORD; + osm_sa_respond(sa, p_madw, sizeof(ib_path_rec_t), &pr_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_node_record.c b/opensm/opensm/osm_sa_node_record.c index 58ce2a0..1e9d7dc 100644 --- a/opensm/opensm/osm_sa_node_record.c +++ b/opensm/opensm/osm_sa_node_record.c @@ -315,17 +315,8 @@ void osm_nr_rcv_process(IN void *ctx, IN void *data) osm_madw_t *p_madw = data; const ib_sa_mad_t *p_rcvd_mad; const ib_node_record_t *p_rcvd_rec; - ib_node_record_t *p_resp_rec; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_nr_search_ctxt_t context; - osm_nr_item_t *p_rec_item; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -377,117 +368,7 @@ void osm_nr_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1D03: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_nr_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != (osm_nr_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = - (osm_nr_item_t *) cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = - (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_node_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_node_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1D06: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_nr_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_node_record_t)); - - p_resp_rec = - (ib_node_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = (osm_nr_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_rec = p_rec_item->rec; - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_node_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c index 8ed44f4..ef3537b 100644 --- a/opensm/opensm/osm_sa_path_record.c +++ b/opensm/opensm/osm_sa_path_record.c @@ -1645,143 +1645,6 @@ Exit: /********************************************************************** **********************************************************************/ -static void -__osm_pr_rcv_respond(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw, - IN cl_qlist_t * const p_list) -{ - osm_madw_t *p_resp_madw; - const ib_sa_mad_t *p_sa_mad; - ib_sa_mad_t *p_resp_sa_mad; - size_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - size_t trim_num_rec; -#endif - ib_path_rec_t *p_resp_pr; - const ib_sa_mad_t *sad_mad = osm_madw_get_sa_mad_ptr(p_madw); - osm_pr_item_t *p_pr_item; - uint32_t i; - - OSM_LOG_ENTER(sa->p_log); - - num_rec = cl_qlist_count(p_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (sad_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F13: " - "Got more than one record for SubnAdmGet (%zu)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - /* need to set the mem free ... */ - p_pr_item = - (osm_pr_item_t *) cl_qlist_remove_head(p_list); - while (p_pr_item != - (osm_pr_item_t *) cl_qlist_end(p_list)) { - free(p_pr_item); - p_pr_item = (osm_pr_item_t *) - cl_qlist_remove_head(p_list); - } - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_path_rec_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Generating response with %zu records\n", num_rec); - - if (sad_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_path_rec_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F14: " - "Unable to allocate MAD\n"); - - for (i = 0; i < num_rec; i++) { - p_pr_item = - (osm_pr_item_t *) cl_qlist_remove_head(p_list); - free(p_pr_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = ib_get_attr_offset(sizeof(ib_path_rec_t)); - - p_resp_pr = (ib_path_rec_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_pr_item = (osm_pr_item_t *) cl_qlist_remove_head(p_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_pr = p_pr_item->path_rec; - - free(p_pr_item); - p_resp_pr++; - } - - CL_ASSERT(cl_is_qlist_empty(p_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); - -Exit: - OSM_LOG_EXIT(sa->p_log); -} - -/********************************************************************** - **********************************************************************/ void osm_pr_rcv_process(IN void *context, IN void *data) { osm_sa_t *sa = context; @@ -1965,7 +1828,7 @@ Unlock: cl_plock_release(sa->p_lock); /* Now, (finally) respond to the PathRecord request */ - __osm_pr_rcv_respond(sa, p_madw, &pr_list); + osm_sa_respond(sa, p_madw, sizeof(ib_path_rec_t), &pr_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_pkey_record.c b/opensm/opensm/osm_sa_pkey_record.c index 4cf8f9d..5cea525 100644 --- a/opensm/opensm/osm_sa_pkey_record.c +++ b/opensm/opensm/osm_sa_pkey_record.c @@ -236,16 +236,7 @@ void osm_pkey_rec_rcv_process(IN void *ctx, IN void *data) const osm_port_t *p_port = NULL; const ib_pkey_table_t *p_pkey; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - ib_pkey_table_record_t *p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_pkey_search_ctxt_t context; - osm_pkey_item_t *p_rec_item; ib_api_status_t status = IB_SUCCESS; ib_net64_t comp_mask; osm_physp_t *p_req_physp; @@ -351,129 +342,7 @@ void osm_pkey_rec_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 460A: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_pkey_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_pkey_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_pkey_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - - IB_SA_MAD_HDR_SIZE) / sizeof(ib_pkey_table_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * - sizeof(ib_pkey_table_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4606: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_pkey_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_pkey_table_record_t)); - - p_resp_rec = (ib_pkey_table_record_t *) - ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = - (osm_pkey_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_rec = p_rec_item->rec; - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_pkey_table_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_portinfo_record.c b/opensm/opensm/osm_sa_portinfo_record.c index 0f7c23d..ad9c9ae 100644 --- a/opensm/opensm/osm_sa_portinfo_record.c +++ b/opensm/opensm/osm_sa_portinfo_record.c @@ -479,20 +479,10 @@ void osm_pir_rcv_process(IN void *ctx, IN void *data) const osm_port_t *p_port = NULL; const ib_port_info_t *p_pi; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - ib_portinfo_record_t *p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_pir_search_ctxt_t context; - osm_pir_item_t *p_rec_item; ib_api_status_t status = IB_SUCCESS; ib_net64_t comp_mask; osm_physp_t *p_req_physp; - boolean_t trusted_req = TRUE; CL_ASSERT(sa); @@ -587,115 +577,6 @@ void osm_pir_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2108: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_pir_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_pir_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_pir_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - - IB_SA_MAD_HDR_SIZE) / sizeof(ib_portinfo_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_portinfo_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2106: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_pir_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_portinfo_record_t)); - - p_resp_rec = (ib_portinfo_record_t *) - ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - /* p922 - The M_Key returned shall be zero, except in the case of a trusted request. @@ -703,24 +584,15 @@ void osm_pir_rcv_process(IN void *ctx, IN void *data) the mad is valid. Meaning - is either zero or equal to the local sm_key. */ - if (p_rcvd_mad->sm_key == 0) - trusted_req = FALSE; - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = (osm_pir_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) { - *p_resp_rec = p_rec_item->rec; - if (trusted_req == FALSE) - p_resp_rec->port_info.m_key = 0; - } - free(p_rec_item); - p_resp_rec++; + if (!p_rcvd_mad->sm_key) { + osm_pir_item_t *item; + for (item = (osm_pir_item_t *) cl_qlist_head(&rec_list); + item != (osm_pir_item_t *) cl_qlist_end(&rec_list); + item = (osm_pir_item_t *)cl_qlist_next(&item->list_item)) + item->rec.port_info.m_key = 0; } - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_portinfo_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_service_record.c b/opensm/opensm/osm_sa_service_record.c index 457934a..ce99db2 100644 --- a/opensm/opensm/osm_sa_service_record.c +++ b/opensm/opensm/osm_sa_service_record.c @@ -211,158 +211,25 @@ Exit: **********************************************************************/ static void __osm_sr_rcv_respond(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw, + IN osm_madw_t * const p_madw, IN cl_qlist_t * const p_list) { - osm_madw_t *p_resp_madw; - const ib_sa_mad_t *p_sa_mad; - ib_sa_mad_t *p_resp_sa_mad; - uint32_t num_rec, num_copied; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - ib_service_record_t *p_resp_sr; - osm_sr_item_t *p_sr_item; - const ib_sa_mad_t *p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); - boolean_t trusted_req = TRUE; - - OSM_LOG_ENTER(sa->p_log); - - num_rec = cl_qlist_count(p_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2406: " - "Got more than one record for SubnAdmGet (%u).\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_sr_item = (osm_sr_item_t *) cl_qlist_remove_head(p_list); - while (p_sr_item != (osm_sr_item_t *) cl_qlist_end(p_list)) { - free(p_sr_item); - p_sr_item = - (osm_sr_item_t *) cl_qlist_remove_head(p_list); - } - - goto Exit; - } -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_service_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Generating response with %u records\n", num_rec); - } - - /* - Get a MAD to reply. Address of Mad is in the received mad_wrapper + /* p923 - The ServiceKey shall be set to 0, except in the case of + a trusted request. + Note: In the mad controller we check that the SM_Key received on + the mad is valid. Meaning - is either zero or equal to the local + sm_key. */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_service_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2402: " - "Unable to allocate MAD\n"); - /* Release the quick pool items */ - p_sr_item = (osm_sr_item_t *) cl_qlist_remove_head(p_list); - while (p_sr_item != (osm_sr_item_t *) cl_qlist_end(p_list)) { - free(p_sr_item); - p_sr_item = - (osm_sr_item_t *) cl_qlist_remove_head(p_list); - } - - goto Exit; + if (!osm_madw_get_sa_mad_ptr(p_madw)->sm_key) { + osm_sr_item_t *item; + for (item = (osm_sr_item_t *) cl_qlist_head(p_list); + item != (osm_sr_item_t *) cl_qlist_end(p_list); + item = (osm_sr_item_t *)cl_qlist_next(&item->list_item)) + memset(item->service_rec.service_key, 0, + sizeof(item->service_rec.service_key)); } - p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); - - /* but what if it was a SET ? setting the response bit is not enough */ - if (p_rcvd_mad->method == IB_MAD_METHOD_SET) - p_resp_sa_mad->method = IB_MAD_METHOD_GET; - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_service_record_t)); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - p_resp_sr = - (ib_service_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - - if (p_resp_sa_mad->method != IB_MAD_METHOD_GETTABLE_RESP && - num_rec == 0) { - p_resp_sa_mad->status = IB_SA_MAD_STATUS_NO_RECORDS; - memset(p_resp_sr, 0, sizeof(*p_resp_sr)); - } else { - /* - p923 - The ServiceKey shall be set to 0, except in the case of a trusted - request. - Note: In the mad controller we check that the SM_Key received on - the mad is valid. Meaning - is either zero or equal to the local - sm_key. - */ - if (p_sa_mad->sm_key == 0) - trusted_req = FALSE; - - p_sr_item = (osm_sr_item_t *) cl_qlist_remove_head(p_list); - - /* we need to track the number of copied items so we can - * stop the copy - but clear them all - */ - num_copied = 0; - while (p_sr_item != (osm_sr_item_t *) cl_qlist_end(p_list)) { - /* Copy the Link Records from the list into the MAD */ - if (num_copied < num_rec) { - *p_resp_sr = p_sr_item->service_rec; - if (trusted_req == FALSE) - memset(p_resp_sr->service_key, 0, - sizeof(p_resp_sr->service_key)); - - num_copied++; - } - free(p_sr_item); - p_resp_sr++; - p_sr_item = - (osm_sr_item_t *) cl_qlist_remove_head(p_list); - } - } - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); - -Exit: - OSM_LOG_EXIT(sa->p_log); + osm_sa_respond(sa, p_madw, sizeof(ib_service_record_t), p_list); } /********************************************************************** @@ -598,7 +465,7 @@ Exit: **********************************************************************/ static void osm_sr_rcv_process_get_method(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { ib_sa_mad_t *p_sa_mad; ib_service_record_t *p_recvd_service_rec; @@ -666,7 +533,7 @@ Exit: **********************************************************************/ static void osm_sr_rcv_process_set_method(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { ib_sa_mad_t *p_sa_mad; ib_service_record_t *p_recvd_service_rec; @@ -776,7 +643,7 @@ Exit: **********************************************************************/ static void osm_sr_rcv_process_delete_method(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw) + IN osm_madw_t * const p_madw) { ib_sa_mad_t *p_sa_mad; ib_service_record_t *p_recvd_service_rec; diff --git a/opensm/opensm/osm_sa_slvl_record.c b/opensm/opensm/osm_sa_slvl_record.c index c56f7eb..ae72623 100644 --- a/opensm/opensm/osm_sa_slvl_record.c +++ b/opensm/opensm/osm_sa_slvl_record.c @@ -227,16 +227,7 @@ void osm_slvl_rec_rcv_process(IN void *ctx, IN void *data) const cl_ptr_vector_t *p_tbl; const osm_port_t *p_port = NULL; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - ib_slvl_table_record_t *p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_slvl_search_ctxt_t context; - osm_slvl_item_t *p_rec_item; ib_api_status_t status = IB_SUCCESS; ib_net64_t comp_mask; osm_physp_t *p_req_physp; @@ -328,129 +319,7 @@ void osm_slvl_rec_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2607: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_slvl_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_slvl_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_slvl_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - - IB_SA_MAD_HDR_SIZE) / sizeof(ib_slvl_table_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (p_rcvd_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * - sizeof(ib_slvl_table_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2605: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_slvl_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_slvl_table_record_t)); - - p_resp_rec = (ib_slvl_table_record_t *) - ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = - (osm_slvl_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_rec = p_rec_item->rec; - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_slvl_table_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_sminfo_record.c b/opensm/opensm/osm_sa_sminfo_record.c index 77eee45..75c50ad 100644 --- a/opensm/opensm/osm_sa_sminfo_record.c +++ b/opensm/opensm/osm_sa_sminfo_record.c @@ -187,16 +187,7 @@ void osm_smir_rcv_process(IN void *ctx, IN void *data) const osm_port_t *p_port = NULL; const ib_sm_info_t *p_smi; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - ib_sminfo_record_t *p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_smir_search_ctxt_t context; - osm_smir_item_t *p_rec_item; ib_api_status_t status = IB_SUCCESS; ib_net64_t comp_mask; ib_net64_t port_guid; @@ -341,129 +332,7 @@ void osm_smir_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (sad_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2808: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = - (osm_smir_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_smir_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_smir_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_sminfo_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (sad_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * sizeof(ib_sminfo_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2807: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_smir_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, sad_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_sminfo_record_t)); - - p_resp_rec = (ib_sminfo_record_t *) - ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = - (osm_smir_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) { - *p_resp_rec = p_rec_item->rec; - p_resp_rec->sm_info.sm_key = 0; - } - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_sminfo_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_sw_info_record.c b/opensm/opensm/osm_sa_sw_info_record.c index c2cb740..bae4c75 100644 --- a/opensm/opensm/osm_sa_sw_info_record.c +++ b/opensm/opensm/osm_sa_sw_info_record.c @@ -245,17 +245,8 @@ void osm_sir_rcv_process(IN void *ctx, IN void *data) osm_madw_t *p_madw = data; const ib_sa_mad_t *sad_mad; const ib_switch_info_record_t *p_rcvd_rec; - ib_switch_info_record_t *p_resp_rec; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_sir_search_ctxt_t context; - osm_sir_item_t *p_rec_item; osm_physp_t *p_req_physp; CL_ASSERT(sa); @@ -310,119 +301,7 @@ void osm_sir_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (sad_mad->method == IB_MAD_METHOD_GET && num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5303: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_sir_item_t *) cl_qlist_remove_head(&rec_list); - while (p_rec_item != (osm_sir_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = - (osm_sir_item_t *) cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = - (MAD_BLOCK_SIZE - - IB_SA_MAD_HDR_SIZE) / sizeof(ib_switch_info_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (sad_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * - sizeof(ib_switch_info_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5306: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = - (osm_sir_item_t *) cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, sad_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_switch_info_record_t)); - - p_resp_rec = (ib_switch_info_record_t *) - ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = (osm_sir_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_rec = p_rec_item->rec; - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_switch_info_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/opensm/osm_sa_vlarb_record.c b/opensm/opensm/osm_sa_vlarb_record.c index a06f8e9..5948cc1 100644 --- a/opensm/opensm/osm_sa_vlarb_record.c +++ b/opensm/opensm/osm_sa_vlarb_record.c @@ -242,16 +242,7 @@ void osm_vlarb_rec_rcv_process(IN void *ctx, IN void *data) const osm_port_t *p_port = NULL; const ib_vl_arb_table_t *p_vl_arb; cl_qlist_t rec_list; - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_resp_sa_mad; - ib_vl_arb_table_record_t *p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; -#ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; -#endif - uint32_t i; osm_vl_arb_search_ctxt_t context; - osm_vl_arb_item_t *p_rec_item; ib_api_status_t status = IB_SUCCESS; ib_net64_t comp_mask; osm_physp_t *p_req_physp; @@ -344,129 +335,7 @@ void osm_vlarb_rec_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); - num_rec = cl_qlist_count(&rec_list); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (sad_mad->method == IB_MAD_METHOD_GET) { - if (num_rec == 0) { - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - if (num_rec > 1) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2A08: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec); - osm_sa_send_error(sa, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_vl_arb_item_t *) - cl_qlist_remove_head(&rec_list); - while (p_rec_item != - (osm_vl_arb_item_t *) cl_qlist_end(&rec_list)) { - free(p_rec_item); - p_rec_item = (osm_vl_arb_item_t *) - cl_qlist_remove_head(&rec_list); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; -#ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = - (MAD_BLOCK_SIZE - - IB_SA_MAD_HDR_SIZE) / sizeof(ib_vl_arb_table_record_t); - if (trim_num_rec < num_rec) { - OSM_LOG(sa->p_log, OSM_LOG_VERBOSE, - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec); - num_rec = trim_num_rec; - } -#endif - - OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Returning %u records\n", num_rec); - - if (sad_mad->method == IB_MAD_METHOD_GET && num_rec == 0) { - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RECORDS); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(sa->p_mad_pool, p_madw->h_bind, - num_rec * - sizeof(ib_vl_arb_table_record_t) + - IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); - - if (!p_resp_madw) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 2A06: " - "osm_mad_pool_get failed\n"); - - for (i = 0; i < num_rec; i++) { - p_rec_item = (osm_vl_arb_item_t *) - cl_qlist_remove_head(&rec_list); - free(p_rec_item); - } - - osm_sa_send_error(sa, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy(p_resp_sa_mad, sad_mad, IB_SA_MAD_HDR_SIZE); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset(sizeof(ib_vl_arb_table_record_t)); - - p_resp_rec = (ib_vl_arb_table_record_t *) - ib_sa_mad_get_payload_ptr(p_resp_sa_mad); - -#ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = - IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | - IB_RMPP_FLAG_ACTIVE; - } -#else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; -#endif - - for (i = 0; i < pre_trim_num_rec; i++) { - p_rec_item = - (osm_vl_arb_item_t *) cl_qlist_remove_head(&rec_list); - /* copy only if not trimmed */ - if (i < num_rec) - *p_resp_rec = p_rec_item->rec; - free(p_rec_item); - p_resp_rec++; - } - - CL_ASSERT(cl_is_qlist_empty(&rec_list)); - - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_respond(sa, p_madw, sizeof(ib_vl_arb_table_record_t), &rec_list); Exit: OSM_LOG_EXIT(sa->p_log); -- 1.5.4.rc2.60.gb2e62 From sashak at voltaire.com Sun Mar 2 12:05:44 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 2 Mar 2008 20:05:44 +0000 Subject: [ofa-general] [PATCH] opensm: rename osm_sa_vendor_send() to osm_sa_send() In-Reply-To: <20080302200505.GF30723@sashak.voltaire.com> References: <20080302200505.GF30723@sashak.voltaire.com> Message-ID: <20080302200544.GG30723@sashak.voltaire.com> Rename osm_sa_vendor_send() to osm_sa_send() (since it is not part of vendor library). Also it changes prototype to match better other SA sender functions. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_sa.h | 17 ++++++----------- opensm/opensm/osm_inform.c | 3 +-- opensm/opensm/osm_sa.c | 21 +++++++++------------ opensm/opensm/osm_sa_class_port_info.c | 2 +- 4 files changed, 17 insertions(+), 26 deletions(-) diff --git a/opensm/include/opensm/osm_sa.h b/opensm/include/opensm/osm_sa.h index f4f751b..370e4e0 100644 --- a/opensm/include/opensm/osm_sa.h +++ b/opensm/include/opensm/osm_sa.h @@ -351,20 +351,17 @@ osm_sa_bind(IN osm_sa_t * const p_sa, IN const ib_net64_t port_guid); * SEE ALSO *********/ -/****f* OpenSM: SA/osm_sa_vendor_send +/****f* OpenSM: SA/osm_sa_send * NAME -* osm_sa_vendor_send +* osm_sa_send * * DESCRIPTION * Sends SA MAD via osm_vendor_send and maintains the QP1 sent statistic * * SYNOPSIS */ -ib_api_status_t -osm_sa_vendor_send(IN osm_bind_handle_t h_bind, - IN osm_madw_t * const p_madw, - IN boolean_t const resp_expected, - IN osm_subn_t * const p_subn); +ib_api_status_t osm_sa_send(osm_sa_t *sa, IN osm_madw_t * const p_madw, + IN boolean_t const resp_expected); /****f* IBA Base: Types/osm_sa_send_error * NAME @@ -376,10 +373,8 @@ osm_sa_vendor_send(IN osm_bind_handle_t h_bind, * * SYNOPSIS */ -void -osm_sa_send_error(IN osm_sa_t * sa, - IN const osm_madw_t * const p_madw, - IN const ib_net16_t sa_status); +void osm_sa_send_error(IN osm_sa_t * sa, IN const osm_madw_t * const p_madw, + IN const ib_net16_t sa_status); /* * PARAMETERS * sa diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index bbd573c..9553f7f 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -365,8 +365,7 @@ static ib_api_status_t __osm_send_report(IN osm_infr_t * p_infr_rec, /* the info *p_report_ntc = *p_ntc; /* The TRUE is for: response is expected */ - osm_sa_vendor_send(p_report_madw->h_bind, p_report_madw, TRUE, - p_infr_rec->sa->p_subn); + osm_sa_send(p_infr_rec->sa, p_report_madw, TRUE); Exit: OSM_LOG_EXIT(p_log); diff --git a/opensm/opensm/osm_sa.c b/opensm/opensm/osm_sa.c index 4edce47..d85463e 100644 --- a/opensm/opensm/osm_sa.c +++ b/opensm/opensm/osm_sa.c @@ -318,19 +318,17 @@ Exit: return (status); } -ib_api_status_t -osm_sa_vendor_send(IN osm_bind_handle_t h_bind, - IN osm_madw_t * const p_madw, - IN boolean_t const resp_expected, - IN osm_subn_t * const p_subn) +ib_api_status_t osm_sa_send(osm_sa_t *sa, + IN osm_madw_t * const p_madw, + IN boolean_t const resp_expected) { ib_api_status_t status; - cl_atomic_inc(&p_subn->p_osm->stats.sa_mads_sent); - status = osm_vendor_send(h_bind, p_madw, resp_expected); + cl_atomic_inc(&sa->p_subn->p_osm->stats.sa_mads_sent); + status = osm_vendor_send(p_madw->h_bind, p_madw, resp_expected); if (status != IB_SUCCESS) { - cl_atomic_dec(&p_subn->p_osm->stats.sa_mads_sent); - OSM_LOG(&p_subn->p_osm->log, OSM_LOG_ERROR, "ERR 4C04: " + cl_atomic_dec(&sa->p_subn->p_osm->stats.sa_mads_sent); + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 4C04: " "osm_vendor_send failed, status = %s\n", ib_get_err_str(status)); } @@ -392,8 +390,7 @@ osm_sa_send_error(IN osm_sa_t * sa, if (osm_log_is_active(sa->p_log, OSM_LOG_FRAMES)) osm_dump_sa_mad(sa->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); - osm_sa_vendor_send(osm_madw_get_bind_handle(p_resp_madw), - p_resp_madw, FALSE, sa->p_subn); + osm_sa_send(sa, p_resp_madw, FALSE); Exit: OSM_LOG_EXIT(sa->p_log); @@ -501,7 +498,7 @@ void osm_sa_respond(osm_sa_t *sa, osm_madw_t *madw, size_t attr_size, p += attr_size; } - osm_sa_vendor_send(resp_madw->h_bind, resp_madw, FALSE, sa->p_subn); + osm_sa_send(sa, resp_madw, FALSE); osm_dump_sa_mad(sa->p_log, resp_sa_mad, OSM_LOG_FRAMES); Exit: diff --git a/opensm/opensm/osm_sa_class_port_info.c b/opensm/opensm/osm_sa_class_port_info.c index 3a76a69..f0afb32 100644 --- a/opensm/opensm/osm_sa_class_port_info.c +++ b/opensm/opensm/osm_sa_class_port_info.c @@ -174,7 +174,7 @@ __osm_cpi_rcv_respond(IN osm_sa_t * sa, if (osm_log_is_active(sa->p_log, OSM_LOG_FRAMES)) osm_dump_sa_mad(sa->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); - osm_sa_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE, sa->p_subn); + osm_sa_send(sa, p_resp_madw, FALSE); Exit: OSM_LOG_EXIT(sa->p_log); -- 1.5.4.rc2.60.gb2e62 From sashak at voltaire.com Sun Mar 2 12:07:24 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 2 Mar 2008 20:07:24 +0000 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <20080302200544.GG30723@sashak.voltaire.com> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> Message-ID: <20080302200724.GH30723@sashak.voltaire.com> IBA 1.2.1 clarifies (t.187, p.897) that SA Attribute offset shell be set to zero if zero attributes are returned. Fix this. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_sa.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/opensm/opensm/osm_sa.c b/opensm/opensm/osm_sa.c index d85463e..46c5bf7 100644 --- a/opensm/opensm/osm_sa.c +++ b/opensm/opensm/osm_sa.c @@ -372,6 +372,8 @@ osm_sa_send_error(IN osm_sa_t * sa, if (p_resp_sa_mad->method == IB_MAD_METHOD_SET) p_resp_sa_mad->method = IB_MAD_METHOD_GET; + else if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE) + p_resp_sa_mad->attr_offset = 0; p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; @@ -473,7 +475,7 @@ void osm_sa_respond(osm_sa_t *sa, osm_madw_t *madw, size_t attr_size, resp_sa_mad->sm_key = 0; /* Fill in the offset (paylen will be done by the rmpp SAR) */ - resp_sa_mad->attr_offset = ib_get_attr_offset(attr_size); + resp_sa_mad->attr_offset = num_rec ? ib_get_attr_offset(attr_size) : 0; p = ib_sa_mad_get_payload_ptr(resp_sa_mad); -- 1.5.4.rc2.60.gb2e62 From kliteyn at mellanox.co.il Sun Mar 2 14:21:36 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 03 Mar 2008 00:21:36 +0200 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <20080302200724.GH30723@sashak.voltaire.com> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> Message-ID: <47CB2870.50602@mellanox.co.il> Sasha Khapyorsky wrote: > IBA 1.2.1 clarifies (t.187, p.897) that SA Attribute offset shell be set > to zero if zero attributes are returned. Fix this. > Nice catch, thanks. BTW, are you aware of any other IBA 1.2.1 - related issues that need to be fixed? I mean, is OpenSM fully IBA 1.2.1 compliant? -- Yevgeny > Signed-off-by: Sasha Khapyorsky > --- > opensm/opensm/osm_sa.c | 4 +++- > 1 files changed, 3 insertions(+), 1 deletions(-) > > diff --git a/opensm/opensm/osm_sa.c b/opensm/opensm/osm_sa.c > index d85463e..46c5bf7 100644 > --- a/opensm/opensm/osm_sa.c > +++ b/opensm/opensm/osm_sa.c > @@ -372,6 +372,8 @@ osm_sa_send_error(IN osm_sa_t * sa, > > if (p_resp_sa_mad->method == IB_MAD_METHOD_SET) > p_resp_sa_mad->method = IB_MAD_METHOD_GET; > + else if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE) > + p_resp_sa_mad->attr_offset = 0; > > p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; > > @@ -473,7 +475,7 @@ void osm_sa_respond(osm_sa_t *sa, osm_madw_t *madw, size_t attr_size, > resp_sa_mad->sm_key = 0; > > /* Fill in the offset (paylen will be done by the rmpp SAR) */ > - resp_sa_mad->attr_offset = ib_get_attr_offset(attr_size); > + resp_sa_mad->attr_offset = num_rec ? ib_get_attr_offset(attr_size) : 0; > > p = ib_sa_mad_get_payload_ptr(resp_sa_mad); > > From tom at opengridcomputing.com Sun Mar 2 16:48:11 2008 From: tom at opengridcomputing.com (Tom Tucker) Date: Sun, 02 Mar 2008 18:48:11 -0600 Subject: [nfs-rdma-devel] [ofa-general] Status of NFS-RDMA ? (fwd) In-Reply-To: <47C7C261.2060007@hamburgnet.de> References: <20080206154701.GA11384@cefeid.wcss.wroc.pl> <1202320491.14810.29.camel@trinity.ogc.int> <20080206213543.GA21176@cefeid.wcss.wroc.pl> <20080209221711.GB20332@cefeid.wcss.wroc.pl> <47C7C261.2060007@hamburgnet.de> Message-ID: <1204505291.1567.15.camel@trinity.ogc.int> On Fri, 2008-02-29 at 09:29 +0100, Sebastian Schmitzdorff wrote: > hi pawel, > > I was wondering if you have achieved better nfs rdma benchmark results > by now? Pawel: What is your network hardware setup? Thanks, Tom > > regards > Sebastian > > Pawel Dziekonski schrieb: > > hi, > > > > the saga continues. ;) > > > > very basic benchmarks and surprising (at least for me) results - it > > look's like reading is much slower than writing and NFS/RDMA is twice > > slower in reading than classic NFS. :o > > > > results below - comments appreciated! > > regards, Pawel > > > > > > both nfs server and client have 8-cores, 16 GB RAM, Mellanox DDR HCAs > > (MT25204) connected port-port (no switch). > > > > local_hdd - 2 sata2 disks in soft-raid0, > > nfs_ipoeth - classic nfs over ethernet, > > nfs_ipoib - classic nfs over IPoIB, > > nfs_rdma - NFS/RDMA. > > > > simple write of 36GB file with dd (both machines have 16GB RAM): > > /usr/bin/time -p dd if=/dev/zero of=/mnt/qqq bs=1M count=36000 > > > > local_hdd sys 54.52 user 0.04 real 254.59 > > > > nfs_ipoib sys 36.35 user 0.00 real 266.63 > > nfs_rdma sys 39.03 user 0.02 real 323.77 > > nfs_ipoeth sys 34.21 user 0.01 real 375.24 > > > > remount /mnt to clear cache and read a file from nfs share and > > write it to /dev/: > > /usr/bin/time -p dd if=/mnt/qqq of=/scratch/qqq bs=1M > > > > nfs_ipoib sys 59.04 user 0.02 real 571.57 > > nfs_ipoeth sys 58.92 user 0.02 real 606.61 > > nfs_rdma sys 62.57 user 0.03 real 1296.36 > > > > > > > > results from bonnie++: > > > > Version 1.03c ------Sequential Write ------ --Sequential Read -- --Random- > > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > > local_hdd 35G:128k 93353 12 58329 6 143293 7 243.6 1 > > local_hdd 35G:256k 92283 11 58189 6 144202 8 172.2 2 > > local_hdd 35G:512k 93879 12 57715 6 144167 8 128.2 4 > > local_hdd 35G:1024k 93075 12 58637 6 144172 8 95.3 7 > > nfs_ipoeth 35G:128k 91325 7 31848 4 64299 4 170.2 1 > > nfs_ipoeth 35G:256k 90668 7 32036 5 64542 4 163.2 2 > > nfs_ipoeth 35G:512k 93348 7 31757 5 64454 4 85.7 3 > > nfs_ipoet 35G:1024k 91283 7 31869 5 64241 5 51.7 4 > > nfs_ipoib 35G:128k 91733 7 36641 5 65839 4 178.4 2 > > nfs_ipoib 35G:256k 92453 7 36567 6 66682 4 166.9 3 > > nfs_ipoib 35G:512k 91157 7 37660 6 66318 4 86.8 3 > > nfs_ipoib 35G:1024k 92111 7 35786 6 66277 5 53.3 4 > > nfs_rdma 35G:128k 91152 8 29942 5 32147 2 187.0 1 > > nfs_rdma 35G:256k 89772 7 30560 5 34587 2 158.4 3 > > nfs_rdma 35G:512k 91290 7 29698 5 34277 2 60.9 2 > > nfs_rdma 35G:1024k 91336 8 29052 5 31742 2 41.5 3 > > ------Sequential Create------ --------Random Create-------- > > -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > > files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > > local_hdd 16 10587 36 +++++ +++ 8674 29 10727 35 +++++ +++ 7015 28 > > local_hdd 16 11372 41 +++++ +++ 8490 29 11192 43 +++++ +++ 6881 27 > > local_hdd 16 10789 35 +++++ +++ 8520 29 11468 46 +++++ +++ 6651 24 > > local_hdd 16 10841 40 +++++ +++ 8443 28 11162 41 +++++ +++ 6441 22 > > nfs_ipoeth 16 3753 7 13390 12 3795 7 3773 8 22181 16 3635 7 > > nfs_ipoeth 16 3762 8 12358 7 3713 8 3753 7 20448 13 3632 6 > > nfs_ipoeth 16 3834 7 12697 6 3729 8 3725 9 22807 11 3673 7 > > nfs_ipoeth 16 3729 8 14260 10 3774 7 3744 7 25285 14 3688 7 > > nfs_ipoib 16 6803 17 +++++ +++ 6843 15 6820 14 +++++ +++ 5834 11 > > nfs_ipoib 16 6587 16 +++++ +++ 4959 9 6832 14 +++++ +++ 5608 12 > > nfs_ipoib 16 6820 18 +++++ +++ 6636 15 6479 15 +++++ +++ 5679 13 > > nfs_ipoib 16 6475 14 +++++ +++ 6435 14 5543 11 +++++ +++ 5431 11 > > nfs_rdma 16 7014 15 +++++ +++ 6714 10 7001 14 +++++ +++ 5683 8 > > nfs_rdma 16 7038 13 +++++ +++ 6713 12 6956 11 +++++ +++ 5488 8 > > nfs_rdma 16 7058 12 +++++ +++ 6797 11 6989 14 +++++ +++ 5761 9 > > nfs_rdma 16 7201 13 +++++ +++ 6821 12 7072 15 +++++ +++ 5609 9 > > > > > > > > From tom at opengridcomputing.com Sun Mar 2 16:48:11 2008 From: tom at opengridcomputing.com (Tom Tucker) Date: Sun, 02 Mar 2008 18:48:11 -0600 Subject: [nfs-rdma-devel] [ofa-general] Status of NFS-RDMA ? (fwd) In-Reply-To: <47C7C261.2060007@hamburgnet.de> References: <20080206154701.GA11384@cefeid.wcss.wroc.pl> <1202320491.14810.29.camel@trinity.ogc.int> <20080206213543.GA21176@cefeid.wcss.wroc.pl> <20080209221711.GB20332@cefeid.wcss.wroc.pl> <47C7C261.2060007@hamburgnet.de> Message-ID: <1204505291.1567.15.camel@trinity.ogc.int> On Fri, 2008-02-29 at 09:29 +0100, Sebastian Schmitzdorff wrote: > hi pawel, > > I was wondering if you have achieved better nfs rdma benchmark results > by now? Pawel: What is your network hardware setup? Thanks, Tom > > regards > Sebastian > > Pawel Dziekonski schrieb: > > hi, > > > > the saga continues. ;) > > > > very basic benchmarks and surprising (at least for me) results - it > > look's like reading is much slower than writing and NFS/RDMA is twice > > slower in reading than classic NFS. :o > > > > results below - comments appreciated! > > regards, Pawel > > > > > > both nfs server and client have 8-cores, 16 GB RAM, Mellanox DDR HCAs > > (MT25204) connected port-port (no switch). > > > > local_hdd - 2 sata2 disks in soft-raid0, > > nfs_ipoeth - classic nfs over ethernet, > > nfs_ipoib - classic nfs over IPoIB, > > nfs_rdma - NFS/RDMA. > > > > simple write of 36GB file with dd (both machines have 16GB RAM): > > /usr/bin/time -p dd if=/dev/zero of=/mnt/qqq bs=1M count=36000 > > > > local_hdd sys 54.52 user 0.04 real 254.59 > > > > nfs_ipoib sys 36.35 user 0.00 real 266.63 > > nfs_rdma sys 39.03 user 0.02 real 323.77 > > nfs_ipoeth sys 34.21 user 0.01 real 375.24 > > > > remount /mnt to clear cache and read a file from nfs share and > > write it to /dev/: > > /usr/bin/time -p dd if=/mnt/qqq of=/scratch/qqq bs=1M > > > > nfs_ipoib sys 59.04 user 0.02 real 571.57 > > nfs_ipoeth sys 58.92 user 0.02 real 606.61 > > nfs_rdma sys 62.57 user 0.03 real 1296.36 > > > > > > > > results from bonnie++: > > > > Version 1.03c ------Sequential Write ------ --Sequential Read -- --Random- > > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > > local_hdd 35G:128k 93353 12 58329 6 143293 7 243.6 1 > > local_hdd 35G:256k 92283 11 58189 6 144202 8 172.2 2 > > local_hdd 35G:512k 93879 12 57715 6 144167 8 128.2 4 > > local_hdd 35G:1024k 93075 12 58637 6 144172 8 95.3 7 > > nfs_ipoeth 35G:128k 91325 7 31848 4 64299 4 170.2 1 > > nfs_ipoeth 35G:256k 90668 7 32036 5 64542 4 163.2 2 > > nfs_ipoeth 35G:512k 93348 7 31757 5 64454 4 85.7 3 > > nfs_ipoet 35G:1024k 91283 7 31869 5 64241 5 51.7 4 > > nfs_ipoib 35G:128k 91733 7 36641 5 65839 4 178.4 2 > > nfs_ipoib 35G:256k 92453 7 36567 6 66682 4 166.9 3 > > nfs_ipoib 35G:512k 91157 7 37660 6 66318 4 86.8 3 > > nfs_ipoib 35G:1024k 92111 7 35786 6 66277 5 53.3 4 > > nfs_rdma 35G:128k 91152 8 29942 5 32147 2 187.0 1 > > nfs_rdma 35G:256k 89772 7 30560 5 34587 2 158.4 3 > > nfs_rdma 35G:512k 91290 7 29698 5 34277 2 60.9 2 > > nfs_rdma 35G:1024k 91336 8 29052 5 31742 2 41.5 3 > > ------Sequential Create------ --------Random Create-------- > > -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > > files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > > local_hdd 16 10587 36 +++++ +++ 8674 29 10727 35 +++++ +++ 7015 28 > > local_hdd 16 11372 41 +++++ +++ 8490 29 11192 43 +++++ +++ 6881 27 > > local_hdd 16 10789 35 +++++ +++ 8520 29 11468 46 +++++ +++ 6651 24 > > local_hdd 16 10841 40 +++++ +++ 8443 28 11162 41 +++++ +++ 6441 22 > > nfs_ipoeth 16 3753 7 13390 12 3795 7 3773 8 22181 16 3635 7 > > nfs_ipoeth 16 3762 8 12358 7 3713 8 3753 7 20448 13 3632 6 > > nfs_ipoeth 16 3834 7 12697 6 3729 8 3725 9 22807 11 3673 7 > > nfs_ipoeth 16 3729 8 14260 10 3774 7 3744 7 25285 14 3688 7 > > nfs_ipoib 16 6803 17 +++++ +++ 6843 15 6820 14 +++++ +++ 5834 11 > > nfs_ipoib 16 6587 16 +++++ +++ 4959 9 6832 14 +++++ +++ 5608 12 > > nfs_ipoib 16 6820 18 +++++ +++ 6636 15 6479 15 +++++ +++ 5679 13 > > nfs_ipoib 16 6475 14 +++++ +++ 6435 14 5543 11 +++++ +++ 5431 11 > > nfs_rdma 16 7014 15 +++++ +++ 6714 10 7001 14 +++++ +++ 5683 8 > > nfs_rdma 16 7038 13 +++++ +++ 6713 12 6956 11 +++++ +++ 5488 8 > > nfs_rdma 16 7058 12 +++++ +++ 6797 11 6989 14 +++++ +++ 5761 9 > > nfs_rdma 16 7201 13 +++++ +++ 6821 12 7072 15 +++++ +++ 5609 9 > > > > > > > > From npiggin at suse.de Sun Mar 2 19:29:34 2008 From: npiggin at suse.de (Nick Piggin) Date: Mon, 3 Mar 2008 04:29:34 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080302155457.GK8091@v2.random> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> Message-ID: <20080303032934.GA3301@wotan.suse.de> On Sun, Mar 02, 2008 at 04:54:57PM +0100, Andrea Arcangeli wrote: > Difference between #v7 and #v8: > > 1) s/age_page/clear_flush_young/ (Nick's suggestion) > 2) macro fix (Andrew) > 3) move release before final unmap_vmas (for GRU, Jack/Christoph) > 4) microoptimize mmu_notifier_unregister (Christoph) > 5) use mmap_sem for registration serialization (Christoph) > > The (void)xxx in macros doesn't work with "args". Christoph's solution > look best in avoiding warnings, even if it forces to make the mmu > notifier operation structure visible even if MMU_NOTIFIER=n (that's > the only downside). I have a couple of "cleanup" patches that change the structure of this to something I prefer. Others may not, but I'll post them for debate anyway. > I didn't drop invalidate_page, because invalidate_range_begin/end > would be slower for usages like KVM/GRU (we don't need a begin/end > there because where invalidate_page is called, the VM holds a > reference on the page). do_wp_page should also use invalidate_page > since it can free the page after dropping the PT lock without losing > any performance (that's not true for the places where invalidate_range > is called). I'm still not completely happy with this. I had a very quick look at the GRU driver, but I don't see why it can't be implemented more like the regular TLB model, and have TLB insertions depend on the linux pte, and do invalidates _after_ restricting permissions to the pte. Ie. I'd still like to get rid of invalidate_range_begin, and get rid of invalidate calls from places where permissions are relaxed. > It'd be nice if everyone involved can agree to converge on this API > for .25. KVM/GRU (and perhaps Quadrics) and similar usages will be > fully covered in .25. If we can agree on the API, then I don't see any reason why it can't go into 2.6.25, unless someome wants more time to review it (but 2.6.25 release should be quite far away still so there should be quite a bit of time). From npiggin at suse.de Sun Mar 2 19:33:54 2008 From: npiggin at suse.de (Nick Piggin) Date: Mon, 3 Mar 2008 04:33:54 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080302155457.GK8091@v2.random> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> Message-ID: <20080303033354.GC3301@wotan.suse.de> On Sun, Mar 02, 2008 at 04:54:57PM +0100, Andrea Arcangeli wrote: > Difference between #v7 and #v8: [patch] mmu-v8: demacro Remove the macros from mmu_notifier.h, in favour of functions. This requires untangling the include order circular dependencies as well, so just remove struct mmu_notifier_head in favour of just using the hlist in mm_struct. Signed-off-by: Nick Piggin --- Index: linux-2.6/include/linux/mmu_notifier.h =================================================================== --- linux-2.6.orig/include/linux/mmu_notifier.h +++ linux-2.6/include/linux/mmu_notifier.h @@ -55,12 +55,13 @@ struct mmu_notifier { #ifdef CONFIG_MMU_NOTIFIER -struct mmu_notifier_head { - struct hlist_head head; -}; - #include +static inline int mm_has_notifiers(struct mm_struct *mm) +{ + return unlikely(!hlist_empty(&mm->mmu_notifier_list)); +} + /* * Must hold the mmap_sem for write. * @@ -79,33 +80,59 @@ extern void mmu_notifier_register(struct */ extern void mmu_notifier_unregister(struct mmu_notifier *mn, struct mm_struct *mm); -extern void mmu_notifier_release(struct mm_struct *mm); -extern int mmu_notifier_clear_flush_young(struct mm_struct *mm, + +extern void __mmu_notifier_release(struct mm_struct *mm); +extern int __mmu_notifier_clear_flush_young(struct mm_struct *mm, unsigned long address); +extern void __mmu_notifier_invalidate_page(struct mm_struct *mm, + unsigned long address); +extern void __mmu_notifier_invalidate_range_begin(struct mm_struct *mm, + unsigned long start, unsigned long end); +extern void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, + unsigned long start, unsigned long end); + + +static inline void mmu_notifier_release(struct mm_struct *mm) +{ + if (mm_has_notifiers(mm)) + __mmu_notifier_release(mm); +} + +static inline int mmu_notifier_clear_flush_young(struct mm_struct *mm, + unsigned long address) +{ + if (mm_has_notifiers(mm)) + return __mmu_notifier_clear_flush_young(mm, address); + return 0; +} + +static inline void mmu_notifier_invalidate_page(struct mm_struct *mm, + unsigned long address) +{ + if (mm_has_notifiers(mm)) + __mmu_notifier_invalidate_page(mm, address); +} + +static inline void mmu_notifier_invalidate_range_begin(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + if (mm_has_notifiers(mm)) + __mmu_notifier_invalidate_range_begin(mm, start, end); +} + +static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + if (mm_has_notifiers(mm)) + __mmu_notifier_invalidate_range_end(mm, start, end); +} -static inline void mmu_notifier_head_init(struct mmu_notifier_head *mnh) +static inline void mmu_notifier_mm_init(struct mm_struct *mm) { - INIT_HLIST_HEAD(&mnh->head); + INIT_HLIST_HEAD(&mm->mmu_notifier_list); } -#define mmu_notifier(function, mm, args...) \ - do { \ - struct mmu_notifier *__mn; \ - struct hlist_node *__n; \ - struct mm_struct * __mm = mm; \ - \ - if (unlikely(!hlist_empty(&__mm->mmu_notifier.head))) { \ - rcu_read_lock(); \ - hlist_for_each_entry_rcu(__mn, __n, \ - &__mm->mmu_notifier.head, \ - hlist) \ - if (__mn->ops->function) \ - __mn->ops->function(__mn, \ - __mm, \ - args); \ - rcu_read_unlock(); \ - } \ - } while (0) + #define ptep_clear_flush_notify(__vma, __address, __ptep) \ ({ \ @@ -113,7 +140,7 @@ static inline void mmu_notifier_head_ini struct vm_area_struct * ___vma = __vma; \ unsigned long ___address = __address; \ __pte = ptep_clear_flush(___vma, ___address, __ptep); \ - mmu_notifier(invalidate_page, ___vma->vm_mm, ___address); \ + mmu_notifier_invalidate_page(___vma->vm_mm, ___address); \ __pte; \ }) @@ -130,28 +157,34 @@ static inline void mmu_notifier_head_ini #else /* CONFIG_MMU_NOTIFIER */ -struct mmu_notifier_head {}; +static inline void mmu_notifier_release(struct mm_struct *mm) +{ +} -#define mmu_notifier_register(mn, mm) do {} while(0) -#define mmu_notifier_unregister(mn, mm) do {} while (0) -#define mmu_notifier_release(mm) do {} while (0) -#define mmu_notifier_head_init(mmh) do {} while (0) +static inline int mmu_notifier_clear_flush_young(struct mm_struct *mm, + unsigned long address) +{ + return 0; +} -/* - * Notifiers that use the parameters that they were passed so that the - * compiler does not complain about unused variables but does proper - * parameter checks even if !CONFIG_MMU_NOTIFIER. - * Macros generate no code. - */ -#define mmu_notifier(function, mm, args...) \ - do { \ - if (0) { \ - struct mmu_notifier *__mn; \ - \ - __mn = (struct mmu_notifier *)(0x00ff); \ - __mn->ops->function(__mn, mm, args); \ - }; \ - } while (0) +static inline void mmu_notifier_invalidate_page(struct mm_struct *mm, + unsigned long address) +{ +} + +static inline void mmu_notifier_invalidate_range_begin(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ +} + +static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ +} + +static inline void mmu_notifier_mm_init(struct mm_struct *mm) +{ +} #define ptep_clear_flush_young_notify ptep_clear_flush_young #define ptep_clear_flush_notify ptep_clear_flush Index: linux-2.6/mm/mmu_notifier.c =================================================================== --- linux-2.6.orig/mm/mmu_notifier.c +++ linux-2.6/mm/mmu_notifier.c @@ -17,12 +17,12 @@ * No synchronization. This function can only be called when only a single * process remains that performs teardown. */ -void mmu_notifier_release(struct mm_struct *mm) +void __mmu_notifier_release(struct mm_struct *mm) { struct mmu_notifier *mn; - while (unlikely(!hlist_empty(&mm->mmu_notifier.head))) { - mn = hlist_entry(mm->mmu_notifier.head.first, + while (unlikely(!hlist_empty(&mm->mmu_notifier_list))) { + mn = hlist_entry(mm->mmu_notifier_list.first, struct mmu_notifier, hlist); hlist_del(&mn->hlist); @@ -32,30 +32,69 @@ void mmu_notifier_release(struct mm_stru } /* - * If no young bitflag is supported by the hardware, ->age_page can + * If no young bitflag is supported by the hardware, ->clear_flush_young can * unmap the address and return 1 or 0 depending if the mapping previously * existed or not. */ -int mmu_notifier_clear_flush_young(struct mm_struct *mm, unsigned long address) +int __mmu_notifier_clear_flush_young(struct mm_struct *mm, + unsigned long address) { struct mmu_notifier *mn; struct hlist_node *n; int young = 0; - if (unlikely(!hlist_empty(&mm->mmu_notifier.head))) { - rcu_read_lock(); - hlist_for_each_entry_rcu(mn, n, - &mm->mmu_notifier.head, hlist) { - if (mn->ops->clear_flush_young) - young |= mn->ops->clear_flush_young(mn, mm, - address); - } - rcu_read_unlock(); + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { + if (mn->ops->clear_flush_young) + young |= mn->ops->clear_flush_young(mn, mm, address); } + rcu_read_unlock(); return young; } +void __mmu_notifier_invalidate_page(struct mm_struct *mm, + unsigned long address) +{ + struct mmu_notifier *mn; + struct hlist_node *n; + + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { + if (mn->ops->invalidate_page) + mn->ops->invalidate_page(mn, mm, address); + } + rcu_read_unlock(); +} + +void __mmu_notifier_invalidate_range_begin(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + struct mmu_notifier *mn; + struct hlist_node *n; + + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { + if (mn->ops->invalidate_range_begin) + mn->ops->invalidate_range_begin(mn, mm, start, end); + } + rcu_read_unlock(); +} + +void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + struct mmu_notifier *mn; + struct hlist_node *n; + + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { + if (mn->ops->invalidate_range_end) + mn->ops->invalidate_range_end(mn, mm, start, end); + } + rcu_read_unlock(); +} + /* * Note that all notifiers use RCU. The updates are only guaranteed to * be visible to other processes after a RCU quiescent period! @@ -64,7 +103,7 @@ int mmu_notifier_clear_flush_young(struc */ void mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm) { - hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier.head); + hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_list); } EXPORT_SYMBOL_GPL(mmu_notifier_register); Index: linux-2.6/mm/fremap.c =================================================================== --- linux-2.6.orig/mm/fremap.c +++ linux-2.6/mm/fremap.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include @@ -214,9 +215,9 @@ asmlinkage long sys_remap_file_pages(uns spin_unlock(&mapping->i_mmap_lock); } - mmu_notifier(invalidate_range_begin, mm, start, start + size); + mmu_notifier_invalidate_range_begin(mm, start, start + size); err = populate_range(mm, vma, start, size, pgoff); - mmu_notifier(invalidate_range_end, mm, start, start + size); + mmu_notifier_invalidate_range_end(mm, start, start + size); if (!err && !(flags & MAP_NONBLOCK)) { if (unlikely(has_write_lock)) { downgrade_write(&mm->mmap_sem); Index: linux-2.6/mm/hugetlb.c =================================================================== --- linux-2.6.orig/mm/hugetlb.c +++ linux-2.6/mm/hugetlb.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -755,7 +756,7 @@ void __unmap_hugepage_range(struct vm_ar BUG_ON(start & ~HPAGE_MASK); BUG_ON(end & ~HPAGE_MASK); - mmu_notifier(invalidate_range_begin, mm, start, end); + mmu_notifier_invalidate_range_begin(mm, start, end); spin_lock(&mm->page_table_lock); for (address = start; address < end; address += HPAGE_SIZE) { ptep = huge_pte_offset(mm, address); @@ -776,7 +777,7 @@ void __unmap_hugepage_range(struct vm_ar } spin_unlock(&mm->page_table_lock); flush_tlb_range(vma, start, end); - mmu_notifier(invalidate_range_end, mm, start, end); + mmu_notifier_invalidate_range_end(mm, start, end); list_for_each_entry_safe(page, tmp, &page_list, lru) { list_del(&page->lru); put_page(page); Index: linux-2.6/mm/memory.c =================================================================== --- linux-2.6.orig/mm/memory.c +++ linux-2.6/mm/memory.c @@ -51,6 +51,7 @@ #include #include #include +#include #include #include @@ -612,7 +613,7 @@ int copy_page_range(struct mm_struct *ds return copy_hugetlb_page_range(dst_mm, src_mm, vma); if (is_cow_mapping(vma->vm_flags)) - mmu_notifier(invalidate_range_begin, src_mm, addr, end); + mmu_notifier_invalidate_range_begin(src_mm, addr, end); dst_pgd = pgd_offset(dst_mm, addr); src_pgd = pgd_offset(src_mm, addr); @@ -626,7 +627,7 @@ int copy_page_range(struct mm_struct *ds } while (dst_pgd++, src_pgd++, addr = next, addr != end); if (is_cow_mapping(vma->vm_flags)) - mmu_notifier(invalidate_range_end, src_mm, + mmu_notifier_invalidate_range_end(src_mm, vma->vm_start, end); return 0; @@ -905,9 +906,9 @@ unsigned long zap_page_range(struct vm_a lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); - mmu_notifier(invalidate_range_begin, mm, address, end); + mmu_notifier_invalidate_range_begin(mm, address, end); end = unmap_vmas(&tlb, vma, address, end, &nr_accounted, details); - mmu_notifier(invalidate_range_end, mm, address, end); + mmu_notifier_invalidate_range_end(mm, address, end); if (tlb) tlb_finish_mmu(tlb, address, end); return end; @@ -1477,7 +1478,7 @@ int apply_to_page_range(struct mm_struct int err; BUG_ON(addr >= end); - mmu_notifier(invalidate_range_begin, mm, start, end); + mmu_notifier_invalidate_range_begin(mm, start, end); pgd = pgd_offset(mm, addr); do { next = pgd_addr_end(addr, end); @@ -1485,7 +1486,7 @@ int apply_to_page_range(struct mm_struct if (err) break; } while (pgd++, addr = next, addr != end); - mmu_notifier(invalidate_range_end, mm, start, end); + mmu_notifier_invalidate_range_end(mm, start, end); return err; } EXPORT_SYMBOL_GPL(apply_to_page_range); Index: linux-2.6/mm/mmap.c =================================================================== --- linux-2.6.orig/mm/mmap.c +++ linux-2.6/mm/mmap.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -1747,13 +1748,13 @@ static void unmap_region(struct mm_struc lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); - mmu_notifier(invalidate_range_begin, mm, start, end); + mmu_notifier_invalidate_range_begin(mm, start, end); unmap_vmas(&tlb, vma, start, end, &nr_accounted, NULL); vm_unacct_memory(nr_accounted); free_pgtables(&tlb, vma, prev? prev->vm_end: FIRST_USER_ADDRESS, next? next->vm_start: 0); tlb_finish_mmu(tlb, start, end); - mmu_notifier(invalidate_range_end, mm, start, end); + mmu_notifier_invalidate_range_end(mm, start, end); } /* Index: linux-2.6/mm/mprotect.c =================================================================== --- linux-2.6.orig/mm/mprotect.c +++ linux-2.6/mm/mprotect.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -198,12 +199,12 @@ success: dirty_accountable = 1; } - mmu_notifier(invalidate_range_begin, mm, start, end); + mmu_notifier_invalidate_range_begin(mm, start, end); if (is_vm_hugetlb_page(vma)) hugetlb_change_protection(vma, start, end, vma->vm_page_prot); else change_protection(vma, start, end, vma->vm_page_prot, dirty_accountable); - mmu_notifier(invalidate_range_end, mm, start, end); + mmu_notifier_invalidate_range_end(mm, start, end); vm_stat_account(mm, oldflags, vma->vm_file, -nrpages); vm_stat_account(mm, newflags, vma->vm_file, nrpages); return 0; Index: linux-2.6/mm/mremap.c =================================================================== --- linux-2.6.orig/mm/mremap.c +++ linux-2.6/mm/mremap.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -102,7 +103,7 @@ static void move_ptes(struct vm_area_str arch_enter_lazy_mmu_mode(); old_start = old_addr; - mmu_notifier(invalidate_range_begin, vma->vm_mm, + mmu_notifier_invalidate_range_begin(vma->vm_mm, old_start, old_end); for (; old_addr < old_end; old_pte++, old_addr += PAGE_SIZE, new_pte++, new_addr += PAGE_SIZE) { @@ -112,7 +113,7 @@ static void move_ptes(struct vm_area_str pte = move_pte(pte, new_vma->vm_page_prot, old_addr, new_addr); set_pte_at(mm, new_addr, new_pte, pte); } - mmu_notifier(invalidate_range_end, vma->vm_mm, old_start, old_end); + mmu_notifier_invalidate_range_end(vma->vm_mm, old_start, old_end); arch_leave_lazy_mmu_mode(); if (new_ptl != old_ptl) Index: linux-2.6/include/linux/mm_types.h =================================================================== --- linux-2.6.orig/include/linux/mm_types.h +++ linux-2.6/include/linux/mm_types.h @@ -10,7 +10,6 @@ #include #include #include -#include #include #include @@ -229,8 +228,9 @@ struct mm_struct { #ifdef CONFIG_CGROUP_MEM_CONT struct mem_cgroup *mem_cgroup; #endif - - struct mmu_notifier_head mmu_notifier; /* MMU notifier list */ +#ifdef CONFIG_MMU_NOTIFIER + struct hlist_head mmu_notifier_list; +#endif }; #endif /* _LINUX_MM_TYPES_H */ Index: linux-2.6/mm/rmap.c =================================================================== --- linux-2.6.orig/mm/rmap.c +++ linux-2.6/mm/rmap.c @@ -49,6 +49,7 @@ #include #include #include +#include #include From npiggin at suse.de Sun Mar 2 19:34:28 2008 From: npiggin at suse.de (Nick Piggin) Date: Mon, 3 Mar 2008 04:34:28 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080302155457.GK8091@v2.random> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> Message-ID: <20080303033428.GD3301@wotan.suse.de> On Sun, Mar 02, 2008 at 04:54:57PM +0100, Andrea Arcangeli wrote: > Difference between #v7 and #v8: This one on top of the previous patch [patch] mmu-v8: typesafe Move definition of struct mmu_notifier and struct mmu_notifier_ops under CONFIG_MMU_NOTIFIER to ensure they doesn't get dereferenced when they don't make sense. Signed-off-by: Nick Piggin --- Index: linux-2.6/include/linux/mmu_notifier.h =================================================================== --- linux-2.6.orig/include/linux/mmu_notifier.h +++ linux-2.6/include/linux/mmu_notifier.h @@ -3,8 +3,12 @@ #include #include +#include struct mmu_notifier; +struct mmu_notifier_ops; + +#ifdef CONFIG_MMU_NOTIFIER struct mmu_notifier_ops { /* @@ -53,10 +57,6 @@ struct mmu_notifier { const struct mmu_notifier_ops *ops; }; -#ifdef CONFIG_MMU_NOTIFIER - -#include - static inline int mm_has_notifiers(struct mm_struct *mm) { return unlikely(!hlist_empty(&mm->mmu_notifier_list)); From a-3333 at advtg.com Mon Mar 3 19:37:40 2008 From: a-3333 at advtg.com (Marie William) Date: Mon, 4 Mar 2008 11:37:40 +0800 Subject: [ofa-general] I'd like to show you my pic Message-ID: <01c87dec$26fe1a00$40f59cdb@a-3333> Hello! I am tired today. I am nice girl that would like to chat with you. Email me at Helena at GreatGolovaSite.com only, because I am using my friend's email to write this. Don't miss some of my naughty pictures. From npiggin at suse.de Sun Mar 2 19:39:04 2008 From: npiggin at suse.de (Nick Piggin) Date: Mon, 3 Mar 2008 04:39:04 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080302155457.GK8091@v2.random> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> Message-ID: <20080303033903.GE3301@wotan.suse.de> On Sun, Mar 02, 2008 at 04:54:57PM +0100, Andrea Arcangeli wrote: > Difference between #v7 and #v8: Here is just a couple of checkpatch fixes on top of the last patches. Index: linux-2.6/include/linux/mmu_notifier.h =================================================================== --- linux-2.6.orig/include/linux/mmu_notifier.h +++ linux-2.6/include/linux/mmu_notifier.h @@ -46,7 +46,7 @@ struct mmu_notifier_ops { */ void (*invalidate_range_begin)(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end); + unsigned long start, unsigned long end); void (*invalidate_range_end)(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, unsigned long end); @@ -137,7 +137,7 @@ static inline void mmu_notifier_mm_init( #define ptep_clear_flush_notify(__vma, __address, __ptep) \ ({ \ pte_t __pte; \ - struct vm_area_struct * ___vma = __vma; \ + struct vm_area_struct *___vma = __vma; \ unsigned long ___address = __address; \ __pte = ptep_clear_flush(___vma, ___address, __ptep); \ mmu_notifier_invalidate_page(___vma->vm_mm, ___address); \ @@ -147,7 +147,7 @@ static inline void mmu_notifier_mm_init( #define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ ({ \ int __young; \ - struct vm_area_struct * ___vma = __vma; \ + struct vm_area_struct *___vma = __vma; \ unsigned long ___address = __address; \ __young = ptep_clear_flush_young(___vma, ___address, __ptep); \ __young |= mmu_notifier_clear_flush_young(___vma->vm_mm, \ From nickpiggin at yahoo.com.au Sun Mar 2 21:11:09 2008 From: nickpiggin at yahoo.com.au (Nick Piggin) Date: Mon, 3 Mar 2008 16:11:09 +1100 Subject: [ofa-general] Re: [patch 2/6] mmu_notifier: Callbacks to invalidate address ranges In-Reply-To: References: <20080215064859.384203497@sgi.com> <200802201008.49933.nickpiggin@yahoo.com.au> Message-ID: <200803031611.10275.nickpiggin@yahoo.com.au> On Thursday 28 February 2008 09:35, Christoph Lameter wrote: > On Wed, 20 Feb 2008, Nick Piggin wrote: > > On Friday 15 February 2008 17:49, Christoph Lameter wrote: > > Also, what we are going to need here are not skeleton drivers > > that just do all the *easy* bits (of registering their callbacks), > > but actual fully working examples that do everything that any > > real driver will need to do. If not for the sanity of the driver > > writer, then for the sanity of the VM developers (I don't want > > to have to understand xpmem or infiniband in order to understand > > how the VM works). > > There are 3 different drivers that can already use it but the code is > complex and not easy to review. Skeletons are easy to allow people to get > started with it. Your skeleton is just registering notifiers and saying /* you fill the hard part in */ If somebody needs a skeleton in order just to register the notifiers, then almost by definition they are unqualified to write the hard part ;) > > > lru_add_drain(); > > > tlb = tlb_gather_mmu(mm, 0); > > > update_hiwater_rss(mm); > > > + mmu_notifier(invalidate_range_begin, mm, address, end, atomic); > > > end = unmap_vmas(&tlb, vma, address, end, &nr_accounted, details); > > > if (tlb) > > > tlb_finish_mmu(tlb, address, end); > > > + mmu_notifier(invalidate_range_end, mm, address, end, atomic); > > > return end; > > > } > > > > Where do you invalidate for munmap()? > > zap_page_range() called from unmap_vmas(). But it is not allowed to sleep. Where do you call the sleepable one from? > > Also, how to you resolve the case where you are not allowed to sleep? > > I would have thought either you have to handle it, in which case nobody > > needs to sleep; or you can't handle it, in which case the code is > > broken. > > That can be done in a variety of ways: > > 1. Change VM locking > > 2. Not handle file backed mappings (XPmem could work mostly in such a > config) > > 3. Keep the refcount elevated until pages are freed in another execution > context. OK, there are ways to solve it or hack around it. But this is exactly why I think the implementations should be kept seperate. Andrea's notifiers are coherent, work on all types of mappings, and will hopefully match closely the regular TLB invalidation sequence in the Linux VM (at the moment it is quite close, but I hope to make it a bit closer) so that it requires almost no changes to the mm. All the other things to try to make it sleep are either hacking holes in it (eg by removing coherency). So I don't think it is reasonable to require that any patch handle all cases. I actually think Andrea's patch is quite nice and simple itself, wheras I am against the patches that you posted. What about a completely different approach... XPmem runs over NUMAlink, right? Why not provide some non-sleeping way to basically IPI remote nodes over the NUMAlink where they can process the invalidation? If you intra-node cache coherency has to run over this link anyway, then presumably it is capable. Or another idea, why don't you LD_PRELOAD in the MPT library to also intercept munmap, mprotect, mremap etc as well as just fork()? That would give you similarly "good enough" coherency as the mmu notifier patches except that you can't swap (which Robin said was not a big problem). From LupeperilousVinson at 50states.com Mon Mar 3 01:36:38 2008 From: LupeperilousVinson at 50states.com (Richie Valencia) Date: Mon, 3 Mar 2008 08:36:38 -0100 Subject: [ofa-general] legally Erase Your Credit Card Debt Message-ID: <242501c87d01$6841ef50$280aa8c0@titkar> Get Out of Debt Today. Avoid Bankruptcy. Save Thousands... The Professional Way!! http://ilionf.com.cn/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-annmr at acergardens.com Tue Mar 4 00:08:24 2008 From: a-annmr at acergardens.com (Agnes Carlisle) Date: Mon, 4 Mar 2008 09:08:24 +0100 Subject: [ofa-general] Can we talk? Message-ID: <01c87dd7$4ce2ed00$5a86c7d9@a-annmr> Hello! I am bored this evening. I am nice girl that would like to chat with you. Email me at Carina at GreatGolovaSite.com only, because I am using my friend's email to write this. If you would like to see my pictures. From m91v5ft420s at gmail.com Mon Mar 3 00:24:38 2008 From: m91v5ft420s at gmail.com (=?BIG5?B?vbKpybrV?=) Date: Mon, 3 Mar 2008 16:24:38 +0800 Subject: [ofa-general] =?big5?b?p9qzzLPfxXey4aTRoUKw27pxoUKs3blxvHatWadB?= =?big5?b?wdmko7/5xXeq76jTp+Sn2qFJ?= Message-ID: 61vAi8wY13lRE0mSQ8xHX1wWx6pCw3tB34eE 我是辣美眉，想找人聊天嗎? 喜歡重口味的台灣狼，照過來看唷 我最喜歡聊天、唱歌、看電影若你還不錯歡迎來找我！ 激情的夜晚…期盼你的到來… 請撥打 0941-530-208(每30秒2元) VTpSrtlpIiMBIzXo3AMrZLxDbYPeMuOUE5ol9s8sEwYxC B5zQR7hRH5pD79aDw1zPj9uUl0gZW6hMX3oD ZJHI5UKGbdyPFyY72TNBtL1TLH2FnQO82mqty3HHiCMpJ From arne.redlich at xiranet.com Mon Mar 3 02:35:48 2008 From: arne.redlich at xiranet.com (Arne Redlich) Date: Mon, 03 Mar 2008 11:35:48 +0100 Subject: [ofa-general] [PATCH 0/2] IB/iSER bugfixes Message-ID: <87ejasxfgb.fsf@confield.dd.xiranet.com> Hi, While reading through the iSER code I noticed two rather nasty issues: 1. The iteration through the list of "iser_device"s during device lookup/creation is broken - it might result in an infinite loop if more than 1 HCA is used with iSER. Use list_for_each_entry() instead of the custom, flawed list iteration code. 2. "iser_device" allocation failure is "handled" with a BUG_ON() right before dereferencing the NULL-pointer. This is really scary, so here's my idea of a fix. Someone with a deeper understanding of the code should have a look at it since I'm not sure it does The Right Thing. Both patches are merely compile tested, and patch #2 needs to be applied on top of #1. Cheers, Arne From arne.redlich at xiranet.com Mon Mar 3 02:36:14 2008 From: arne.redlich at xiranet.com (Arne Redlich) Date: Mon, 03 Mar 2008 11:36:14 +0100 Subject: [ofa-general] Re: [PATCH 1/2] IB/iSER: fix list iteration bug Message-ID: <877igkxffl.fsf@confield.dd.xiranet.com> The iteration through the list of "iser_device"s during device lookup/creation is broken - it might result in an infinite loop if more than 1 HCA is used with iSER. Use list_for_each_entry() instead of the custom, flawed list iteration code. Signed-off-by: Arne Redlich --- drivers/infiniband/ulp/iser/iser_verbs.c | 36 ++++++++++++----------------- 1 files changed, 15 insertions(+), 21 deletions(-) diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c index 714b8db..1c0f968 100644 --- a/drivers/infiniband/ulp/iser/iser_verbs.c +++ b/drivers/infiniband/ulp/iser/iser_verbs.c @@ -237,33 +237,27 @@ static int iser_free_ib_conn_res(struct iser_conn *ib_conn) static struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) { - struct list_head *p_list; - struct iser_device *device = NULL; + struct iser_device *device; mutex_lock(&ig.device_list_mutex); - p_list = ig.device_list.next; - while (p_list != &ig.device_list) { - device = list_entry(p_list, struct iser_device, ig_list); - /* find if there's a match using the node GUID */ + list_for_each_entry(device, &ig.device_list, ig_list) if (device->ib_device->node_guid == cma_id->device->node_guid) - break; - } - - if (device == NULL) { - device = kzalloc(sizeof *device, GFP_KERNEL); - if (device == NULL) goto out; - /* assign this device to the device */ - device->ib_device = cma_id->device; - /* init the device and link it into ig device list */ - if (iser_create_device_ib_res(device)) { - kfree(device); - device = NULL; - goto out; - } - list_add(&device->ig_list, &ig.device_list); + + device = kzalloc(sizeof *device, GFP_KERNEL); + if (device == NULL) + goto out; + + device->ib_device = cma_id->device; + /* init the device and link it into ig device list */ + if (iser_create_device_ib_res(device)) { + kfree(device); + device = NULL; + goto out; } + list_add(&device->ig_list, &ig.device_list); + out: BUG_ON(device == NULL); device->refcount++; -- 1.5.4.1 From arne.redlich at xiranet.com Mon Mar 3 02:36:26 2008 From: arne.redlich at xiranet.com (Arne Redlich) Date: Mon, 03 Mar 2008 11:36:26 +0100 Subject: [ofa-general] Re: [PATCH 2/2] IB/iSER: handle iser_device allocation error gracefully Message-ID: <87zltgw0ut.fsf@confield.dd.xiranet.com> "iser_device" allocation failure is "handled" with a BUG_ON() right before dereferencing the NULL-pointer - fix this! Signed-off-by: Arne Redlich --- drivers/infiniband/ulp/iser/iser_verbs.c | 13 +++++++++---- 1 files changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c index 1c0f968..cf70d15 100644 --- a/drivers/infiniband/ulp/iser/iser_verbs.c +++ b/drivers/infiniband/ulp/iser/iser_verbs.c @@ -243,7 +243,7 @@ struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) list_for_each_entry(device, &ig.device_list, ig_list) if (device->ib_device->node_guid == cma_id->device->node_guid) - goto out; + goto inc_refcnt; device = kzalloc(sizeof *device, GFP_KERNEL); if (device == NULL) @@ -258,9 +258,9 @@ struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) } list_add(&device->ig_list, &ig.device_list); -out: - BUG_ON(device == NULL); +inc_refcnt: device->refcount++; +out: mutex_unlock(&ig.device_list_mutex); return device; } @@ -366,6 +366,12 @@ static void iser_addr_handler(struct rdma_cm_id *cma_id) int ret; device = iser_device_find_by_ib_device(cma_id); + if (!device) { + iser_err("device lookup/creation failed\n"); + iser_connect_error(cma_id); + return; + } + ib_conn = (struct iser_conn *)cma_id->context; ib_conn->device = device; @@ -374,7 +380,6 @@ static void iser_addr_handler(struct rdma_cm_id *cma_id) iser_err("resolve route failed: %d\n", ret); iser_connect_error(cma_id); } - return; } static void iser_route_handler(struct rdma_cm_id *cma_id) -- 1.5.4.1 From vlad at lists.openfabrics.org Mon Mar 3 03:08:05 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Mon, 3 Mar 2008 03:08:05 -0800 (PST) Subject: [ofa-general] ofa_1_3_kernel 20080303-0200 daily build status Message-ID: <20080303110805.549C4E602A6@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From a-alonso at acv-csc.be Tue Mar 4 03:33:47 2008 From: a-alonso at acv-csc.be (Kathie Dupree) Date: Mon, 4 Mar 2008 19:33:47 +0800 Subject: [ofa-general] I'd like to show you my pic Message-ID: <01c87e2e$aa405f80$335cea7b@a-alonso> Hello! I am tired this afternoon. I am nice girl that would like to chat with you. Email me at Alice at GreatGolovaSite.com only, because I am using my friend's email to write this. I would like to share some of my pics. From ogerlitz at voltaire.com Mon Mar 3 04:29:16 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 03 Mar 2008 14:29:16 +0200 Subject: [ofa-general] [PATCH RFC] ib_mthca: avoid recycling FMR R_Keys too soon In-Reply-To: <200802211824.34083.okir@lst.de> References: <200802202056.32451.okir@lst.de> <200802211221.53652.jackm@dev.mellanox.co.il> <47BD63BC.4040207@voltaire.com> <200802211824.34083.okir@lst.de> Message-ID: <47CBEF1C.7060109@voltaire.com> Olaf Kirch wrote: > So when the sequence counter overflows, it doesn't spill into any > reserved bit. OK, got it, thanks for the clarification. So this opens the door to a patch as you suggested. Or. From ogerlitz at voltaire.com Mon Mar 3 04:38:22 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 03 Mar 2008 14:38:22 +0200 Subject: [ofa-general] ipoib connected mode In-Reply-To: <97a7c7ed0802210849s593c73ccp8a5589c334f9bed0@mail.gmail.com> References: <97a7c7ed0802210849s593c73ccp8a5589c334f9bed0@mail.gmail.com> Message-ID: <47CBF13E.2030306@voltaire.com> Michael Di Domenico wrote: > 5. On RedHat EL 4 up4, the IPOIB implementation is not spec-compliant: > - ipoib multicast does not work > - ipoib cannot interoperate between RHEL4U4 and other hosts. This is due to > missing code in the kernel which was available in U3 and U5 but removed in > U4. As a workaround, upgrade to RHEL4U5. > > I found this blurb in the ipoib_release_notes file inside > ofed-1.2.5.1. Does this mean that IPoIB does not work between a > RHEL4U4 and RHEL5 x86_64 machine? I ask, because i have one of each > one with Ipath and one with Mthca and i can certainly talk between the > machines. > RH4 U4 kernel has a bug which causes all IP multicast traffic routed to ipoib interfaces to be going over the broadcast group, this does not prevent unicast interoperability between U4 to other systems, but, if you have any IP multicast going in your app/system, I would suggest move a way from U4 (eg to U5 or RH5, etc), see https://bugs.openfabrics.org/show_bug.cgi?id=266 for more details. > Is connected mode supported? > yes From andrea at qumranet.com Mon Mar 3 04:51:53 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Mon, 3 Mar 2008 13:51:53 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303032934.GA3301@wotan.suse.de> References: <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> Message-ID: <20080303125152.GS8091@v2.random> On Mon, Mar 03, 2008 at 04:29:34AM +0100, Nick Piggin wrote: > to something I prefer. Others may not, but I'll post them for debate > anyway. Sure, thanks! > > I didn't drop invalidate_page, because invalidate_range_begin/end > > would be slower for usages like KVM/GRU (we don't need a begin/end > > there because where invalidate_page is called, the VM holds a > > reference on the page). do_wp_page should also use invalidate_page > > since it can free the page after dropping the PT lock without losing > > any performance (that's not true for the places where invalidate_range > > is called). > > I'm still not completely happy with this. I had a very quick look > at the GRU driver, but I don't see why it can't be implemented > more like the regular TLB model, and have TLB insertions depend on > the linux pte, and do invalidates _after_ restricting permissions > to the pte. > > Ie. I'd still like to get rid of invalidate_range_begin, and get > rid of invalidate calls from places where permissions are relaxed. _begin exists because by the time _end is called, the VM already dropped the reference on the page. This way we can do a single invalidate no matter how large the range is. I don't see ways to remove _begin while still invoking _end a single time for the whole range. > If we can agree on the API, then I don't see any reason why it can't > go into 2.6.25, unless someome wants more time to review it (but > 2.6.25 release should be quite far away still so there should be quite > a bit of time). Cool! ;) From moshek at voltaire.com Mon Mar 3 05:04:23 2008 From: moshek at voltaire.com (Moshe Kazir) Date: Mon, 3 Mar 2008 15:04:23 +0200 Subject: [ofa-general] Using --build32 on 32 bits machine (OFED-1.3 GA ) In-Reply-To: <1204467417.3358.123.camel@mtls03> References: <200802281842.19303.bs@q-leap.de> <1204467417.3358.123.camel@mtls03> Message-ID: <39C75744D164D948A170E9792AF8E7CAC5AE0D@exil.voltaire.com> If a user uses ofed.conf file include the line " build32=y" build of libibverbs fails on rh4 u 5 and sles 10 . The following patch fix this problem (also in the attached file ) Moshe --- a/install.pl 2008-03-02 06:23:04.000000000 -0500 +++ b/install.pl 2008-03-03 00:59:53.000000000 -0500 @@ -1463,8 +1463,11 @@ sub supported32bit ### print RED "\n32-bit libraries are not supported on this platform", RESET "\n" if (not $quiet); ### return 0; ### } + if ( $arch eq "x86_64" or $arch eq "ppc64" or $arch eq "ia64" ) { return 1 } + return 0 +} # Check whether compiler $1 exist sub set_compilers ____________________________________________________________ Moshe Katzir | +972-9971-8639 (o) | +972-52-860-6042 (m) Voltaire - The Grid Backbone www.voltaire.com -------------- next part -------------- A non-text attachment was scrubbed... Name: ofed_install_32bits_machine_fix.patch Type: application/octet-stream Size: 445 bytes Desc: ofed_install_32bits_machine_fix.patch URL: From npiggin at suse.de Mon Mar 3 05:10:17 2008 From: npiggin at suse.de (Nick Piggin) Date: Mon, 3 Mar 2008 14:10:17 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303125152.GS8091@v2.random> References: <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> Message-ID: <20080303131017.GC13138@wotan.suse.de> On Mon, Mar 03, 2008 at 01:51:53PM +0100, Andrea Arcangeli wrote: > On Mon, Mar 03, 2008 at 04:29:34AM +0100, Nick Piggin wrote: > > to something I prefer. Others may not, but I'll post them for debate > > anyway. > > Sure, thanks! > > > > I didn't drop invalidate_page, because invalidate_range_begin/end > > > would be slower for usages like KVM/GRU (we don't need a begin/end > > > there because where invalidate_page is called, the VM holds a > > > reference on the page). do_wp_page should also use invalidate_page > > > since it can free the page after dropping the PT lock without losing > > > any performance (that's not true for the places where invalidate_range > > > is called). > > > > I'm still not completely happy with this. I had a very quick look > > at the GRU driver, but I don't see why it can't be implemented > > more like the regular TLB model, and have TLB insertions depend on > > the linux pte, and do invalidates _after_ restricting permissions > > to the pte. > > > > Ie. I'd still like to get rid of invalidate_range_begin, and get > > rid of invalidate calls from places where permissions are relaxed. > > _begin exists because by the time _end is called, the VM already > dropped the reference on the page. This way we can do a single > invalidate no matter how large the range is. I don't see ways to > remove _begin while still invoking _end a single time for the whole > range. Is this just a GRU problem? Can't we just require them to take a ref on the page (IIRC Jack said GRU could be changed to more like a TLB model). From pawel.dziekonski at wcss.pl Mon Mar 3 05:20:38 2008 From: pawel.dziekonski at wcss.pl (Pawel Dziekonski) Date: Mon, 3 Mar 2008 14:20:38 +0100 Subject: [nfs-rdma-devel] [ofa-general] Status of NFS-RDMA ? (fwd) In-Reply-To: <1204505291.1567.15.camel@trinity.ogc.int> References: <20080206154701.GA11384@cefeid.wcss.wroc.pl> <1202320491.14810.29.camel@trinity.ogc.int> <20080206213543.GA21176@cefeid.wcss.wroc.pl> <20080209221711.GB20332@cefeid.wcss.wroc.pl> <47C7C261.2060007@hamburgnet.de> <1204505291.1567.15.camel@trinity.ogc.int> Message-ID: <20080303132038.GH31715@cefeid.wcss.wroc.pl> On Sun, 02 Mar 2008 at 06:48:11PM -0600, Tom Tucker wrote: > > On Fri, 2008-02-29 at 09:29 +0100, Sebastian Schmitzdorff wrote: > > hi pawel, > > > > I was wondering if you have achieved better nfs rdma benchmark results > > by now? > > Pawel: > > What is your network hardware setup? hi, both nfs server and client have Mellanox MT25204 HCAs. tests were done by connecting them port to port (without switch) with DDR cable. reported link was 20Gbps. currently I have a Flextronix switch that reports itself as "MT47396 Infiniscale-III Mellanox Technologies". cheers, P -- Pawel Dziekonski Wroclaw Centre for Networking & Supercomputing, HPC Department Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl From andrea at qumranet.com Mon Mar 3 05:24:59 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Mon, 3 Mar 2008 14:24:59 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303131017.GC13138@wotan.suse.de> References: <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> Message-ID: <20080303132459.GT8091@v2.random> On Mon, Mar 03, 2008 at 02:10:17PM +0100, Nick Piggin wrote: > Is this just a GRU problem? Can't we just require them to take a ref > on the page (IIRC Jack said GRU could be changed to more like a TLB > model). Yes, it's just a GRU problem, it tries to optimize performance by calling follow_page only in the fast path, and fallbacks to get_user_pages; put_page in the slow path. xpmem could also send the message in _begin and wait the message in _end, to reduce the wait time. But if you forge GRU to call get_user_pages only (like KVM does), the _begin can be removed. In theory we could also optimize KVM to use follow_page only if the pte is already established. I'm not sure how much that is a worthwhile optimization though. However note that Quadrics also had a callback before and one after, so they may be using the callback before for similar optimizations. But functionality-wise _end is the only required bit if everyone takes refcounts like KVM and XPMEM do. From hrosenstock at xsigo.com Mon Mar 3 06:11:57 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 03 Mar 2008 06:11:57 -0800 Subject: [ofa-general] [PATCH] OpenSM: Set packet life time to subnet timeout option rather than default Message-ID: <1204553517.6469.225.camel@hrosenstock-ws.xsigo.com> OpenSM: Set packet life time to subnet timeout option rather than default Signed-off-by: Hal Rosenstock diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c index 76227c0..2d0b313 100644 --- a/opensm/opensm/osm_prtn.c +++ b/opensm/opensm/osm_prtn.c @@ -215,7 +215,7 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, mc_rec.tclass = 0; mc_rec.pkey = pkey; mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ - mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; + mc_rec.pkt_life = p_subn->opt.subnet_timeout; mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, hop_limit); /* Scope in MCMemberRecord (if present) needs to be consistent with MGID */ mc_rec.scope_state = ib_member_set_scope_state(scope, IB_MC_REC_STATE_FULL_MEMBER); diff --git a/opensm/opensm/osm_sa_multipath_record.c b/opensm/opensm/osm_sa_multipath_record.c index e9ddea5..214a82b 100644 --- a/opensm/opensm/osm_sa_multipath_record.c +++ b/opensm/opensm/osm_sa_multipath_record.c @@ -533,7 +533,7 @@ __osm_mpr_rcv_get_path_parms(IN osm_sa_t * sa, else if (p_qos_level && p_qos_level->pkt_life_set) pkt_life = p_qos_level->pkt_life; else - pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; + pkt_life = sa->p_subn->opt.subnet_timeout; /* we silently ignore cases where only the PktLife selector is defined */ if ((comp_mask & IB_MPR_COMPMASK_PKTLIFETIMESELEC) && diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c index 8ed44f4..45208f5 100644 --- a/opensm/opensm/osm_sa_path_record.c +++ b/opensm/opensm/osm_sa_path_record.c @@ -452,7 +452,7 @@ __osm_pr_rcv_get_path_parms(IN osm_sa_t * sa, else if (p_qos_level && p_qos_level->pkt_life_set) pkt_life = p_qos_level->pkt_life; else - pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; + pkt_life = sa->p_subn->opt.subnet_timeout; /* Determine if these values meet the user criteria From hrosenstock at xsigo.com Mon Mar 3 06:13:36 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 03 Mar 2008 06:13:36 -0800 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <20080302190504.GE30723@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> <20080302141713.GC30723@sashak.voltaire.com> <1204480309.6469.201.camel@hrosenstock-ws.xsigo.com> <20080302190504.GE30723@sashak.voltaire.com> Message-ID: <1204553616.6469.229.camel@hrosenstock-ws.xsigo.com> On Sun, 2008-03-02 at 19:05 +0000, Sasha Khapyorsky wrote: > On 09:51 Sun 02 Mar , Hal Rosenstock wrote: > > > > A different routing dump reflecting balance or not and how out of > > balance. > > This makes sense. Actually OpenSM has such sort of dump right now, but > it is printed to stdout. Yes, there are various dumps (to stdout or opensm.fdbs) which contain routing information but do they contain balance info ? Also, seems like this sense as part of the console. -- Hal > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Mon Mar 3 06:32:08 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 03 Mar 2008 06:32:08 -0800 Subject: [ofa-general] OpenSM building error Message-ID: <1204554728.6469.237.camel@hrosenstock-ws.xsigo.com> Hi Sasha, When I now build OpenSM, I get the following error at link time: opensm-osm_console_io.o(.text+0x245): In function `is_authorized': osm_console_io.c:134: undefined reference to `hosts_ctl' I think hosts_ctl comes from libwrap but I don't see that as part of what the link includes. What am I missing ? Thanks. -- Hal From monis at voltaire.com Mon Mar 3 06:53:28 2008 From: monis at voltaire.com (Moni Shoua) Date: Mon, 03 Mar 2008 16:53:28 +0200 Subject: [ofa-general] A question regarding multicast send Message-ID: <47CC10E8.9040006@voltaire.com> Hi, The following refers to code in ipoib_mcast_send() http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=drivers/infiniband/ulp/ipoib/ipoib_multicast.c;h=2628339e3a9973b01402250f8e6bf73be5d37c04;hb=HEAD#l642 Some background before the question: I am testing bonding with IPoIB now and I see that sometimes grat. ARP are not being sent when one of the slaves gets down and another takes its place as an active slave (Grat. arps are necessary for reasonable failover). A short research shows that the bonding module sets the multicast list for the new active slave and this causes multicast task restart. When grat. arp arrives to ipoib_mcast_send() it finds that the flag IPOIB_MCAST_STARTED is not set and the packet is dropped. This doesn't happen all the times. Sometimes the grat. arp arrives to ipoib_mcast_send() when IPOIB_MCAST_STARTED is set and packet is sent. Now, my question is: This code is found in the begging of ipoib_mcast_send(). if (!test_bit(IPOIB_MCAST_STARTED, &priv->flags) || !priv->broadcast || !test_bit(IPOIB_MCAST_FLAG_ATTACHED,&priv->broadcast->flags)){ ++dev->stats.tx_dropped; dev_kfree_skb_any(skb); goto unlock; } I wonder, why isn't it possible to 1. Not to drop the packet (at least for the case when IPOIB_MCAST_STARTED is cleared). 2. look for/create mcast structure (already in code) 3. put skb in &mcast->pkt_queue (enhabce the condition !mcast->ah) thanks MoniS From steiner at sgi.com Mon Mar 3 07:18:59 2008 From: steiner at sgi.com (Jack Steiner) Date: Mon, 3 Mar 2008 09:18:59 -0600 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303131017.GC13138@wotan.suse.de> References: <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> Message-ID: <20080303151859.GA19374@sgi.com> On Mon, Mar 03, 2008 at 02:10:17PM +0100, Nick Piggin wrote: > On Mon, Mar 03, 2008 at 01:51:53PM +0100, Andrea Arcangeli wrote: > > On Mon, Mar 03, 2008 at 04:29:34AM +0100, Nick Piggin wrote: > > > to something I prefer. Others may not, but I'll post them for debate > > > anyway. > > > > Sure, thanks! > > > > > > I didn't drop invalidate_page, because invalidate_range_begin/end > > > > would be slower for usages like KVM/GRU (we don't need a begin/end > > > > there because where invalidate_page is called, the VM holds a > > > > reference on the page). do_wp_page should also use invalidate_page > > > > since it can free the page after dropping the PT lock without losing > > > > any performance (that's not true for the places where invalidate_range > > > > is called). > > > > > > I'm still not completely happy with this. I had a very quick look > > > at the GRU driver, but I don't see why it can't be implemented > > > more like the regular TLB model, and have TLB insertions depend on > > > the linux pte, and do invalidates _after_ restricting permissions > > > to the pte. > > > > > > Ie. I'd still like to get rid of invalidate_range_begin, and get > > > rid of invalidate calls from places where permissions are relaxed. > > > > _begin exists because by the time _end is called, the VM already > > dropped the reference on the page. This way we can do a single > > invalidate no matter how large the range is. I don't see ways to > > remove _begin while still invoking _end a single time for the whole > > range. > > Is this just a GRU problem? Can't we just require them to take a ref > on the page (IIRC Jack said GRU could be changed to more like a TLB > model). Maintaining a long-term reference on a page is a problem. The GRU does not currently maintain tables to track the pages for which dropins have been done. The GRU has a large internal TLB and is designed to reference up to 8PB of memory. The size of the tables to track this many referenced pages would be a problem (at best). From arthur.jones at qlogic.com Mon Mar 3 08:05:32 2008 From: arthur.jones at qlogic.com (Arthur Jones) Date: Mon, 3 Mar 2008 08:05:32 -0800 Subject: [ofa-general] Re: [RFC PATCH 07/14] RFC IB/ipath: Fix sparse warning about pointer signedness In-Reply-To: <20082292026.nsb7AGzfyaeVkEEq@cisco.com> References: <20082292026.wGjGlyABtVgGoyKV@cisco.com> <20082292026.nsb7AGzfyaeVkEEq@cisco.com> Message-ID: <20080303160532.GA24062@bauxite.pathscale.com> hi roland, i fixed this one slightly differently, and i prefer it to your fix. the patch is attached... arthur On Fri, Feb 29, 2008 at 08:26:03PM -0800, Roland Dreier wrote: > ipath_count_units() wants its third parameter to be a u32 *, so change > the declaration of maxofallports in find_best_unit() to be a u32 instead > of a signed int. This fixes > > drivers/infiniband/hw/ipath/ipath_file_ops.c:1654:47: warning: incorrect type in argument 3 (different signedness) > drivers/infiniband/hw/ipath/ipath_file_ops.c:1654:47: expected unsigned int [usertype] *maxportsp > drivers/infiniband/hw/ipath/ipath_file_ops.c:1654:47: got int * > > Signed-off-by: Roland Dreier > --- > drivers/infiniband/hw/ipath/ipath_file_ops.c | 3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > diff --git a/drivers/infiniband/hw/ipath/ipath_file_ops.c b/drivers/infiniband/hw/ipath/ipath_file_ops.c > index 7e025c8..338733e 100644 > --- a/drivers/infiniband/hw/ipath/ipath_file_ops.c > +++ b/drivers/infiniband/hw/ipath/ipath_file_ops.c > @@ -1648,7 +1648,8 @@ static int find_best_unit(struct file *fp, > const struct ipath_user_info *uinfo) > { > int ret = 0, i, prefunit = -1, devmax; > - int maxofallports, npresent, nup; > + int npresent, nup; > + u32 maxofallports; > int ndev; > > devmax = ipath_count_units(&npresent, &nup, &maxofallports); > -- > 1.5.4.2 > -------------- next part -------------- IB/ipath - sparse cleanup, use int rather u32 u32 is not necessary here, use int instead, this fixes sparse warning: drivers/infiniband/hw/ipath/ipath_file_ops.c:1654:47: warning: incorrect type in argument 3 (different signedness) drivers/infiniband/hw/ipath/ipath_file_ops.c:1654:47: expected unsigned int [usertype] *maxportsp drivers/infiniband/hw/ipath/ipath_file_ops.c:1654:47: got int * Signed-off-by: Arthur Jones diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index db5f198..72fd16f 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -265,12 +265,12 @@ struct ipath_devdata *ipath_lookup(int unit) return dd; } -int ipath_count_units(int *npresentp, int *nupp, u32 *maxportsp) +int ipath_count_units(int *npresentp, int *nupp, int *maxportsp) { int nunits, npresent, nup; struct ipath_devdata *dd; unsigned long flags; - u32 maxports; + int maxports; nunits = npresent = nup = maxports = 0; diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 5d4ca80..17fc00c 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -834,7 +834,7 @@ extern struct ipath_devdata *ipath_lookup(int unit); int ipath_init_chip(struct ipath_devdata *, int); int ipath_enable_wc(struct ipath_devdata *dd); void ipath_disable_wc(struct ipath_devdata *dd); -int ipath_count_units(int *npresentp, int *nupp, u32 *maxportsp); +int ipath_count_units(int *npresentp, int *nupp, int *maxportsp); void ipath_shutdown_device(struct ipath_devdata *); void ipath_clear_freeze(struct ipath_devdata *); int ipath_signal_procs(struct ipath_devdata *, int); From arthur.jones at qlogic.com Mon Mar 3 08:08:52 2008 From: arthur.jones at qlogic.com (Arthur Jones) Date: Mon, 3 Mar 2008 08:08:52 -0800 Subject: [ofa-general] Re: [RFC PATCH 08/14] RFC IB/ipath: Fix sparse warning about shadowed symbol In-Reply-To: <20082292026.1yZE1Hi3o3kxPkrE@cisco.com> References: <20082292026.nsb7AGzfyaeVkEEq@cisco.com> <20082292026.1yZE1Hi3o3kxPkrE@cisco.com> Message-ID: <20080303160852.GB24062@bauxite.pathscale.com> thanks! Acked-by: Arthur Jones On Fri, Feb 29, 2008 at 08:26:03PM -0800, Roland Dreier wrote: > Fix > > drivers/infiniband/hw/ipath/ipath_init_chip.c:526:10: warning: symbol 'val' shadows an earlier one > drivers/infiniband/hw/ipath/ipath_init_chip.c:473:6: originally declared here > > by giving the second val a different name. > > Signed-off-by: Roland Dreier > --- > drivers/infiniband/hw/ipath/ipath_init_chip.c | 8 ++++---- > 1 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/drivers/infiniband/hw/ipath/ipath_init_chip.c b/drivers/infiniband/hw/ipath/ipath_init_chip.c > index 4471674..5428aff 100644 > --- a/drivers/infiniband/hw/ipath/ipath_init_chip.c > +++ b/drivers/infiniband/hw/ipath/ipath_init_chip.c > @@ -523,16 +523,16 @@ static void enable_chip(struct ipath_devdata *dd, > * initial values of the generation bit correct. > */ > for (i = 0; i < dd->ipath_pioavregs; i++) { > - __le64 val; > + __le64 pioavail; > > /* > * Chip Errata bug 6641; even and odd qwords>3 are swapped. > */ > if (i > 3 && (dd->ipath_flags & IPATH_SWAP_PIOBUFS)) > - val = dd->ipath_pioavailregs_dma[i ^ 1]; > + pioavail = dd->ipath_pioavailregs_dma[i ^ 1]; > else > - val = dd->ipath_pioavailregs_dma[i]; > - dd->ipath_pioavailshadow[i] = le64_to_cpu(val); > + pioavail = dd->ipath_pioavailregs_dma[i]; > + dd->ipath_pioavailshadow[i] = le64_to_cpu(pioavail); > } > /* can get counters, stats, etc. */ > dd->ipath_flags |= IPATH_PRESENT; > -- > 1.5.4.2 > From npiggin at suse.de Mon Mar 3 08:59:10 2008 From: npiggin at suse.de (Nick Piggin) Date: Mon, 3 Mar 2008 17:59:10 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303151859.GA19374@sgi.com> References: <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> Message-ID: <20080303165910.GA23998@wotan.suse.de> On Mon, Mar 03, 2008 at 09:18:59AM -0600, Jack Steiner wrote: > On Mon, Mar 03, 2008 at 02:10:17PM +0100, Nick Piggin wrote: > > On Mon, Mar 03, 2008 at 01:51:53PM +0100, Andrea Arcangeli wrote: > > > On Mon, Mar 03, 2008 at 04:29:34AM +0100, Nick Piggin wrote: > > > > to something I prefer. Others may not, but I'll post them for debate > > > > anyway. > > > > > > Sure, thanks! > > > > > > > > I didn't drop invalidate_page, because invalidate_range_begin/end > > > > > would be slower for usages like KVM/GRU (we don't need a begin/end > > > > > there because where invalidate_page is called, the VM holds a > > > > > reference on the page). do_wp_page should also use invalidate_page > > > > > since it can free the page after dropping the PT lock without losing > > > > > any performance (that's not true for the places where invalidate_range > > > > > is called). > > > > > > > > I'm still not completely happy with this. I had a very quick look > > > > at the GRU driver, but I don't see why it can't be implemented > > > > more like the regular TLB model, and have TLB insertions depend on > > > > the linux pte, and do invalidates _after_ restricting permissions > > > > to the pte. > > > > > > > > Ie. I'd still like to get rid of invalidate_range_begin, and get > > > > rid of invalidate calls from places where permissions are relaxed. > > > > > > _begin exists because by the time _end is called, the VM already > > > dropped the reference on the page. This way we can do a single > > > invalidate no matter how large the range is. I don't see ways to > > > remove _begin while still invoking _end a single time for the whole > > > range. > > > > Is this just a GRU problem? Can't we just require them to take a ref > > on the page (IIRC Jack said GRU could be changed to more like a TLB > > model). > > Maintaining a long-term reference on a page is a problem. The GRU does not > currently maintain tables to track the pages for which dropins have been done. > > The GRU has a large internal TLB and is designed to reference up to 8PB of > memory. The size of the tables to track this many referenced pages would be > a problem (at best). Is it any worse a problem than the pagetables of the processes which have their virtual memory exported to GRU? AFAIKS, no; it is on the same magnitude of difficulty. So you could do it without introducing any fundamental problem (memory usage might be increased by some constant factor, but I think we can cope with that in order to make the core patch really nice and simple). It is going to be really easy to add more weird and wonderful notifiers later that deviate from our standard TLB model. It would be much harder to remove them. So I really want to see everyone conform to this model first. Numbers and comparisons can be brought out afterwards if people want to attempt to make such changes. From sean.hefty at intel.com Mon Mar 3 09:30:49 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 3 Mar 2008 09:30:49 -0800 Subject: [ofa-general] RE: [RFC PATCH 12/14] RFC RDMA/cma: Endianness annotations In-Reply-To: <20082292026.eijz7K8Loa5odTxG@cisco.com> References: <20082292026.LIpsrWh15syGMtf1@cisco.com> <20082292026.eijz7K8Loa5odTxG@cisco.com> Message-ID: <000101c87d54$52867ed0$9c98070a@amr.corp.intel.com> Acked-by: Sean Hefty From sean.hefty at intel.com Mon Mar 3 09:31:14 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 3 Mar 2008 09:31:14 -0800 Subject: [ofa-general] RE: [RFC PATCH 09/14] RFC RDMA/addr: Endianness annotations In-Reply-To: <20082292026.rdppSkcg8Vk0HOBr@cisco.com> References: <20082292026.1yZE1Hi3o3kxPkrE@cisco.com> <20082292026.rdppSkcg8Vk0HOBr@cisco.com> Message-ID: <000201c87d54$614054f0$9c98070a@amr.corp.intel.com> Acked-by: Sean Hefty From sean.hefty at intel.com Mon Mar 3 09:30:12 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 3 Mar 2008 09:30:12 -0800 Subject: [ofa-general] [RFC PATCH 11/14] RFC IB/cm: Endianness annotations In-Reply-To: <20082292026.LIpsrWh15syGMtf1@cisco.com> References: <20082292026.QCkENmMgTjf04vR1@cisco.com> <20082292026.LIpsrWh15syGMtf1@cisco.com> Message-ID: <000001c87d54$3c8f2c80$9c98070a@amr.corp.intel.com> >Mostly update the RB tree comparisons to force __be types to normal >integers, but the change to cm_format_sidr_req() looks like a real >fix: param->path->pkey is already __be16. > >Signed-off-by: Roland Dreier Acked-by: Sean Hefty The change to cm_format_sidr_req() does fix a bug. From steiner at sgi.com Mon Mar 3 10:06:05 2008 From: steiner at sgi.com (Jack Steiner) Date: Mon, 3 Mar 2008 12:06:05 -0600 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303165910.GA23998@wotan.suse.de> References: <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> <20080303165910.GA23998@wotan.suse.de> Message-ID: <20080303180605.GA3552@sgi.com> On Mon, Mar 03, 2008 at 05:59:10PM +0100, Nick Piggin wrote: > On Mon, Mar 03, 2008 at 09:18:59AM -0600, Jack Steiner wrote: > > On Mon, Mar 03, 2008 at 02:10:17PM +0100, Nick Piggin wrote: > > > On Mon, Mar 03, 2008 at 01:51:53PM +0100, Andrea Arcangeli wrote: > > > > On Mon, Mar 03, 2008 at 04:29:34AM +0100, Nick Piggin wrote: > > > > > to something I prefer. Others may not, but I'll post them for debate > > > > > anyway. > > > > > > > > Sure, thanks! > > > > > > > > > > I didn't drop invalidate_page, because invalidate_range_begin/end > > > > > > would be slower for usages like KVM/GRU (we don't need a begin/end > > > > > > there because where invalidate_page is called, the VM holds a > > > > > > reference on the page). do_wp_page should also use invalidate_page > > > > > > since it can free the page after dropping the PT lock without losing > > > > > > any performance (that's not true for the places where invalidate_range > > > > > > is called). > > > > > > > > > > I'm still not completely happy with this. I had a very quick look > > > > > at the GRU driver, but I don't see why it can't be implemented > > > > > more like the regular TLB model, and have TLB insertions depend on > > > > > the linux pte, and do invalidates _after_ restricting permissions > > > > > to the pte. > > > > > > > > > > Ie. I'd still like to get rid of invalidate_range_begin, and get > > > > > rid of invalidate calls from places where permissions are relaxed. > > > > > > > > _begin exists because by the time _end is called, the VM already > > > > dropped the reference on the page. This way we can do a single > > > > invalidate no matter how large the range is. I don't see ways to > > > > remove _begin while still invoking _end a single time for the whole > > > > range. The range invalidates have a performance advantage for the GRU. TLB invalidates on the GRU are relatively slow (usec) and interfere somewhat with the performance of other active GRU instructions. Invalidating a large chunk of addresses with a single GRU TLBINVAL operation is must faster than issuing a stream of single page TLBINVALs. I expect this performance advantage will also apply to other users of mmuops. > > > > > > Is this just a GRU problem? Can't we just require them to take a ref > > > on the page (IIRC Jack said GRU could be changed to more like a TLB > > > model). > > > > Maintaining a long-term reference on a page is a problem. The GRU does not > > currently maintain tables to track the pages for which dropins have been done. > > > > The GRU has a large internal TLB and is designed to reference up to 8PB of > > memory. The size of the tables to track this many referenced pages would be > > a problem (at best). > > Is it any worse a problem than the pagetables of the processes which have > their virtual memory exported to GRU? AFAIKS, no; it is on the same > magnitude of difficulty. So you could do it without introducing any > fundamental problem (memory usage might be increased by some constant > factor, but I think we can cope with that in order to make the core patch > really nice and simple). Functionally, the GRU is very close to what I would consider to be the "standard TLB" model. Dropins and flushs map closely to processor dropins and flushes for cpus. The internal structure of the GRU TLB is identical to the TLB of existing cpus. Requiring the GRU driver to track dropins with long term page references seems to me a deviation from having the basic mmuops support a "standard TLB" model. AFAIK, no other processor requires this. Tracking TLB dropins (and long term page references) could be done but it adds significant complexity and scaling issues. The size of the tables to track many TB (to PB) of memory can get large. If the memory is being referenced by highly threaded applications, then the problem becomes even more complex. Either tables must be replicated per-thread (and require even more memory), or the table structure becomes even more complex to deal with node locality, cacheline bouncing, etc. Try to avoid a requirement to track dropins with long term page references. > It is going to be really easy to add more weird and wonderful notifiers > later that deviate from our standard TLB model. It would be much harder to > remove them. So I really want to see everyone conform to this model first. Agree. From avi at qumranet.com Mon Mar 3 10:09:49 2008 From: avi at qumranet.com (Avi Kivity) Date: Mon, 03 Mar 2008 20:09:49 +0200 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303180605.GA3552@sgi.com> References: <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> <20080303165910.GA23998@wotan.suse.de> <20080303180605.GA3552@sgi.com> Message-ID: <47CC3EED.7090507@qumranet.com> Jack Steiner wrote: > The range invalidates have a performance advantage for the GRU. TLB invalidates > on the GRU are relatively slow (usec) and interfere somewhat with the performance > of other active GRU instructions. Invalidating a large chunk of addresses with > a single GRU TLBINVAL operation is must faster than issuing a stream of single > page TLBINVALs. > > I expect this performance advantage will also apply to other users of mmuops. > In theory this would apply to kvm as well (coalesce tlb flush IPIs, lookup shadow page table once), but is it really a fast path? What triggers range operations for your use cases? -- error compiling committee.c: too many arguments to function From steiner at sgi.com Mon Mar 3 10:23:08 2008 From: steiner at sgi.com (Jack Steiner) Date: Mon, 3 Mar 2008 12:23:08 -0600 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <47CC3EED.7090507@qumranet.com> References: <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> <20080303165910.GA23998@wotan.suse.de> <20080303180605.GA3552@sgi.com> <47CC3EED.7090507@qumranet.com> Message-ID: <20080303182308.GC3552@sgi.com> On Mon, Mar 03, 2008 at 08:09:49PM +0200, Avi Kivity wrote: > Jack Steiner wrote: > >The range invalidates have a performance advantage for the GRU. TLB > >invalidates > >on the GRU are relatively slow (usec) and interfere somewhat with the > >performance > >of other active GRU instructions. Invalidating a large chunk of addresses > >with > >a single GRU TLBINVAL operation is must faster than issuing a stream of > >single > >page TLBINVALs. > > > >I expect this performance advantage will also apply to other users of > >mmuops. > > > > In theory this would apply to kvm as well (coalesce tlb flush IPIs, > lookup shadow page table once), but is it really a fast path? What > triggers range operations for your use cases? Although not frequent, an unmap of a multiple TB object could be quite painful if each page was invalidated individually instead of 1 invalidate for the entire range. This is even worse if the application is threaded and the object has been reference by many GRUs (there are 16 GRU ports per node - each potentially has to be invalidated). Forks (again, not frequent) would be another case. From bogus@does.not.exist.com Mon Mar 3 03:33:51 2008 From: bogus@does.not.exist.com (FRANK JIM) Date: 3 Mar 2008 11:33:51 -0000 Subject: [ofa-general] CONTACT IMPEX TRUST COURIER COMPANY FOR THE DELIVERY OF YOUR CONSIGNMENT Message-ID: <20080303113351.18114.qmail@ns.iram.co.kr> My Dear Friend! How are you today? This is to thank you for your effort to help me in the past. I understand that your hands were tied. Not to worry. I have succeeded in getting the money transferred into the account provided by a newly found friend of mine in India. I am in London with my family presently. To compensate your past assistance and commitments to me, I have made available 10% of the total money which is $800,000usd (EIGHT HUNDRED THOUSAND DOLLAR) cash for you. I made it cash because my account officer advice me to do so, he said that since I will not be around, it will be safe to keep cash than to keep check, because of the validity of the check might expire before you get the check. I have, through the help of my account officer packaged the money in a security proof box. The box was sealed with synthetic nylon seal and padded with machine. It was deposited with IMPEX TRUST COURIER COMPANY for safe keeping, as (Sensitive Photographic Film Material), The Company does not know the original contents of the box. What l declared to them as the content is Sensitive Photographic Film Material, I did not declare money to them please. that is what you will mention to them when contacting them. Also, note that I have paid the delivery fee and insurance fee in advance pending the time you will contact them, so you don't have any problem, the only fee they will ask you is their registration fee which is not much. Below is the Deposit details: Deposit Number: PLCC-101-PL45 Sort/Clearance Code: PLC/101-45/P50 Deposit Certificate N0.: 405576 Consignment Description: 1 Box Sensitive Photographic Film Material Depositor: Mr. Frank Jim. Make sure you quote the deposit details well to avoid misinformation. Write a letter of application and quote the deposit details to the given address below. Company: IMPEX TRUST COURIER COMPANY, BENIN REPUBLIC Contact Person: Dr. Chris mark (Director) Contact Email: dr_chrismark_isu at yahoo.fr contact phone number: +229 930 129 43 Forward your FULL NAME, HOME ADDRESS AND PHONE NUMBER, and register with them for immediate delivery of your package to your door step. Take good care of your self. Regards, Mr. Frank Jim. NOTE THEIR MANAGER MRS. PAT UGWU IS A VERY CLOSE FRIEND OF MY FAMILY, SHE WILL GIVE YOU SPECIAL ATTENTION IF YOU MENTION MY NAME TO HER. HAPPY CHEERS!!!!! From npiggin at suse.de Mon Mar 3 10:45:17 2008 From: npiggin at suse.de (Nick Piggin) Date: Mon, 3 Mar 2008 19:45:17 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303180605.GA3552@sgi.com> References: <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> <20080303165910.GA23998@wotan.suse.de> <20080303180605.GA3552@sgi.com> Message-ID: <20080303184517.GA4951@wotan.suse.de> On Mon, Mar 03, 2008 at 12:06:05PM -0600, Jack Steiner wrote: > On Mon, Mar 03, 2008 at 05:59:10PM +0100, Nick Piggin wrote: > > > Maintaining a long-term reference on a page is a problem. The GRU does not > > > currently maintain tables to track the pages for which dropins have been done. > > > > > > The GRU has a large internal TLB and is designed to reference up to 8PB of > > > memory. The size of the tables to track this many referenced pages would be > > > a problem (at best). > > > > Is it any worse a problem than the pagetables of the processes which have > > their virtual memory exported to GRU? AFAIKS, no; it is on the same > > magnitude of difficulty. So you could do it without introducing any > > fundamental problem (memory usage might be increased by some constant > > factor, but I think we can cope with that in order to make the core patch > > really nice and simple). > > Functionally, the GRU is very close to what I would consider to be the > "standard TLB" model. Dropins and flushs map closely to processor dropins > and flushes for cpus. The internal structure of the GRU TLB is identical to > the TLB of existing cpus. Requiring the GRU driver to track dropins with > long term page references seems to me a deviation from having the basic > mmuops support a "standard TLB" model. AFAIK, no other processor requires > this. That is because the CPU TLBs have the mmu_gather batching APIs which avoid the problem. It would be possible to do something similar for GRU which would involve taking a reference for each page-to-be-invalidated in invalidate_page, and release them when you invalidate_range. Or else do some other scheme which makes mmu notifiers work similarly to the mmu gather API. But not just go an invent something completely different in the form of this invalidate_begin,clear linux pte,invalidate_end API. > Tracking TLB dropins (and long term page references) could be done but it > adds significant complexity and scaling issues. The size of the tables to > track many TB (to PB) of memory can get large. If the memory is being > referenced by highly threaded applications, then the problem becomes even > more complex. Either tables must be replicated per-thread (and require even > more memory), or the table structure becomes even more complex to deal with > node locality, cacheline bouncing, etc. I don't think it would be that significant in terms of complexity or scaling. For a quick solution, you could stick a radix tree in each of your mmu notifiers registered (ie. one per mm), which is indexed on virtual address >> PAGE_SHIFT, and returns the struct page *. Size is no different than page tables, and locking is pretty scalable. After that, I would really like to see whether the numbers justify larger changes. From weiny2 at llnl.gov Mon Mar 3 10:58:33 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Mon, 3 Mar 2008 10:58:33 -0800 Subject: [ofa-general] Re: [PATCH] Fix bug which prevented some GUIDs from being found due to formating issues. In-Reply-To: <20080301020441.GI1485@sashak.voltaire.com> References: <20080229112509.64f1bbb4.weiny2@llnl.gov> <20080301020441.GI1485@sashak.voltaire.com> Message-ID: <20080303105833.35c8d01c.weiny2@llnl.gov> On Sat, 1 Mar 2008 02:04:41 +0000 Sasha Khapyorsky wrote: > Hi Ira, > > On 11:25 Fri 29 Feb , Ira Weiny wrote: > > > > # ========================================================================= > > +# format_guid(guid) > > +# The diags store the guids as strings. This converts the guid supplied > > +# to the correct string format. > > +# eg: 0x0008f10400411f56 == 0x8f10400411f56 > > +# > > +sub format_guid > > +{ > > + my $guid = hex $_[0]; > > + my $guid_str = ""; > > + $guid_str = sprintf("0x%016x", $guid); > > + return ($guid_str); > > +} > > And now I have this on 32-bit machine: > > $ iblinkinfo.pl -S 0x8f10400410bc0 > Integer overflow in hexadecimal number at /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi/IBswcountlimits.pm line 441. Frell! Ok, some research on the web indicates that there might be issues with how perl is compiled and if it can handle 64-bit integers on a 32-bit machine. I think it would work with bigint but here is an attempt without using ints. Ira >From 8bc9c94ae20fffcd4eff5d557ce6efb4ef1f7d38 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 28 Feb 2008 15:03:23 -0800 Subject: [PATCH] Fix bug which prevented some GUIDs from being found due to formating issues. Signed-off-by: Ira K. Weiny --- infiniband-diags/scripts/IBswcountlimits.pm | 22 +++++++++++++++++++++- infiniband-diags/scripts/ibfindnodesusing.pl | 2 +- infiniband-diags/scripts/iblinkinfo.pl | 8 +++++--- infiniband-diags/scripts/ibprintca.pl | 2 +- infiniband-diags/scripts/ibprintrt.pl | 2 +- infiniband-diags/scripts/ibprintswitch.pl | 2 +- infiniband-diags/scripts/ibqueryerrors.pl | 4 +++- 7 files changed, 33 insertions(+), 9 deletions(-) diff --git a/infiniband-diags/scripts/IBswcountlimits.pm b/infiniband-diags/scripts/IBswcountlimits.pm index bddc421..9bc356f 100755 --- a/infiniband-diags/scripts/IBswcountlimits.pm +++ b/infiniband-diags/scripts/IBswcountlimits.pm @@ -431,6 +431,26 @@ sub get_num_ports } # ========================================================================= +# format_guid(guid) +# The diags store the guids as strings. This converts the guid supplied +# to the correct string format. +# eg: 0x0008f10400411f56 == 0x8f10400411f56 +# +sub format_guid +{ + my $guid = $_[0]; + my $guid_str = ""; + + $guid =~ tr/[A-F]/[a-f]/; + if ($guid =~ /0x(.*)/) { + $guid_str = sprintf("0x%016s", $1); + } else { + $guid_str = sprintf("0x%016s", $guid); + } + return ($guid_str); +} + +# ========================================================================= # convert_dr_to_guid(direct_route) # sub convert_dr_to_guid @@ -442,7 +462,7 @@ sub convert_dr_to_guid foreach my $line (@lines) { if ($line =~ /^PortGuid:\.+(.*)/) { $guid = $1; } } - return $guid; + return format_guid($guid); } # ========================================================================= diff --git a/infiniband-diags/scripts/ibfindnodesusing.pl b/infiniband-diags/scripts/ibfindnodesusing.pl index 2521255..1bf0987 100755 --- a/infiniband-diags/scripts/ibfindnodesusing.pl +++ b/infiniband-diags/scripts/ibfindnodesusing.pl @@ -92,7 +92,7 @@ if (defined $Getopt::Std::opt_R) { $regenerate_map = $Getopt::Std::opt_R; } if (defined $Getopt::Std::opt_C) { $ca_name = $Getopt::Std::opt_C; } if (defined $Getopt::Std::opt_P) { $ca_port = $Getopt::Std::opt_P; } -my $target_switch = $ARGV[0]; +my $target_switch = format_guid($ARGV[0]); my $target_port = $ARGV[1]; get_link_ends($regenerate_map, $ca_name, $ca_port); diff --git a/infiniband-diags/scripts/iblinkinfo.pl b/infiniband-diags/scripts/iblinkinfo.pl index 195c8cf..b2c90a1 100755 --- a/infiniband-diags/scripts/iblinkinfo.pl +++ b/infiniband-diags/scripts/iblinkinfo.pl @@ -80,9 +80,11 @@ chomp $argv0; if (!getopts("hcpldRS:D:C:P:g")) { usage_and_exit $argv0; } if (defined $Getopt::Std::opt_h) { usage_and_exit $argv0; } -if (defined $Getopt::Std::opt_D) { $direct_route = $Getopt::Std::opt_D; } -if (defined $Getopt::Std::opt_R) { $regenerate_map = $Getopt::Std::opt_R; } -if (defined $Getopt::Std::opt_S) { $single_switch = $Getopt::Std::opt_S; } +if (defined $Getopt::Std::opt_D) { $direct_route = $Getopt::Std::opt_D; } +if (defined $Getopt::Std::opt_R) { $regenerate_map = $Getopt::Std::opt_R; } +if (defined $Getopt::Std::opt_S) { + $single_switch = format_guid($Getopt::Std::opt_S); +} if (defined $Getopt::Std::opt_d) { $only_down_links = $Getopt::Std::opt_d; } if (defined $Getopt::Std::opt_l) { $line_mode = $Getopt::Std::opt_l; } if (defined $Getopt::Std::opt_p) { $print_add_switch = $Getopt::Std::opt_p; } diff --git a/infiniband-diags/scripts/ibprintca.pl b/infiniband-diags/scripts/ibprintca.pl index 0c6ca0e..38b4330 100755 --- a/infiniband-diags/scripts/ibprintca.pl +++ b/infiniband-diags/scripts/ibprintca.pl @@ -67,7 +67,7 @@ if (defined $Getopt::Std::opt_l) { $list_hcas = $Getopt::Std::opt_l; } if (defined $Getopt::Std::opt_C) { $ca_name = $Getopt::Std::opt_C; } if (defined $Getopt::Std::opt_P) { $ca_port = $Getopt::Std::opt_P; } -my $target_hca = $ARGV[0]; +my $target_hca = format_guid($ARGV[0]); my $cache_file = get_cache_file($ca_name, $ca_port); diff --git a/infiniband-diags/scripts/ibprintrt.pl b/infiniband-diags/scripts/ibprintrt.pl index 2918dc6..86dcb64 100755 --- a/infiniband-diags/scripts/ibprintrt.pl +++ b/infiniband-diags/scripts/ibprintrt.pl @@ -67,7 +67,7 @@ if (defined $Getopt::Std::opt_l) { $list_rts = $Getopt::Std::opt_l; } if (defined $Getopt::Std::opt_C) { $ca_name = $Getopt::Std::opt_C; } if (defined $Getopt::Std::opt_P) { $ca_port = $Getopt::Std::opt_P; } -my $target_rt = $ARGV[0]; +my $target_rt = format_guid($ARGV[0]); my $cache_file = get_cache_file($ca_name, $ca_port); diff --git a/infiniband-diags/scripts/ibprintswitch.pl b/infiniband-diags/scripts/ibprintswitch.pl index b7f78f3..6712201 100755 --- a/infiniband-diags/scripts/ibprintswitch.pl +++ b/infiniband-diags/scripts/ibprintswitch.pl @@ -66,7 +66,7 @@ if (defined $Getopt::Std::opt_l) { $list_switches = $Getopt::Std::opt_l; } if (defined $Getopt::Std::opt_C) { $ca_name = $Getopt::Std::opt_C; } if (defined $Getopt::Std::opt_P) { $ca_port = $Getopt::Std::opt_P; } -my $target_switch = $ARGV[0]; +my $target_switch = format_guid($ARGV[0]); my $cache_file = get_cache_file($ca_name, $ca_port); diff --git a/infiniband-diags/scripts/ibqueryerrors.pl b/infiniband-diags/scripts/ibqueryerrors.pl index ef61e9b..200a40c 100755 --- a/infiniband-diags/scripts/ibqueryerrors.pl +++ b/infiniband-diags/scripts/ibqueryerrors.pl @@ -171,7 +171,9 @@ if (defined $Getopt::Std::opt_c) { if (defined $Getopt::Std::opt_r) { $report_port_info = $Getopt::Std::opt_r; } if (defined $Getopt::Std::opt_R) { $regenerate_map = $Getopt::Std::opt_R; } if (defined $Getopt::Std::opt_D) { $direct_route = $Getopt::Std::opt_D; } -if (defined $Getopt::Std::opt_S) { $single_switch = $Getopt::Std::opt_S; } +if (defined $Getopt::Std::opt_S) { + $single_switch = format_guid($Getopt::Std::opt_S); +} if (defined $Getopt::Std::opt_d) { $include_data_counters = $Getopt::Std::opt_d; } -- 1.5.1 From weiny2 at llnl.gov Mon Mar 3 10:59:23 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Mon, 3 Mar 2008 10:59:23 -0800 Subject: [ofa-general] [PATCH] infiniband-diags/scripts/ib[linkinfo][queryerrors].pl: report switch not found Message-ID: <20080303105923.7ba71092.weiny2@llnl.gov> >From 2580633c3842aac4257c6b6a330444ff7fe4a6bf Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Mon, 3 Mar 2008 10:44:56 -0800 Subject: [PATCH] infiniband-diags/scripts/ib[linkinfo][queryerrors].pl: report switch not found Signed-off-by: Ira K. Weiny --- infiniband-diags/scripts/iblinkinfo.pl | 6 ++++++ infiniband-diags/scripts/ibqueryerrors.pl | 10 +++++++++- 2 files changed, 15 insertions(+), 1 deletions(-) diff --git a/infiniband-diags/scripts/iblinkinfo.pl b/infiniband-diags/scripts/iblinkinfo.pl index b2c90a1..0d3b3ea 100755 --- a/infiniband-diags/scripts/iblinkinfo.pl +++ b/infiniband-diags/scripts/iblinkinfo.pl @@ -76,6 +76,7 @@ my $only_down_links = undef; my $ca_name = ""; my $ca_port = ""; my $print_port_guids = undef; +my $switch_found = "no"; chomp $argv0; if (!getopts("hcpldRS:D:C:P:g")) { usage_and_exit $argv0; } @@ -110,6 +111,8 @@ sub main foreach my $switch (sort (keys(%IBswcountlimits::link_ends))) { if ($single_switch && $switch ne $single_switch) { next; + } else { + $switch_found = "yes"; } my $switch_prompt = "no"; my $num_ports = get_num_ports($switch, $ca_name, $ca_port); @@ -304,6 +307,9 @@ sub main foreach my $line (@output_lines) { print $line; } } } + if ($single_switch && $switch_found ne "yes") { + printf("Switch \"%s\" not found.\n", $single_switch); + } } main; diff --git a/infiniband-diags/scripts/ibqueryerrors.pl b/infiniband-diags/scripts/ibqueryerrors.pl index 200a40c..e4e24cb 100755 --- a/infiniband-diags/scripts/ibqueryerrors.pl +++ b/infiniband-diags/scripts/ibqueryerrors.pl @@ -44,6 +44,7 @@ my $report_port_info = undef; my $single_switch = undef; my $include_data_counters = undef; my $cache_file = ""; +my $switch_found = "no"; # ========================================================================= # @@ -200,7 +201,11 @@ sub main } } foreach my $sw_addr (keys %switches) { - if ($single_switch && $sw_addr ne "$single_switch") { next; } + if ($single_switch && $sw_addr ne "$single_switch") { + next; + } else { + $switch_found = "yes"; + } my $switch_prompt = "no"; foreach my $sw_port (1 .. $switches{$sw_addr}) { @@ -214,6 +219,9 @@ sub main report_counts($sw_addr, $sw_port); } } + if ($single_switch && $switch_found ne "yes") { + printf("Switch \"%s\" not found.\n", $single_switch); + } } main; -- 1.5.1 From clameter at sgi.com Mon Mar 3 11:01:22 2008 From: clameter at sgi.com (Christoph Lameter) Date: Mon, 3 Mar 2008 11:01:22 -0800 (PST) Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303032934.GA3301@wotan.suse.de> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> Message-ID: On Mon, 3 Mar 2008, Nick Piggin wrote: > I'm still not completely happy with this. I had a very quick look > at the GRU driver, but I don't see why it can't be implemented > more like the regular TLB model, and have TLB insertions depend on > the linux pte, and do invalidates _after_ restricting permissions > to the pte. > > Ie. I'd still like to get rid of invalidate_range_begin, and get > rid of invalidate calls from places where permissions are relaxed. Isnt this more a job for paravirt ops if it is so tightly bound to page tables? Are we not adding another similar API? > If we can agree on the API, then I don't see any reason why it can't > go into 2.6.25, unless someome wants more time to review it (but > 2.6.25 release should be quite far away still so there should be quite > a bit of time). API still has rcu issues and the example given for making things sleepable is only working for the aging callback. The most important callback is for try_to_unmao and page_mkclean. This means the API is still not generic enough and likely not extendable as needed in its present form. From weiny2 at llnl.gov Mon Mar 3 11:02:06 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Mon, 3 Mar 2008 11:02:06 -0800 Subject: [ofa-general] [PATCH] Update documentation for guid format Message-ID: <20080303110206.0d9f8e0e.weiny2@llnl.gov> >From b7581a29d6ad58de01c2dc6c2bce420f300210ab Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Mon, 3 Mar 2008 11:01:00 -0800 Subject: [PATCH] Update documentation for guid format Signed-off-by: Ira K. Weiny --- infiniband-diags/man/iblinkinfo.8 | 2 +- infiniband-diags/man/ibqueryerrors.8 | 2 +- infiniband-diags/scripts/iblinkinfo.pl | 2 +- infiniband-diags/scripts/ibqueryerrors.pl | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infiniband-diags/man/iblinkinfo.8 b/infiniband-diags/man/iblinkinfo.8 index fe01af3..ebb0394 100644 --- a/infiniband-diags/man/iblinkinfo.8 +++ b/infiniband-diags/man/iblinkinfo.8 @@ -23,7 +23,7 @@ not been used for some time or if there are other reasons to believe the fabric has changed. .TP \fB\-S \fR -Output only the switch specified by +Output only the switch specified by (hex format) .TP \fB\-D \fR Output only the switch specified by the direct route path. diff --git a/infiniband-diags/man/ibqueryerrors.8 b/infiniband-diags/man/ibqueryerrors.8 index 8cde440..5c7eb17 100644 --- a/infiniband-diags/man/ibqueryerrors.8 +++ b/infiniband-diags/man/ibqueryerrors.8 @@ -43,7 +43,7 @@ the fabric has changed. Suppress the errors listed in the comma separated list provided. .TP \fB\-S \fR -Report results only for the switch specified. +Report results only for the switch specified. (hex format) .TP \fB\-D \fR Report results only for the switch specified by the direct route path. diff --git a/infiniband-diags/scripts/iblinkinfo.pl b/infiniband-diags/scripts/iblinkinfo.pl index 0d3b3ea..890567c 100755 --- a/infiniband-diags/scripts/iblinkinfo.pl +++ b/infiniband-diags/scripts/iblinkinfo.pl @@ -52,7 +52,7 @@ sub usage_and_exit " -R Recalculate ibnetdiscover information (Default is to reuse ibnetdiscover output)\n"; print " -D output only the switch specified by direct route path\n"; - print " -S output only the switch specified by guid\n"; + print " -S output only the switch specified by (hex format)\n"; print " -d print only down links\n"; print " -l (line mode) print all information for each link on each line\n"; diff --git a/infiniband-diags/scripts/ibqueryerrors.pl b/infiniband-diags/scripts/ibqueryerrors.pl index e4e24cb..c807c02 100755 --- a/infiniband-diags/scripts/ibqueryerrors.pl +++ b/infiniband-diags/scripts/ibqueryerrors.pl @@ -145,7 +145,7 @@ sub usage_and_exit print " -s suppress errors listed\n"; print " -D output only the switch specified by direct route path\n"; - print " -S query only \n"; + print " -S query only (hex format)\n"; print " -d include the data counters in the output\n"; print " -C use selected Channel Adaptor name for queries\n"; print " -P use selected channel adaptor port for queries\n"; -- 1.5.1 From clameter at sgi.com Mon Mar 3 11:02:07 2008 From: clameter at sgi.com (Christoph Lameter) Date: Mon, 3 Mar 2008 11:02:07 -0800 (PST) Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303165910.GA23998@wotan.suse.de> References: <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> <20080303165910.GA23998@wotan.suse.de> Message-ID: On Mon, 3 Mar 2008, Nick Piggin wrote: > It is going to be really easy to add more weird and wonderful notifiers > later that deviate from our standard TLB model. It would be much harder to > remove them. So I really want to see everyone conform to this model first. > Numbers and comparisons can be brought out afterwards if people want to > attempt to make such changes. Still do not see how that could be done. The model here is tightly bound to ptes. AFAICT this could be implemented in arch code like the paravirt ops. From clameter at sgi.com Mon Mar 3 11:03:27 2008 From: clameter at sgi.com (Christoph Lameter) Date: Mon, 3 Mar 2008 11:03:27 -0800 (PST) Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303033354.GC3301@wotan.suse.de> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303033354.GC3301@wotan.suse.de> Message-ID: I like the patch. From clameter at sgi.com Mon Mar 3 11:04:24 2008 From: clameter at sgi.com (Christoph Lameter) Date: Mon, 3 Mar 2008 11:04:24 -0800 (PST) Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303033428.GD3301@wotan.suse.de> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303033428.GD3301@wotan.suse.de> Message-ID: On Mon, 3 Mar 2008, Nick Piggin wrote: > Move definition of struct mmu_notifier and struct mmu_notifier_ops under > CONFIG_MMU_NOTIFIER to ensure they doesn't get dereferenced when they > don't make sense. The callbacks take a mmu_notifier parameter. So how does this compile for !MMU_NOTIFIER? From tq9_kocke at addcoinc.com Mon Mar 3 09:21:53 2008 From: tq9_kocke at addcoinc.com (clint simply) Date: Mon, 03 Mar 2008 17:21:53 +0000 Subject: [ofa-general] To: openib-general Message-ID: <04531565.20080303190915@openib.org> Voted the #1 most effective male enlargement supplement by FHM readers: V_P_X_L is guaranteed to add inches within just a few short weeks. In addition to permanent, massive gains to length, you can also expect: 1. up to 20-30% gains in girth 2. Longer sex, never subside penis 3. More volume of ejaculate Satisfaction and results are absolutely guaranteed. Why hesitate?Try the only Doctor endorsed Herbal enlargement supplement today! Click here: http://awpoloxe.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From steiner at sgi.com Mon Mar 3 11:15:40 2008 From: steiner at sgi.com (Jack Steiner) Date: Mon, 3 Mar 2008 13:15:40 -0600 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303184517.GA4951@wotan.suse.de> References: <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> <20080303165910.GA23998@wotan.suse.de> <20080303180605.GA3552@sgi.com> <20080303184517.GA4951@wotan.suse.de> Message-ID: <20080303191540.GB11156@sgi.com> On Mon, Mar 03, 2008 at 07:45:17PM +0100, Nick Piggin wrote: > On Mon, Mar 03, 2008 at 12:06:05PM -0600, Jack Steiner wrote: > > On Mon, Mar 03, 2008 at 05:59:10PM +0100, Nick Piggin wrote: > > > > Maintaining a long-term reference on a page is a problem. The GRU does not > > > > currently maintain tables to track the pages for which dropins have been done. > > > > > > > > The GRU has a large internal TLB and is designed to reference up to 8PB of > > > > memory. The size of the tables to track this many referenced pages would be > > > > a problem (at best). > > > > > > Is it any worse a problem than the pagetables of the processes which have > > > their virtual memory exported to GRU? AFAIKS, no; it is on the same > > > magnitude of difficulty. So you could do it without introducing any > > > fundamental problem (memory usage might be increased by some constant > > > factor, but I think we can cope with that in order to make the core patch > > > really nice and simple). > > > > Functionally, the GRU is very close to what I would consider to be the > > "standard TLB" model. Dropins and flushs map closely to processor dropins > > and flushes for cpus. The internal structure of the GRU TLB is identical to > > the TLB of existing cpus. Requiring the GRU driver to track dropins with > > long term page references seems to me a deviation from having the basic > > mmuops support a "standard TLB" model. AFAIK, no other processor requires > > this. > > That is because the CPU TLBs have the mmu_gather batching APIs which > avoid the problem. It would be possible to do something similar for > GRU which would involve taking a reference for each page-to-be-invalidated > in invalidate_page, and release them when you invalidate_range. Or else > do some other scheme which makes mmu notifiers work similarly to the > mmu gather API. But not just go an invent something completely different > in the form of this invalidate_begin,clear linux pte,invalidate_end API. Correct. If the mmu_gather were passed on the mmuops callout and the callout were done at the same point as the tlb_finish_mmu(), the GRU could efficiently work w/o the range invalidates. A range invalidate might still be slightly more efficient but not measureable so. The net difference is not worth the extra complexity of range callouts. > > > > Tracking TLB dropins (and long term page references) could be done but it > > adds significant complexity and scaling issues. The size of the tables to > > track many TB (to PB) of memory can get large. If the memory is being > > referenced by highly threaded applications, then the problem becomes even > > more complex. Either tables must be replicated per-thread (and require even > > more memory), or the table structure becomes even more complex to deal with > > node locality, cacheline bouncing, etc. > > I don't think it would be that significant in terms of complexity or > scaling. > > For a quick solution, you could stick a radix tree in each of your mmu > notifiers registered (ie. one per mm), which is indexed on virtual address > >> PAGE_SHIFT, and returns the struct page *. Size is no different than > page tables, and locking is pretty scalable. > > After that, I would really like to see whether the numbers justify > larger changes. I'm still concerned about performance. Each dropin would first have to access an additional data structure that would most likely be non-node-local and non-cache-resident. The net effect would be measurable but not a killer. I haven't thought about locking requirements for the radix tree. Most accesses would be read-only & updates infrequent. Any chance of an RCU-based radix implementation? Otherwise, don't we add the potential for hot locks/cachelines for threaded applications ??? From clameter at sgi.com Mon Mar 3 11:28:21 2008 From: clameter at sgi.com (Christoph Lameter) Date: Mon, 3 Mar 2008 11:28:21 -0800 (PST) Subject: [ofa-general] Re: [patch 2/6] mmu_notifier: Callbacks to invalidate address ranges In-Reply-To: <200803031611.10275.nickpiggin@yahoo.com.au> References: <20080215064859.384203497@sgi.com> <200802201008.49933.nickpiggin@yahoo.com.au> <200803031611.10275.nickpiggin@yahoo.com.au> Message-ID: On Mon, 3 Mar 2008, Nick Piggin wrote: > Your skeleton is just registering notifiers and saying > > /* you fill the hard part in */ > > If somebody needs a skeleton in order just to register the notifiers, > then almost by definition they are unqualified to write the hard > part ;) Its also providing a locking scheme. > OK, there are ways to solve it or hack around it. But this is exactly > why I think the implementations should be kept seperate. Andrea's > notifiers are coherent, work on all types of mappings, and will > hopefully match closely the regular TLB invalidation sequence in the > Linux VM (at the moment it is quite close, but I hope to make it a > bit closer) so that it requires almost no changes to the mm. Then put it into the arch code for TLB invalidation. Paravirt ops gives good examples on how to do that. > What about a completely different approach... XPmem runs over NUMAlink, > right? Why not provide some non-sleeping way to basically IPI remote > nodes over the NUMAlink where they can process the invalidation? If you > intra-node cache coherency has to run over this link anyway, then > presumably it is capable. There is another Linux instance at the remote end that first has to remove its own ptes. Also would not work for Inifiniband and other solutions. All the approaches that require evictions in an atomic context are limiting the approach and do not allow the generic functionality that we want in order to not add alternate APIs for this. > Or another idea, why don't you LD_PRELOAD in the MPT library to also > intercept munmap, mprotect, mremap etc as well as just fork()? That > would give you similarly "good enough" coherency as the mmu notifier > patches except that you can't swap (which Robin said was not a big > problem). The good enough solution right now is to pin pages by elevating refcounts. From Eleanor at cargill.com Mon Mar 3 12:10:52 2008 From: Eleanor at cargill.com (Eleanor Flanagan) Date: Mon, 03 Mar 2008 21:10:52 +0100 Subject: [ofa-general] You won't spend to much for these gifts! Message-ID: <00ba01c87d6a$ae1dc0d0$c0a80102@Eleanor> The best gifts for your nearest and dearest! The store of these chic accessories is decreasing! (http://Pimmonester.com/) -------------- next part -------------- An HTML attachment was scrubbed... URL: From terapias_florales_chile at yahoo.es Mon Mar 3 12:21:05 2008 From: terapias_florales_chile at yahoo.es (Terapias Florales Chile) Date: Mon, 03 Mar 2008 20:21:05 -0000 Subject: [ofa-general] Agresividad, temores, rabia, pena, bajo interes por la vida, etc. Message-ID: <861277-22008231310310119@Mauricio> Terapia Floral más que una moda. La vida diaria es un duro desafió para nuestros niños, adolescentes y adultos, La velocidad de estos tiempos nos abruma y muchas veces sucumbimos en el intento no encontrando el apoyo adecuado. Es por esto que la medicina complementaria o alternativa es, como su nombre lo sugiere, una alternativa que puede ayudarnos desde lo profundo de nuestras emociones. Si presenta trastornos del sueño, poca concentración, cansancio, déficit atencional, bajo rendimiento escolar o laboral, baja autoestima, agresividad, adicciones, temores, rabia, frustración, estress, pocas ganas de vivir, pena, desazón, problemas de relaciones interpersonales, problemas laborales, bajo nivel de interés por la vida, etc., La Terapia Floral puede ayudarle. Lo que produce el tratamiento es un cambio de conciencia, al quedar los sentimientos negativos lejos de la psiquis, Las flores le permiten al paciente entender por qué tiene el trastorno y hacer un cambio interno real. Pida su hora: Martes, Jueves y Sábado, de 9:00 a 17:00 horas La Concepción N°246 Of.301 � Providencia Fono: 236 39 94 � 235 12 07 Metro Pedro de Valdivia Valor Consulta $15.000.- * Pida su hora F: 236 39 94 - 235 12 07 Este mensaje se envía en base al art. 28b de la ley 19.955 que reforma la la ley de derechos del consumidor, y los artículos 2 y 4 de la ley 19.628 sobre protección de la vida privada o datos de carácter personal, todo esto en conformidad a los numerales 4 y 12 de la constitución política. Su dirección ha sido extraída manualmente por personal de nuestra compañía desde su sitio Web en Internet, o ha sido introducida por usted al aceptar el envío de mensajes publicitarios al inscribirse en alguno de los sitios o foros de nuestra Red de trabajo. Para ser removido presione Borrarme de su Base de Datos -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrea at qumranet.com Mon Mar 3 13:15:32 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Mon, 3 Mar 2008 22:15:32 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: References: <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> Message-ID: <20080303211532.GX8091@v2.random> On Mon, Mar 03, 2008 at 11:01:22AM -0800, Christoph Lameter wrote: > API still has rcu issues and the example given for making things sleepable > is only working for the aging callback. The most important callback is for > try_to_unmao and page_mkclean. This means the API is still not generic > enough and likely not extendable as needed in its present form. I converted only one of those _notify as an example of how it should be done, because I assumed you volunteer to convert the other ones yourself during .26. It's useless to convert all of them right now, because the i_mmap_lock and anon_vma locks are still going to be spinlocks in .25. From andrea at qumranet.com Mon Mar 3 13:37:07 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Mon, 3 Mar 2008 22:37:07 +0100 Subject: [ofa-general] [PATCH] mmu notifiers #v9 In-Reply-To: <20080302155457.GK8091@v2.random> References: <20080219084357.GA22249@wotan.suse.de> <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> Message-ID: <20080303213707.GA8091@v2.random> The only difference are Nick's changes (thanks Nick, nice work!) plus a fix to make it compile. About the removal of _begin I'm not strongly opposed to it, but I personally think that it's unnecessary if _begin avoids to build new data structures with a fixed ram (and cpu) cost per_page_ and at the same time deferring _end after the whole tlb_gather page freeing is reducing the number of invalidates. .26 will allow all the methods to sleep by following the roadmap described in the #v8 patch. KVM so far is swapping fine on top of this. Signed-off-by: Andrea Arcangeli Signed-off-by: Christoph Lameter Signed-off-by: Nick Piggin diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -228,6 +228,9 @@ struct mm_struct { #ifdef CONFIG_CGROUP_MEM_CONT struct mem_cgroup *mem_cgroup; #endif +#ifdef CONFIG_MMU_NOTIFIER + struct hlist_head mmu_notifier_list; +#endif }; #endif /* _LINUX_MM_TYPES_H */ diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h new file mode 100644 --- /dev/null +++ b/include/linux/mmu_notifier.h @@ -0,0 +1,194 @@ +#ifndef _LINUX_MMU_NOTIFIER_H +#define _LINUX_MMU_NOTIFIER_H + +#include +#include +#include + +struct mmu_notifier; +struct mmu_notifier_ops; + +#ifdef CONFIG_MMU_NOTIFIER + +struct mmu_notifier_ops { + /* + * Called when nobody can register any more notifier in the mm + * and after the "mn" notifier has been disarmed already. + */ + void (*release)(struct mmu_notifier *mn, + struct mm_struct *mm); + + /* + * clear_flush_young is called after the VM is + * test-and-clearing the young/accessed bitflag in the + * pte. This way the VM will provide proper aging to the + * accesses to the page through the secondary MMUs and not + * only to the ones through the Linux pte. + */ + int (*clear_flush_young)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long address); + + /* + * Before this is invoked any secondary MMU is still ok to + * read/write to the page previously pointed by the Linux pte + * because the old page hasn't been freed yet. If required + * set_page_dirty has to be called internally to this method. + */ + void (*invalidate_page)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long address); + + /* + * invalidate_range_begin() and invalidate_range_end() must be + * paired. Multiple invalidate_range_begin/ends may be nested + * or called concurrently. + */ + void (*invalidate_range_begin)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long start, unsigned long end); + void (*invalidate_range_end)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long start, unsigned long end); +}; + +struct mmu_notifier { + struct hlist_node hlist; + const struct mmu_notifier_ops *ops; +}; + +static inline int mm_has_notifiers(struct mm_struct *mm) +{ + return unlikely(!hlist_empty(&mm->mmu_notifier_list)); +} + +/* + * Must hold the mmap_sem for write. + * + * RCU is used to traverse the list. A quiescent period needs to pass + * before the notifier is guaranteed to be visible to all threads. + */ +extern void mmu_notifier_register(struct mmu_notifier *mn, + struct mm_struct *mm); +/* + * Must hold the mmap_sem for write. + * + * RCU is used to traverse the list. A quiescent period needs to pass + * before the "struct mmu_notifier" can be freed. Alternatively it + * can be synchronously freed inside ->release when the list can't + * change anymore and nobody could possibly walk it. + */ +extern void mmu_notifier_unregister(struct mmu_notifier *mn, + struct mm_struct *mm); + +extern void __mmu_notifier_release(struct mm_struct *mm); +extern int __mmu_notifier_clear_flush_young(struct mm_struct *mm, + unsigned long address); +extern void __mmu_notifier_invalidate_page(struct mm_struct *mm, + unsigned long address); +extern void __mmu_notifier_invalidate_range_begin(struct mm_struct *mm, + unsigned long start, unsigned long end); +extern void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, + unsigned long start, unsigned long end); + + +static inline void mmu_notifier_release(struct mm_struct *mm) +{ + if (mm_has_notifiers(mm)) + __mmu_notifier_release(mm); +} + +static inline int mmu_notifier_clear_flush_young(struct mm_struct *mm, + unsigned long address) +{ + if (mm_has_notifiers(mm)) + return __mmu_notifier_clear_flush_young(mm, address); + return 0; +} + +static inline void mmu_notifier_invalidate_page(struct mm_struct *mm, + unsigned long address) +{ + if (mm_has_notifiers(mm)) + __mmu_notifier_invalidate_page(mm, address); +} + +static inline void mmu_notifier_invalidate_range_begin(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + if (mm_has_notifiers(mm)) + __mmu_notifier_invalidate_range_begin(mm, start, end); +} + +static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + if (mm_has_notifiers(mm)) + __mmu_notifier_invalidate_range_end(mm, start, end); +} + +static inline void mmu_notifier_mm_init(struct mm_struct *mm) +{ + INIT_HLIST_HEAD(&mm->mmu_notifier_list); +} + + + +#define ptep_clear_flush_notify(__vma, __address, __ptep) \ +({ \ + pte_t __pte; \ + struct vm_area_struct *___vma = __vma; \ + unsigned long ___address = __address; \ + __pte = ptep_clear_flush(___vma, ___address, __ptep); \ + mmu_notifier_invalidate_page(___vma->vm_mm, ___address); \ + __pte; \ +}) + +#define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ +({ \ + int __young; \ + struct vm_area_struct *___vma = __vma; \ + unsigned long ___address = __address; \ + __young = ptep_clear_flush_young(___vma, ___address, __ptep); \ + __young |= mmu_notifier_clear_flush_young(___vma->vm_mm, \ + ___address); \ + __young; \ +}) + +#else /* CONFIG_MMU_NOTIFIER */ + +static inline void mmu_notifier_release(struct mm_struct *mm) +{ +} + +static inline int mmu_notifier_clear_flush_young(struct mm_struct *mm, + unsigned long address) +{ + return 0; +} + +static inline void mmu_notifier_invalidate_page(struct mm_struct *mm, + unsigned long address) +{ +} + +static inline void mmu_notifier_invalidate_range_begin(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ +} + +static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ +} + +static inline void mmu_notifier_mm_init(struct mm_struct *mm) +{ +} + +#define ptep_clear_flush_young_notify ptep_clear_flush_young +#define ptep_clear_flush_notify ptep_clear_flush + +#endif /* CONFIG_MMU_NOTIFIER */ + +#endif /* _LINUX_MMU_NOTIFIER_H */ diff --git a/kernel/fork.c b/kernel/fork.c --- a/kernel/fork.c +++ b/kernel/fork.c @@ -53,6 +53,7 @@ #include #include #include +#include #include #include @@ -362,6 +363,7 @@ static struct mm_struct * mm_init(struct if (likely(!mm_alloc_pgd(mm))) { mm->def_flags = 0; + mmu_notifier_mm_init(mm); return mm; } diff --git a/mm/Kconfig b/mm/Kconfig --- a/mm/Kconfig +++ b/mm/Kconfig @@ -193,3 +193,7 @@ config VIRT_TO_BUS config VIRT_TO_BUS def_bool y depends on !ARCH_NO_VIRT_TO_BUS + +config MMU_NOTIFIER + def_bool y + bool "MMU notifier, for paging KVM/RDMA" diff --git a/mm/Makefile b/mm/Makefile --- a/mm/Makefile +++ b/mm/Makefile @@ -33,4 +33,4 @@ obj-$(CONFIG_SMP) += allocpercpu.o obj-$(CONFIG_SMP) += allocpercpu.o obj-$(CONFIG_QUICKLIST) += quicklist.o obj-$(CONFIG_CGROUP_MEM_CONT) += memcontrol.o - +obj-$(CONFIG_MMU_NOTIFIER) += mmu_notifier.o diff --git a/mm/filemap_xip.c b/mm/filemap_xip.c --- a/mm/filemap_xip.c +++ b/mm/filemap_xip.c @@ -194,7 +194,7 @@ __xip_unmap (struct address_space * mapp if (pte) { /* Nuke the page table entry. */ flush_cache_page(vma, address, pte_pfn(*pte)); - pteval = ptep_clear_flush(vma, address, pte); + pteval = ptep_clear_flush_notify(vma, address, pte); page_remove_rmap(page, vma); dec_mm_counter(mm, file_rss); BUG_ON(pte_dirty(pteval)); diff --git a/mm/fremap.c b/mm/fremap.c --- a/mm/fremap.c +++ b/mm/fremap.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include @@ -214,7 +215,9 @@ asmlinkage long sys_remap_file_pages(uns spin_unlock(&mapping->i_mmap_lock); } + mmu_notifier_invalidate_range_begin(mm, start, start + size); err = populate_range(mm, vma, start, size, pgoff); + mmu_notifier_invalidate_range_end(mm, start, start + size); if (!err && !(flags & MAP_NONBLOCK)) { if (unlikely(has_write_lock)) { downgrade_write(&mm->mmap_sem); diff --git a/mm/hugetlb.c b/mm/hugetlb.c --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -755,6 +756,7 @@ void __unmap_hugepage_range(struct vm_ar BUG_ON(start & ~HPAGE_MASK); BUG_ON(end & ~HPAGE_MASK); + mmu_notifier_invalidate_range_begin(mm, start, end); spin_lock(&mm->page_table_lock); for (address = start; address < end; address += HPAGE_SIZE) { ptep = huge_pte_offset(mm, address); @@ -775,6 +777,7 @@ void __unmap_hugepage_range(struct vm_ar } spin_unlock(&mm->page_table_lock); flush_tlb_range(vma, start, end); + mmu_notifier_invalidate_range_end(mm, start, end); list_for_each_entry_safe(page, tmp, &page_list, lru) { list_del(&page->lru); put_page(page); diff --git a/mm/memory.c b/mm/memory.c --- a/mm/memory.c +++ b/mm/memory.c @@ -51,6 +51,7 @@ #include #include #include +#include #include #include @@ -611,6 +612,9 @@ int copy_page_range(struct mm_struct *ds if (is_vm_hugetlb_page(vma)) return copy_hugetlb_page_range(dst_mm, src_mm, vma); + if (is_cow_mapping(vma->vm_flags)) + mmu_notifier_invalidate_range_begin(src_mm, addr, end); + dst_pgd = pgd_offset(dst_mm, addr); src_pgd = pgd_offset(src_mm, addr); do { @@ -621,6 +625,11 @@ int copy_page_range(struct mm_struct *ds vma, addr, next)) return -ENOMEM; } while (dst_pgd++, src_pgd++, addr = next, addr != end); + + if (is_cow_mapping(vma->vm_flags)) + mmu_notifier_invalidate_range_end(src_mm, + vma->vm_start, end); + return 0; } @@ -897,7 +906,9 @@ unsigned long zap_page_range(struct vm_a lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); + mmu_notifier_invalidate_range_begin(mm, address, end); end = unmap_vmas(&tlb, vma, address, end, &nr_accounted, details); + mmu_notifier_invalidate_range_end(mm, address, end); if (tlb) tlb_finish_mmu(tlb, address, end); return end; @@ -1463,10 +1474,11 @@ int apply_to_page_range(struct mm_struct { pgd_t *pgd; unsigned long next; - unsigned long end = addr + size; + unsigned long start = addr, end = addr + size; int err; BUG_ON(addr >= end); + mmu_notifier_invalidate_range_begin(mm, start, end); pgd = pgd_offset(mm, addr); do { next = pgd_addr_end(addr, end); @@ -1474,6 +1486,7 @@ int apply_to_page_range(struct mm_struct if (err) break; } while (pgd++, addr = next, addr != end); + mmu_notifier_invalidate_range_end(mm, start, end); return err; } EXPORT_SYMBOL_GPL(apply_to_page_range); @@ -1675,7 +1688,7 @@ gotten: * seen in the presence of one thread doing SMC and another * thread doing COW. */ - ptep_clear_flush(vma, address, page_table); + ptep_clear_flush_notify(vma, address, page_table); set_pte_at(mm, address, page_table, entry); update_mmu_cache(vma, address, entry); lru_cache_add_active(new_page); diff --git a/mm/mmap.c b/mm/mmap.c --- a/mm/mmap.c +++ b/mm/mmap.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -1747,11 +1748,13 @@ static void unmap_region(struct mm_struc lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); + mmu_notifier_invalidate_range_begin(mm, start, end); unmap_vmas(&tlb, vma, start, end, &nr_accounted, NULL); vm_unacct_memory(nr_accounted); free_pgtables(&tlb, vma, prev? prev->vm_end: FIRST_USER_ADDRESS, next? next->vm_start: 0); tlb_finish_mmu(tlb, start, end); + mmu_notifier_invalidate_range_end(mm, start, end); } /* @@ -2037,6 +2040,7 @@ void exit_mmap(struct mm_struct *mm) unsigned long end; /* mm's last user has gone, and its about to be pulled down */ + mmu_notifier_release(mm); arch_exit_mmap(mm); lru_add_drain(); diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c new file mode 100644 --- /dev/null +++ b/mm/mmu_notifier.c @@ -0,0 +1,114 @@ +/* + * linux/mm/mmu_notifier.c + * + * Copyright (C) 2008 Qumranet, Inc. + * Copyright (C) 2008 SGI + * Christoph Lameter + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + */ + +#include +#include +#include + +/* + * No synchronization. This function can only be called when only a single + * process remains that performs teardown. + */ +void __mmu_notifier_release(struct mm_struct *mm) +{ + struct mmu_notifier *mn; + + while (unlikely(!hlist_empty(&mm->mmu_notifier_list))) { + mn = hlist_entry(mm->mmu_notifier_list.first, + struct mmu_notifier, + hlist); + hlist_del(&mn->hlist); + if (mn->ops->release) + mn->ops->release(mn, mm); + } +} + +/* + * If no young bitflag is supported by the hardware, ->clear_flush_young can + * unmap the address and return 1 or 0 depending if the mapping previously + * existed or not. + */ +int __mmu_notifier_clear_flush_young(struct mm_struct *mm, + unsigned long address) +{ + struct mmu_notifier *mn; + struct hlist_node *n; + int young = 0; + + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { + if (mn->ops->clear_flush_young) + young |= mn->ops->clear_flush_young(mn, mm, address); + } + rcu_read_unlock(); + + return young; +} + +void __mmu_notifier_invalidate_page(struct mm_struct *mm, + unsigned long address) +{ + struct mmu_notifier *mn; + struct hlist_node *n; + + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { + if (mn->ops->invalidate_page) + mn->ops->invalidate_page(mn, mm, address); + } + rcu_read_unlock(); +} + +void __mmu_notifier_invalidate_range_begin(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + struct mmu_notifier *mn; + struct hlist_node *n; + + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { + if (mn->ops->invalidate_range_begin) + mn->ops->invalidate_range_begin(mn, mm, start, end); + } + rcu_read_unlock(); +} + +void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + struct mmu_notifier *mn; + struct hlist_node *n; + + rcu_read_lock(); + hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { + if (mn->ops->invalidate_range_end) + mn->ops->invalidate_range_end(mn, mm, start, end); + } + rcu_read_unlock(); +} + +/* + * Note that all notifiers use RCU. The updates are only guaranteed to + * be visible to other processes after a RCU quiescent period! + * + * Must hold mmap_sem writably when calling registration functions. + */ +void mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm) +{ + hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_list); +} +EXPORT_SYMBOL_GPL(mmu_notifier_register); + +void mmu_notifier_unregister(struct mmu_notifier *mn, struct mm_struct *mm) +{ + hlist_del_rcu(&mn->hlist); +} +EXPORT_SYMBOL_GPL(mmu_notifier_unregister); diff --git a/mm/mprotect.c b/mm/mprotect.c --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -198,10 +199,12 @@ success: dirty_accountable = 1; } + mmu_notifier_invalidate_range_begin(mm, start, end); if (is_vm_hugetlb_page(vma)) hugetlb_change_protection(vma, start, end, vma->vm_page_prot); else change_protection(vma, start, end, vma->vm_page_prot, dirty_accountable); + mmu_notifier_invalidate_range_end(mm, start, end); vm_stat_account(mm, oldflags, vma->vm_file, -nrpages); vm_stat_account(mm, newflags, vma->vm_file, nrpages); return 0; diff --git a/mm/mremap.c b/mm/mremap.c --- a/mm/mremap.c +++ b/mm/mremap.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -74,6 +75,7 @@ static void move_ptes(struct vm_area_str struct mm_struct *mm = vma->vm_mm; pte_t *old_pte, *new_pte, pte; spinlock_t *old_ptl, *new_ptl; + unsigned long old_start; if (vma->vm_file) { /* @@ -100,6 +102,9 @@ static void move_ptes(struct vm_area_str spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); arch_enter_lazy_mmu_mode(); + old_start = old_addr; + mmu_notifier_invalidate_range_begin(vma->vm_mm, + old_start, old_end); for (; old_addr < old_end; old_pte++, old_addr += PAGE_SIZE, new_pte++, new_addr += PAGE_SIZE) { if (pte_none(*old_pte)) @@ -108,6 +113,7 @@ static void move_ptes(struct vm_area_str pte = move_pte(pte, new_vma->vm_page_prot, old_addr, new_addr); set_pte_at(mm, new_addr, new_pte, pte); } + mmu_notifier_invalidate_range_end(vma->vm_mm, old_start, old_end); arch_leave_lazy_mmu_mode(); if (new_ptl != old_ptl) diff --git a/mm/rmap.c b/mm/rmap.c --- a/mm/rmap.c +++ b/mm/rmap.c @@ -49,6 +49,7 @@ #include #include #include +#include #include @@ -287,7 +288,7 @@ static int page_referenced_one(struct pa if (vma->vm_flags & VM_LOCKED) { referenced++; *mapcount = 1; /* break early from loop */ - } else if (ptep_clear_flush_young(vma, address, pte)) + } else if (ptep_clear_flush_young_notify(vma, address, pte)) referenced++; /* Pretend the page is referenced if the task has the @@ -454,7 +455,7 @@ static int page_mkclean_one(struct page pte_t entry; flush_cache_page(vma, address, pte_pfn(*pte)); - entry = ptep_clear_flush(vma, address, pte); + entry = ptep_clear_flush_notify(vma, address, pte); entry = pte_wrprotect(entry); entry = pte_mkclean(entry); set_pte_at(mm, address, pte, entry); @@ -712,14 +713,14 @@ static int try_to_unmap_one(struct page * skipped over this mm) then we should reactivate it. */ if (!migration && ((vma->vm_flags & VM_LOCKED) || - (ptep_clear_flush_young(vma, address, pte)))) { + (ptep_clear_flush_young_notify(vma, address, pte)))) { ret = SWAP_FAIL; goto out_unmap; } /* Nuke the page table entry. */ flush_cache_page(vma, address, page_to_pfn(page)); - pteval = ptep_clear_flush(vma, address, pte); + pteval = ptep_clear_flush_notify(vma, address, pte); /* Move the dirty bit to the physical page now the pte is gone. */ if (pte_dirty(pteval)) @@ -844,12 +845,12 @@ static void try_to_unmap_cluster(unsigne page = vm_normal_page(vma, address, *pte); BUG_ON(!page || PageAnon(page)); - if (ptep_clear_flush_young(vma, address, pte)) + if (ptep_clear_flush_young_notify(vma, address, pte)) continue; /* Nuke the page table entry. */ flush_cache_page(vma, address, pte_pfn(*pte)); - pteval = ptep_clear_flush(vma, address, pte); + pteval = ptep_clear_flush_notify(vma, address, pte); /* If nonlinear, store the file page offset in the pte. */ if (page->index != linear_page_index(vma, address)) From andrea at qumranet.com Mon Mar 3 14:05:02 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Mon, 3 Mar 2008 23:05:02 +0100 Subject: [ofa-general] [PATCH] KVM swapping with mmu notifiers #v9 In-Reply-To: <20080303213707.GA8091@v2.random> References: <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> Message-ID: <20080303220502.GA5301@v2.random> Notably the registration now requires the mmap_sem in write mode. Signed-off-by: Andrea Arcangeli diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 41962e7..e1287ab 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -21,6 +21,7 @@ config KVM tristate "Kernel-based Virtual Machine (KVM) support" depends on HAVE_KVM && EXPERIMENTAL select PREEMPT_NOTIFIERS + select MMU_NOTIFIER select ANON_INODES ---help--- Support hosting fully virtualized guest machines using hardware diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index 4583329..4067b0f 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -642,6 +642,110 @@ static void rmap_write_protect(struct kvm *kvm, u64 gfn) account_shadowed(kvm, gfn); } +static void kvm_unmap_spte(struct kvm *kvm, u64 *spte) +{ + struct page *page = pfn_to_page((*spte & PT64_BASE_ADDR_MASK) >> PAGE_SHIFT); + get_page(page); + rmap_remove(kvm, spte); + set_shadow_pte(spte, shadow_trap_nonpresent_pte); + kvm_flush_remote_tlbs(kvm); + __free_page(page); +} + +static void kvm_unmap_rmapp(struct kvm *kvm, unsigned long *rmapp) +{ + u64 *spte, *curr_spte; + + spte = rmap_next(kvm, rmapp, NULL); + while (spte) { + BUG_ON(!(*spte & PT_PRESENT_MASK)); + rmap_printk("kvm_rmap_unmap_hva: spte %p %llx\n", spte, *spte); + curr_spte = spte; + spte = rmap_next(kvm, rmapp, spte); + kvm_unmap_spte(kvm, curr_spte); + } +} + +void kvm_unmap_hva(struct kvm *kvm, unsigned long hva) +{ + int i; + + /* + * If mmap_sem isn't taken, we can look the memslots with only + * the mmu_lock by skipping over the slots with userspace_addr == 0. + */ + spin_lock(&kvm->mmu_lock); + for (i = 0; i < kvm->nmemslots; i++) { + struct kvm_memory_slot *memslot = &kvm->memslots[i]; + unsigned long start = memslot->userspace_addr; + unsigned long end; + + /* mmu_lock protects userspace_addr */ + if (!start) + continue; + + end = start + (memslot->npages << PAGE_SHIFT); + if (hva >= start && hva < end) { + gfn_t gfn_offset = (hva - start) >> PAGE_SHIFT; + kvm_unmap_rmapp(kvm, &memslot->rmap[gfn_offset]); + } + } + spin_unlock(&kvm->mmu_lock); +} + +static int kvm_age_rmapp(struct kvm *kvm, unsigned long *rmapp) +{ + u64 *spte; + int young = 0; + + spte = rmap_next(kvm, rmapp, NULL); + while (spte) { + int _young; + u64 _spte = *spte; + BUG_ON(!(_spte & PT_PRESENT_MASK)); + _young = _spte & PT_ACCESSED_MASK; + if (_young) { + young = !!_young; + set_shadow_pte(spte, _spte & ~PT_ACCESSED_MASK); + } + spte = rmap_next(kvm, rmapp, spte); + } + return young; +} + +int kvm_age_hva(struct kvm *kvm, unsigned long hva) +{ + int i; + int young = 0; + + /* + * If mmap_sem isn't taken, we can look the memslots with only + * the mmu_lock by skipping over the slots with userspace_addr == 0. + */ + spin_lock(&kvm->mmu_lock); + for (i = 0; i < kvm->nmemslots; i++) { + struct kvm_memory_slot *memslot = &kvm->memslots[i]; + unsigned long start = memslot->userspace_addr; + unsigned long end; + + /* mmu_lock protects userspace_addr */ + if (!start) + continue; + + end = start + (memslot->npages << PAGE_SHIFT); + if (hva >= start && hva < end) { + gfn_t gfn_offset = (hva - start) >> PAGE_SHIFT; + young |= kvm_age_rmapp(kvm, &memslot->rmap[gfn_offset]); + } + } + spin_unlock(&kvm->mmu_lock); + + if (young) + kvm_flush_remote_tlbs(kvm); + + return young; +} + #ifdef MMU_DEBUG static int is_empty_shadow_page(u64 *spt) { diff --git a/arch/x86/kvm/paging_tmpl.h b/arch/x86/kvm/paging_tmpl.h index 17f9d16..b014b19 100644 --- a/arch/x86/kvm/paging_tmpl.h +++ b/arch/x86/kvm/paging_tmpl.h @@ -380,6 +380,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gva_t addr, int r; struct page *page; int largepage = 0; + unsigned mmu_seq; pgprintk("%s: addr %lx err %x\n", __FUNCTION__, addr, error_code); kvm_mmu_audit(vcpu, "pre page fault"); @@ -415,6 +416,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gva_t addr, largepage = 1; } } + mmu_seq = read_seqbegin(&vcpu->kvm->arch.mmu_notifier_invalidate_lock); page = gfn_to_page(vcpu->kvm, walker.gfn); up_read(¤t->mm->mmap_sem); @@ -440,6 +442,15 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gva_t addr, ++vcpu->stat.pf_fixed; kvm_mmu_audit(vcpu, "post page fault (fixed)"); spin_unlock(&vcpu->kvm->mmu_lock); + + if (read_seqretry(&vcpu->kvm->arch.mmu_notifier_invalidate_lock, mmu_seq)) { + down_read(¤t->mm->mmap_sem); + if (page != gfn_to_page(vcpu->kvm, walker.gfn)) + BUG(); + up_read(¤t->mm->mmap_sem); + kvm_release_page_clean(page); + } + up_read(&vcpu->kvm->slots_lock); return write_pt; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6f09840..1dfb1c9 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -3319,6 +3320,48 @@ void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) free_page((unsigned long)vcpu->arch.pio_data); } +static inline struct kvm *mmu_notifier_to_kvm(struct mmu_notifier *mn) +{ + struct kvm_arch *kvm_arch; + kvm_arch = container_of(mn, struct kvm_arch, mmu_notifier); + return container_of(kvm_arch, struct kvm, arch); +} + +void kvm_mmu_notifier_invalidate_page(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long address) +{ + struct kvm *kvm = mmu_notifier_to_kvm(mn); + BUG_ON(mm != kvm->mm); + write_seqlock(&kvm->arch.mmu_notifier_invalidate_lock); + kvm_unmap_hva(kvm, address); + write_sequnlock(&kvm->arch.mmu_notifier_invalidate_lock); +} + +void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long start, + unsigned long end) +{ + for (; start < end; start += PAGE_SIZE) + kvm_mmu_notifier_invalidate_page(mn, mm, start); +} + +int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long address) +{ + struct kvm *kvm = mmu_notifier_to_kvm(mn); + BUG_ON(mm != kvm->mm); + return kvm_age_hva(kvm, address); +} + +static const struct mmu_notifier_ops kvm_mmu_notifier_ops = { + .invalidate_page = kvm_mmu_notifier_invalidate_page, + .invalidate_range_end = kvm_mmu_notifier_invalidate_range_end, + .clear_flush_young = kvm_mmu_notifier_clear_flush_young, +}; + struct kvm *kvm_arch_create_vm(void) { struct kvm *kvm = kzalloc(sizeof(struct kvm), GFP_KERNEL); @@ -3328,6 +3371,12 @@ struct kvm *kvm_arch_create_vm(void) INIT_LIST_HEAD(&kvm->arch.active_mmu_pages); + kvm->arch.mmu_notifier.ops = &kvm_mmu_notifier_ops; + down_write(¤t->mm->mmap_sem); + mmu_notifier_register(&kvm->arch.mmu_notifier, current->mm); + up_write(¤t->mm->mmap_sem); + seqlock_init(&kvm->arch.mmu_notifier_invalidate_lock); + return kvm; } diff --git a/include/asm-x86/kvm_host.h b/include/asm-x86/kvm_host.h index 024b57c..305b7c3 100644 --- a/include/asm-x86/kvm_host.h +++ b/include/asm-x86/kvm_host.h @@ -13,6 +13,7 @@ #include #include +#include #include #include @@ -303,6 +304,9 @@ struct kvm_arch{ struct page *apic_access_page; gpa_t wall_clock; + + struct mmu_notifier mmu_notifier; + seqlock_t mmu_notifier_invalidate_lock; }; struct kvm_vm_stat { @@ -422,6 +426,8 @@ int kvm_mmu_create(struct kvm_vcpu *vcpu); int kvm_mmu_setup(struct kvm_vcpu *vcpu); void kvm_mmu_set_nonpresent_ptes(u64 trap_pte, u64 notrap_pte); +void kvm_unmap_hva(struct kvm *kvm, unsigned long hva); +int kvm_age_hva(struct kvm *kvm, unsigned long hva); int kvm_mmu_reset_context(struct kvm_vcpu *vcpu); void kvm_mmu_slot_remove_write_access(struct kvm *kvm, int slot); void kvm_mmu_zap_all(struct kvm *kvm); As usual memslot browsing with mmu_lock. Signed-off-by: Andrea Arcangeli diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6f09840..a519fd8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3379,16 +3379,23 @@ int kvm_arch_set_memory_region(struct kvm *kvm, */ if (!user_alloc) { if (npages && !old.rmap) { + unsigned long userspace_addr; + down_write(¤t->mm->mmap_sem); - memslot->userspace_addr = do_mmap(NULL, 0, - npages * PAGE_SIZE, - PROT_READ | PROT_WRITE, - MAP_SHARED | MAP_ANONYMOUS, - 0); + userspace_addr = do_mmap(NULL, 0, + npages * PAGE_SIZE, + PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_ANONYMOUS, + 0); up_write(¤t->mm->mmap_sem); - if (IS_ERR((void *)memslot->userspace_addr)) - return PTR_ERR((void *)memslot->userspace_addr); + if (IS_ERR((void *)userspace_addr)) + return PTR_ERR((void *)userspace_addr); + + /* set userspace_addr atomically for kvm_hva_to_rmapp */ + spin_lock(&kvm->mmu_lock); + memslot->userspace_addr = userspace_addr; + spin_unlock(&kvm->mmu_lock); } else { if (!old.user_alloc && old.rmap) { int ret; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 30bf832..8f3b6d6 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -326,7 +326,15 @@ int __kvm_set_memory_region(struct kvm *kvm, memset(new.rmap, 0, npages * sizeof(*new.rmap)); new.user_alloc = user_alloc; - new.userspace_addr = mem->userspace_addr; + /* + * hva_to_rmmap() serialzies with the mmu_lock and to be + * safe it has to ignore memslots with !user_alloc && + * !userspace_addr. + */ + if (user_alloc) + new.userspace_addr = mem->userspace_addr; + else + new.userspace_addr = 0; } if (npages && !new.lpage_info) { int largepages = npages / KVM_PAGES_PER_HPAGE; @@ -355,14 +363,18 @@ int __kvm_set_memory_region(struct kvm *kvm, memset(new.dirty_bitmap, 0, dirty_bytes); } + spin_lock(&kvm->mmu_lock); if (mem->slot >= kvm->nmemslots) kvm->nmemslots = mem->slot + 1; *memslot = new; + spin_unlock(&kvm->mmu_lock); r = kvm_arch_set_memory_region(kvm, mem, old, user_alloc); if (r) { + spin_lock(&kvm->mmu_lock); *memslot = old; + spin_unlock(&kvm->mmu_lock); goto out_free; } From chu11 at llnl.gov Mon Mar 3 15:08:44 2008 From: chu11 at llnl.gov (Al Chu) Date: Mon, 03 Mar 2008 15:08:44 -0800 Subject: [ofa-general] [PATCH] opensm: multi lid routing balancing for updn/minhop Message-ID: <1204585724.761.77.camel@cardanus.llnl.gov> Hey Sasha, I was originally going to submit this later on, but given the recent "rebalancing switch connections" threads, I figure perhaps was now a good time to post this patch. When we turn on lmc > 0, we noticed that sometimes extra lids from a port would be forwarded through one parent switch than another. For example, suppose LMC = 2 and we are trying to route lids (1,2,3,4). The lids can be forwarded out of 8 ports, which go to two different switches. We would see something like this: switch port 1 (to switch A): 1 switch port 2 (to switch A): 3 switch port 3 (to switch A): 4 switch port 4 (to switch A): switch port 5 (to switch B): 2 switch port 6 (to switch B) switch port 7 (to switch B): switch port 8 (to switch B): This occurs because the routing for LMC only favors those sys_guids and node_guids that have not been seen before. But it does not consider how many times we have routed through a sys_guid/node_guid before. The patch is fairly straight forward. We just count how many times we have forwarded to a sys_guid/node_guid before. If there is a port that has an equal number of paths to another port, but has not been forwarded out as much, we pick that port. Most of the patch is architectural changes. I stuff the sys_guid, node_guid, and a counter inside one struct and array, because we can't count properly using the multiple uint64_t arrays from before. Thanks, Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-support-balanced-multi-lid-routing.patch Type: text/x-patch Size: 15441 bytes Desc: not available URL: From chu11 at llnl.gov Mon Mar 3 15:15:17 2008 From: chu11 at llnl.gov (Al Chu) Date: Mon, 03 Mar 2008 15:15:17 -0800 Subject: [ofa-general] [PATCH] opensm: multi lid routing balancing for updn/minhop In-Reply-To: <1204585724.761.77.camel@cardanus.llnl.gov> References: <1204585724.761.77.camel@cardanus.llnl.gov> Message-ID: <1204586117.761.79.camel@cardanus.llnl.gov> Hey Sasha, I just noticed a comment typo. Here's a fixed patch :P Al On Mon, 2008-03-03 at 15:08 -0800, Al Chu wrote: > Hey Sasha, > > I was originally going to submit this later on, but given the recent > "rebalancing switch connections" threads, I figure perhaps was now a > good time to post this patch. > > When we turn on lmc > 0, we noticed that sometimes extra lids from a > port would be forwarded through one parent switch than another. For > example, suppose LMC = 2 and we are trying to route lids (1,2,3,4). The > lids can be forwarded out of 8 ports, which go to two different > switches. We would see something like this: > > switch port 1 (to switch A): 1 > switch port 2 (to switch A): 3 > switch port 3 (to switch A): 4 > switch port 4 (to switch A): > switch port 5 (to switch B): 2 > switch port 6 (to switch B) > switch port 7 (to switch B): > switch port 8 (to switch B): > > This occurs because the routing for LMC only favors those sys_guids and > node_guids that have not been seen before. But it does not consider how > many times we have routed through a sys_guid/node_guid before. > > The patch is fairly straight forward. We just count how many times we > have forwarded to a sys_guid/node_guid before. If there is a port that > has an equal number of paths to another port, but has not been forwarded > out as much, we pick that port. Most of the patch is architectural > changes. I stuff the sys_guid, node_guid, and a counter inside one > struct and array, because we can't count properly using the multiple > uint64_t arrays from before. > > Thanks, > Al > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-support-balanced-multi-lid-routing.patch Type: text/x-patch Size: 15411 bytes Desc: not available URL: From izike at qumranet.com Mon Mar 3 16:44:07 2008 From: izike at qumranet.com (izik eidus) Date: Tue, 04 Mar 2008 02:44:07 +0200 Subject: [ofa-general] Re: [PATCH] KVM swapping with mmu notifiers #v9 In-Reply-To: <20080303220502.GA5301@v2.random> References: <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> Message-ID: <47CC9B57.5050402@qumranet.com> ציטוט Andrea Arcangeli: > Notably the registration now requires the mmap_sem in write mode. > > Signed-off-by: Andrea Arcangeli > > diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig > index 41962e7..e1287ab 100644 > --- a/arch/x86/kvm/Kconfig > +++ b/arch/x86/kvm/Kconfig > @@ -21,6 +21,7 @@ config KVM > tristate "Kernel-based Virtual Machine (KVM) support" > depends on HAVE_KVM && EXPERIMENTAL > select PREEMPT_NOTIFIERS > + select MMU_NOTIFIER > select ANON_INODES > ---help--- > Support hosting fully virtualized guest machines using hardware > diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c > index 4583329..4067b0f 100644 > --- a/arch/x86/kvm/mmu.c > +++ b/arch/x86/kvm/mmu.c > @@ -642,6 +642,110 @@ static void rmap_write_protect(struct kvm *kvm, u64 gfn) > account_shadowed(kvm, gfn); > } > > +static void kvm_unmap_spte(struct kvm *kvm, u64 *spte) > +{ > + struct page *page = pfn_to_page((*spte & PT64_BASE_ADDR_MASK) >> PAGE_SHIFT); > + get_page(page); > + rmap_remove(kvm, spte); > + set_shadow_pte(spte, shadow_trap_nonpresent_pte); > + kvm_flush_remote_tlbs(kvm); > + __free_page(page); > i wrote to you about this before (i didnt get answer for this so i write again) with large pages support i think we need to use here put_page > +} > + > +static void kvm_unmap_rmapp(struct kvm *kvm, unsigned long *rmapp) > +{ > + u64 *spte, *curr_spte; > + > + spte = rmap_next(kvm, rmapp, NULL); > + while (spte) { > + BUG_ON(!(*spte & PT_PRESENT_MASK)); > + rmap_printk("kvm_rmap_unmap_hva: spte %p %llx\n", spte, *spte); > + curr_spte = spte; > + spte = rmap_next(kvm, rmapp, spte); > + kvm_unmap_spte(kvm, curr_spte); > + } > +} > + > +void kvm_unmap_hva(struct kvm *kvm, unsigned long hva) > +{ > + int i; > + > + /* > + * If mmap_sem isn't taken, we can look the memslots with only > + * the mmu_lock by skipping over the slots with userspace_addr == 0. > + */ > + spin_lock(&kvm->mmu_lock); > + for (i = 0; i < kvm->nmemslots; i++) { > + struct kvm_memory_slot *memslot = &kvm->memslots[i]; > + unsigned long start = memslot->userspace_addr; > + unsigned long end; > + > + /* mmu_lock protects userspace_addr */ > + if (!start) > + continue; > + > + end = start + (memslot->npages << PAGE_SHIFT); > + if (hva >= start && hva < end) { > + gfn_t gfn_offset = (hva - start) >> PAGE_SHIFT; > + kvm_unmap_rmapp(kvm, &memslot->rmap[gfn_offset]); > + } > + } > + spin_unlock(&kvm->mmu_lock); > +} > + > +static int kvm_age_rmapp(struct kvm *kvm, unsigned long *rmapp) > +{ > + u64 *spte; > + int young = 0; > + > + spte = rmap_next(kvm, rmapp, NULL); > + while (spte) { > + int _young; > + u64 _spte = *spte; > + BUG_ON(!(_spte & PT_PRESENT_MASK)); > + _young = _spte & PT_ACCESSED_MASK; > + if (_young) { > + young = !!_young; > + set_shadow_pte(spte, _spte & ~PT_ACCESSED_MASK); > + } > + spte = rmap_next(kvm, rmapp, spte); > + } > + return young; > +} > + > +int kvm_age_hva(struct kvm *kvm, unsigned long hva) > +{ > + int i; > + int young = 0; > + > + /* > + * If mmap_sem isn't taken, we can look the memslots with only > + * the mmu_lock by skipping over the slots with userspace_addr == 0. > + */ > + spin_lock(&kvm->mmu_lock); > + for (i = 0; i < kvm->nmemslots; i++) { > + struct kvm_memory_slot *memslot = &kvm->memslots[i]; > + unsigned long start = memslot->userspace_addr; > + unsigned long end; > + > + /* mmu_lock protects userspace_addr */ > + if (!start) > + continue; > + > + end = start + (memslot->npages << PAGE_SHIFT); > + if (hva >= start && hva < end) { > + gfn_t gfn_offset = (hva - start) >> PAGE_SHIFT; > + young |= kvm_age_rmapp(kvm, &memslot->rmap[gfn_offset]); > + } > + } > + spin_unlock(&kvm->mmu_lock); > + > + if (young) > + kvm_flush_remote_tlbs(kvm); > + > + return young; > +} > + > #ifdef MMU_DEBUG > static int is_empty_shadow_page(u64 *spt) > { > diff --git a/arch/x86/kvm/paging_tmpl.h b/arch/x86/kvm/paging_tmpl.h > index 17f9d16..b014b19 100644 > --- a/arch/x86/kvm/paging_tmpl.h > +++ b/arch/x86/kvm/paging_tmpl.h > @@ -380,6 +380,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gva_t addr, > int r; > struct page *page; > int largepage = 0; > + unsigned mmu_seq; > > pgprintk("%s: addr %lx err %x\n", __FUNCTION__, addr, error_code); > kvm_mmu_audit(vcpu, "pre page fault"); > @@ -415,6 +416,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gva_t addr, > largepage = 1; > } > } > + mmu_seq = read_seqbegin(&vcpu->kvm->arch.mmu_notifier_invalidate_lock); > page = gfn_to_page(vcpu->kvm, walker.gfn); > up_read(¤t->mm->mmap_sem); > > @@ -440,6 +442,15 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gva_t addr, > ++vcpu->stat.pf_fixed; > kvm_mmu_audit(vcpu, "post page fault (fixed)"); > spin_unlock(&vcpu->kvm->mmu_lock); > + > + if (read_seqretry(&vcpu->kvm->arch.mmu_notifier_invalidate_lock, mmu_seq)) { > + down_read(¤t->mm->mmap_sem); > + if (page != gfn_to_page(vcpu->kvm, walker.gfn)) > + BUG(); > + up_read(¤t->mm->mmap_sem); > + kvm_release_page_clean(page); > + } > + > up_read(&vcpu->kvm->slots_lock); > > return write_pt; > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 6f09840..1dfb1c9 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -25,6 +25,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -3319,6 +3320,48 @@ void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) > free_page((unsigned long)vcpu->arch.pio_data); > } > > +static inline struct kvm *mmu_notifier_to_kvm(struct mmu_notifier *mn) > +{ > + struct kvm_arch *kvm_arch; > + kvm_arch = container_of(mn, struct kvm_arch, mmu_notifier); > + return container_of(kvm_arch, struct kvm, arch); > +} > + > +void kvm_mmu_notifier_invalidate_page(struct mmu_notifier *mn, > + struct mm_struct *mm, > + unsigned long address) > +{ > + struct kvm *kvm = mmu_notifier_to_kvm(mn); > + BUG_ON(mm != kvm->mm); > + write_seqlock(&kvm->arch.mmu_notifier_invalidate_lock); > + kvm_unmap_hva(kvm, address); > + write_sequnlock(&kvm->arch.mmu_notifier_invalidate_lock); > +} > + > +void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, > + struct mm_struct *mm, > + unsigned long start, > + unsigned long end) > +{ > + for (; start < end; start += PAGE_SIZE) > + kvm_mmu_notifier_invalidate_page(mn, mm, start); > +} > + > +int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn, > + struct mm_struct *mm, > + unsigned long address) > +{ > + struct kvm *kvm = mmu_notifier_to_kvm(mn); > + BUG_ON(mm != kvm->mm); > + return kvm_age_hva(kvm, address); > +} > + > +static const struct mmu_notifier_ops kvm_mmu_notifier_ops = { > + .invalidate_page = kvm_mmu_notifier_invalidate_page, > + .invalidate_range_end = kvm_mmu_notifier_invalidate_range_end, > + .clear_flush_young = kvm_mmu_notifier_clear_flush_young, > +}; > + > struct kvm *kvm_arch_create_vm(void) > { > struct kvm *kvm = kzalloc(sizeof(struct kvm), GFP_KERNEL); > @@ -3328,6 +3371,12 @@ struct kvm *kvm_arch_create_vm(void) > > INIT_LIST_HEAD(&kvm->arch.active_mmu_pages); > > + kvm->arch.mmu_notifier.ops = &kvm_mmu_notifier_ops; > + down_write(¤t->mm->mmap_sem); > + mmu_notifier_register(&kvm->arch.mmu_notifier, current->mm); > + up_write(¤t->mm->mmap_sem); > + seqlock_init(&kvm->arch.mmu_notifier_invalidate_lock); > + > return kvm; > } > > diff --git a/include/asm-x86/kvm_host.h b/include/asm-x86/kvm_host.h > index 024b57c..305b7c3 100644 > --- a/include/asm-x86/kvm_host.h > +++ b/include/asm-x86/kvm_host.h > @@ -13,6 +13,7 @@ > > #include > #include > +#include > > #include > #include > @@ -303,6 +304,9 @@ struct kvm_arch{ > struct page *apic_access_page; > > gpa_t wall_clock; > + > + struct mmu_notifier mmu_notifier; > + seqlock_t mmu_notifier_invalidate_lock; > }; > > struct kvm_vm_stat { > @@ -422,6 +426,8 @@ int kvm_mmu_create(struct kvm_vcpu *vcpu); > int kvm_mmu_setup(struct kvm_vcpu *vcpu); > void kvm_mmu_set_nonpresent_ptes(u64 trap_pte, u64 notrap_pte); > > +void kvm_unmap_hva(struct kvm *kvm, unsigned long hva); > +int kvm_age_hva(struct kvm *kvm, unsigned long hva); > int kvm_mmu_reset_context(struct kvm_vcpu *vcpu); > void kvm_mmu_slot_remove_write_access(struct kvm *kvm, int slot); > void kvm_mmu_zap_all(struct kvm *kvm); > > > As usual memslot browsing with mmu_lock. > > Signed-off-by: Andrea Arcangeli > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 6f09840..a519fd8 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -3379,16 +3379,23 @@ int kvm_arch_set_memory_region(struct kvm *kvm, > */ > if (!user_alloc) { > if (npages && !old.rmap) { > + unsigned long userspace_addr; > + > down_write(¤t->mm->mmap_sem); > - memslot->userspace_addr = do_mmap(NULL, 0, > - npages * PAGE_SIZE, > - PROT_READ | PROT_WRITE, > - MAP_SHARED | MAP_ANONYMOUS, > - 0); > + userspace_addr = do_mmap(NULL, 0, > + npages * PAGE_SIZE, > + PROT_READ | PROT_WRITE, > + MAP_SHARED | MAP_ANONYMOUS, > + 0); > up_write(¤t->mm->mmap_sem); > > - if (IS_ERR((void *)memslot->userspace_addr)) > - return PTR_ERR((void *)memslot->userspace_addr); > + if (IS_ERR((void *)userspace_addr)) > + return PTR_ERR((void *)userspace_addr); > + > + /* set userspace_addr atomically for kvm_hva_to_rmapp */ > + spin_lock(&kvm->mmu_lock); > + memslot->userspace_addr = userspace_addr; > + spin_unlock(&kvm->mmu_lock); > } else { > if (!old.user_alloc && old.rmap) { > int ret; > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > index 30bf832..8f3b6d6 100644 > --- a/virt/kvm/kvm_main.c > +++ b/virt/kvm/kvm_main.c > @@ -326,7 +326,15 @@ int __kvm_set_memory_region(struct kvm *kvm, > memset(new.rmap, 0, npages * sizeof(*new.rmap)); > > new.user_alloc = user_alloc; > - new.userspace_addr = mem->userspace_addr; > + /* > + * hva_to_rmmap() serialzies with the mmu_lock and to be > + * safe it has to ignore memslots with !user_alloc && > + * !userspace_addr. > + */ > + if (user_alloc) > + new.userspace_addr = mem->userspace_addr; > + else > + new.userspace_addr = 0; > } > if (npages && !new.lpage_info) { > int largepages = npages / KVM_PAGES_PER_HPAGE; > @@ -355,14 +363,18 @@ int __kvm_set_memory_region(struct kvm *kvm, > memset(new.dirty_bitmap, 0, dirty_bytes); > } > > + spin_lock(&kvm->mmu_lock); > if (mem->slot >= kvm->nmemslots) > kvm->nmemslots = mem->slot + 1; > > *memslot = new; > + spin_unlock(&kvm->mmu_lock); > > r = kvm_arch_set_memory_region(kvm, mem, old, user_alloc); > if (r) { > + spin_lock(&kvm->mmu_lock); > *memslot = old; > + spin_unlock(&kvm->mmu_lock); > goto out_free; > } > > From jbernstein at penguincomputing.com Mon Mar 3 17:43:56 2008 From: jbernstein at penguincomputing.com (Joshua Bernstein) Date: Mon, 03 Mar 2008 17:43:56 -0800 Subject: [ofa-general] UDAPL with OFED1.2 Howto Message-ID: <47CCA95C.2070102@penguincomputing.com> Hi Everyone, I caught this post some time ago: http://www.mail-archive.com/openib-general at openib.org/msg07877.html In this post, it was asked if there was a UDAPL OFED HOWTO guide. Since this post was quite old, and nothing more recent seemed to come up in my searches, I was wondering if anybody had written any more of a howto other then whats covered in a OFED Wiki. Any pointers? -Joshua Bernstein Software Engineer Penguin Computing From timhadeen33 at gmail.com Mon Mar 3 18:24:24 2008 From: timhadeen33 at gmail.com (Tim Hadeen) Date: Mon, 3 Mar 2008 20:24:24 -0600 Subject: [ofa-general] Does QLogic have their own HCA? Message-ID: Hello, I know that we have ipath driver which is for QLogic IB part. Can somebody let us know more information on QLogic's IB HCA (Chip) offerings? Thanks Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From clameter at sgi.com Mon Mar 3 23:31:15 2008 From: clameter at sgi.com (Christoph Lameter) Date: Mon, 3 Mar 2008 23:31:15 -0800 (PST) Subject: [ofa-general] [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <47CC9B57.5050402@qumranet.com> References: <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> Message-ID: Stripped things down and did what Andrea and I talked about last Friday. No invalidate_page callbacks. No ops anymore. Simple linked list for notifier. No RCU. Added the code to rmap.h and rmap.c (after all it is concerned with handling mappings). This patch implements a simple callback for device drivers that establish their own references to pages (KVM, GRU, XPmem, RDMA/Infiniband, DMA engines etc). These references are unknown to the VM (therefore external). With these callbacks it is possible for the device driver to release external references when the VM requests it. This enables swapping, page migration and allows support of remapping, permission changes etc etc for externally mapped memory. With this functionality it becomes possible to avoid pinning or mlocking pages (commonly done to stop the VM from unmapping pages). A device driver must subscribe to a process using emm_register_notifier The VM will then perform callbacks for operations that unmap or change permissions of pages in that address space. When the process terminates the callback function is called with emm_release. Callbacks are performed before and after the unmapping action of the VM. emm_invalidate_start before emm_invalidate_end after Callbacks are mostly performed in a non atomic context. However, in various places spinlocks are held to traverse rmaps. So this patch here is only useful for those devices that can remove mappings in an atomic context (f.e. KVM/GRU). If the rmap traversal spinlocks are converted to semaphores then all callbacks willbe performed in a nonatomic context. Callouts can stay where they are. Signed-off-by: Christoph Lameter --- include/linux/mm_types.h | 3 + include/linux/rmap.h | 51 +++++++++++++++++++++++++++++++++ kernel/fork.c | 3 + mm/Kconfig | 5 +++ mm/filemap_xip.c | 5 +++ mm/fremap.c | 2 + mm/hugetlb.c | 4 ++ mm/memory.c | 32 ++++++++++++++++++-- mm/mmap.c | 3 + mm/mprotect.c | 3 + mm/mremap.c | 5 +++ mm/rmap.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++- 12 files changed, 183 insertions(+), 5 deletions(-) Index: linux-2.6/include/linux/mm_types.h =================================================================== --- linux-2.6.orig/include/linux/mm_types.h 2008-03-03 22:54:11.961264684 -0800 +++ linux-2.6/include/linux/mm_types.h 2008-03-03 22:55:13.333569600 -0800 @@ -225,6 +225,9 @@ struct mm_struct { /* aio bits */ rwlock_t ioctx_list_lock; struct kioctx *ioctx_list; +#ifdef CONFIG_EMM_NOTIFIER + struct emm_notifier *emm_notifier; +#endif #ifdef CONFIG_CGROUP_MEM_CONT struct mem_cgroup *mem_cgroup; #endif Index: linux-2.6/mm/Kconfig =================================================================== --- linux-2.6.orig/mm/Kconfig 2008-03-03 22:54:11.993264520 -0800 +++ linux-2.6/mm/Kconfig 2008-03-03 22:55:13.337569625 -0800 @@ -193,3 +193,8 @@ config NR_QUICK config VIRT_TO_BUS def_bool y depends on !ARCH_NO_VIRT_TO_BUS + +config EMM_NOTIFIER + def_bool n + bool "External Mapped Memory Notifier for drivers directly mapping memory" + Index: linux-2.6/mm/mmap.c =================================================================== --- linux-2.6.orig/mm/mmap.c 2008-03-03 22:54:12.053265354 -0800 +++ linux-2.6/mm/mmap.c 2008-03-03 22:59:25.522848812 -0800 @@ -1747,11 +1747,13 @@ static void unmap_region(struct mm_struc lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); + emm_notify(mm, emm_invalidate_start, start, end); unmap_vmas(&tlb, vma, start, end, &nr_accounted, NULL); vm_unacct_memory(nr_accounted); free_pgtables(&tlb, vma, prev? prev->vm_end: FIRST_USER_ADDRESS, next? next->vm_start: 0); tlb_finish_mmu(tlb, start, end); + emm_notify(mm, emm_invalidate_end, start, end); } /* @@ -2038,6 +2040,7 @@ void exit_mmap(struct mm_struct *mm) /* mm's last user has gone, and its about to be pulled down */ arch_exit_mmap(mm); + emm_notify(mm, emm_release, 0, TASK_SIZE); lru_add_drain(); flush_cache_mm(mm); Index: linux-2.6/mm/mprotect.c =================================================================== --- linux-2.6.orig/mm/mprotect.c 2008-03-03 22:54:12.069264942 -0800 +++ linux-2.6/mm/mprotect.c 2008-03-03 22:55:13.337569625 -0800 @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -198,10 +199,12 @@ success: dirty_accountable = 1; } + emm_notify(mm, emm_invalidate_start, start, end); if (is_vm_hugetlb_page(vma)) hugetlb_change_protection(vma, start, end, vma->vm_page_prot); else change_protection(vma, start, end, vma->vm_page_prot, dirty_accountable); + emm_notify(mm, emm_invalidate_end, start, end); vm_stat_account(mm, oldflags, vma->vm_file, -nrpages); vm_stat_account(mm, newflags, vma->vm_file, nrpages); return 0; Index: linux-2.6/mm/mremap.c =================================================================== --- linux-2.6.orig/mm/mremap.c 2008-03-03 22:54:12.077265005 -0800 +++ linux-2.6/mm/mremap.c 2008-03-03 22:59:25.530848880 -0800 @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -74,7 +75,9 @@ static void move_ptes(struct vm_area_str struct mm_struct *mm = vma->vm_mm; pte_t *old_pte, *new_pte, pte; spinlock_t *old_ptl, *new_ptl; + unsigned long old_start = old_addr; + emm_notify(mm, emm_invalidate_start, old_start, old_end); if (vma->vm_file) { /* * Subtle point from Rajesh Venkatasubramanian: before @@ -98,6 +101,7 @@ static void move_ptes(struct vm_area_str new_ptl = pte_lockptr(mm, new_pmd); if (new_ptl != old_ptl) spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); + arch_enter_lazy_mmu_mode(); for (; old_addr < old_end; old_pte++, old_addr += PAGE_SIZE, @@ -116,6 +120,7 @@ static void move_ptes(struct vm_area_str pte_unmap_unlock(old_pte - 1, old_ptl); if (mapping) spin_unlock(&mapping->i_mmap_lock); + emm_notify(mm, emm_invalidate_end, old_start, old_end); } #define LATENCY_LIMIT (64 * PAGE_SIZE) Index: linux-2.6/mm/rmap.c =================================================================== --- linux-2.6.orig/mm/rmap.c 2008-03-03 22:54:12.089265604 -0800 +++ linux-2.6/mm/rmap.c 2008-03-03 22:59:25.542848702 -0800 @@ -298,6 +298,10 @@ static int page_referenced_one(struct pa (*mapcount)--; pte_unmap_unlock(pte, ptl); + if (!referenced) + /* rmap lock held */ + referenced = emm_notify(mm, emm_referenced, + address, address + PAGE_SIZE); out: return referenced; } @@ -446,6 +450,8 @@ static int page_mkclean_one(struct page if (address == -EFAULT) goto out; + /* rmap lock held */ + emm_notify(mm, emm_invalidate_start, address, address + PAGE_SIZE); pte = page_check_address(page, mm, address, &ptl); if (!pte) goto out; @@ -462,6 +468,7 @@ static int page_mkclean_one(struct page } pte_unmap_unlock(pte, ptl); + emm_notify(mm, emm_invalidate_end, address, address + PAGE_SIZE); out: return ret; } @@ -702,9 +709,11 @@ static int try_to_unmap_one(struct page if (address == -EFAULT) goto out; + /* rmap lock held */ + emm_notify(mm, emm_invalidate_start, address, address + PAGE_SIZE); pte = page_check_address(page, mm, address, &ptl); if (!pte) - goto out; + goto out_notify; /* * If the page is mlock()d, we cannot swap it out. @@ -774,6 +783,8 @@ static int try_to_unmap_one(struct page out_unmap: pte_unmap_unlock(pte, ptl); +out_notify: + emm_notify(mm, emm_invalidate_end, address, address + PAGE_SIZE); out: return ret; } @@ -812,6 +823,7 @@ static void try_to_unmap_cluster(unsigne spinlock_t *ptl; struct page *page; unsigned long address; + unsigned long start; unsigned long end; address = (vma->vm_start + cursor) & CLUSTER_MASK; @@ -833,6 +845,8 @@ static void try_to_unmap_cluster(unsigne if (!pmd_present(*pmd)) return; + start = address; + emm_notify(mm, emm_invalidate_start, start, end); pte = pte_offset_map_lock(mm, pmd, address, &ptl); /* Update high watermark before we lower rss */ @@ -865,6 +879,7 @@ static void try_to_unmap_cluster(unsigne (*mapcount)--; } pte_unmap_unlock(pte - 1, ptl); + emm_notify(mm, emm_invalidate_end, start, end); } static int try_to_unmap_anon(struct page *page, int migration) @@ -1011,3 +1026,58 @@ int try_to_unmap(struct page *page, int return ret; } +/* + * Notifier for devices establishing their own references to Linux + * kernel pages in addition to the regular mapping via page + * table and rmap. The notifier allows the device to drop the mapping + * when the VM removes references to pages. + * + * Copyright (C) 2008 SGI + * Christoph Lameter + */ + +#ifdef CONFIG_EMM_NOTIFIER +/* + * No synchronization. This function can only be called when only a single + * process remains that performs teardown. + */ +void emm_notifier_release(struct mm_struct *mm) +{ + struct emm_notifier *e; + + while (mm->emm_notifier) { + e = mm->emm_notifier; + mm->emm_notifier = e->next; + e->func(e, mm, emm_release, 0, 0); + } +} +EXPORT_SYMBOL_GPL(emm_notifier_release); + +/* Register a notifier */ +void emm_notifier_register(struct emm_notifier *e, struct mm_struct *mm) +{ + e->next = mm->emm_notifier; + mm->emm_notifier = e; +} +EXPORT_SYMBOL_GPL(emm_notifier_register); + +/* Perform a callback */ +int __emm_notify(struct mm_struct *mm, enum emm_operations op, + unsigned long start, unsigned long end) +{ + struct emm_notifier *e = mm->emm_notifier; + int x; + + while (e) { + if (e->func) { + x = e->func(e, mm, op, start, end); + if (x) + return x; + } + e = e->next; + } + return 0; +} +EXPORT_SYMBOL_GPL(__emm_notify); +#endif + Index: linux-2.6/mm/memory.c =================================================================== --- linux-2.6.orig/mm/memory.c 2008-03-03 22:54:12.041265025 -0800 +++ linux-2.6/mm/memory.c 2008-03-03 22:59:25.502849006 -0800 @@ -611,6 +611,9 @@ int copy_page_range(struct mm_struct *ds if (is_vm_hugetlb_page(vma)) return copy_hugetlb_page_range(dst_mm, src_mm, vma); + if (is_cow_mapping(vma->vm_flags)) + emm_notify(src_mm, emm_invalidate_start, addr, end); + dst_pgd = pgd_offset(dst_mm, addr); src_pgd = pgd_offset(src_mm, addr); do { @@ -621,6 +624,10 @@ int copy_page_range(struct mm_struct *ds vma, addr, next)) return -ENOMEM; } while (dst_pgd++, src_pgd++, addr = next, addr != end); + + if (is_cow_mapping(vma->vm_flags)) + emm_notify(src_mm, emm_invalidate_end, addr, end); + return 0; } @@ -897,7 +904,11 @@ unsigned long zap_page_range(struct vm_a lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); + + /* i_mmap_lock may be held */ + emm_notify(mm, emm_invalidate_start, address, end); end = unmap_vmas(&tlb, vma, address, end, &nr_accounted, details); + emm_notify(mm, emm_invalidate_end, address, end); if (tlb) tlb_finish_mmu(tlb, address, end); return end; @@ -1340,6 +1351,7 @@ int remap_pfn_range(struct vm_area_struc pgd_t *pgd; unsigned long next; unsigned long end = addr + PAGE_ALIGN(size); + unsigned long start = addr; struct mm_struct *mm = vma->vm_mm; int err; @@ -1372,6 +1384,7 @@ int remap_pfn_range(struct vm_area_struc BUG_ON(addr >= end); pfn -= addr >> PAGE_SHIFT; pgd = pgd_offset(mm, addr); + emm_notify(mm, emm_invalidate_start, start, end); flush_cache_range(vma, addr, end); do { next = pgd_addr_end(addr, end); @@ -1380,6 +1393,7 @@ int remap_pfn_range(struct vm_area_struc if (err) break; } while (pgd++, addr = next, addr != end); + emm_notify(mm, emm_invalidate_end, start, end); return err; } EXPORT_SYMBOL(remap_pfn_range); @@ -1463,10 +1477,12 @@ int apply_to_page_range(struct mm_struct { pgd_t *pgd; unsigned long next; + unsigned long start = addr; unsigned long end = addr + size; int err; BUG_ON(addr >= end); + emm_notify(mm, emm_invalidate_start, start, end); pgd = pgd_offset(mm, addr); do { next = pgd_addr_end(addr, end); @@ -1474,6 +1490,7 @@ int apply_to_page_range(struct mm_struct if (err) break; } while (pgd++, addr = next, addr != end); + emm_notify(mm, emm_invalidate_end, start, end); return err; } EXPORT_SYMBOL_GPL(apply_to_page_range); @@ -1614,8 +1631,10 @@ static int do_wp_page(struct mm_struct * page_table = pte_offset_map_lock(mm, pmd, address, &ptl); page_cache_release(old_page); - if (!pte_same(*page_table, orig_pte)) - goto unlock; + if (!pte_same(*page_table, orig_pte)) { + pte_unmap_unlock(page_table, ptl); + goto check_dirty; + } page_mkwrite = 1; } @@ -1631,7 +1650,8 @@ static int do_wp_page(struct mm_struct * if (ptep_set_access_flags(vma, address, page_table, entry,1)) update_mmu_cache(vma, address, entry); ret |= VM_FAULT_WRITE; - goto unlock; + pte_unmap_unlock(page_table, ptl); + goto check_dirty; } /* @@ -1653,6 +1673,7 @@ gotten: if (mem_cgroup_charge(new_page, mm, GFP_KERNEL)) goto oom_free_new; + emm_notify(mm, emm_invalidate_start, address, address + PAGE_SIZE); /* * Re-check the pte - we dropped the lock */ @@ -1691,8 +1712,11 @@ gotten: page_cache_release(new_page); if (old_page) page_cache_release(old_page); -unlock: + pte_unmap_unlock(page_table, ptl); + emm_notify(mm, emm_invalidate_end, address, address + PAGE_SIZE); + +check_dirty: if (dirty_page) { if (vma->vm_file) file_update_time(vma->vm_file); Index: linux-2.6/include/linux/rmap.h =================================================================== --- linux-2.6.orig/include/linux/rmap.h 2008-02-14 15:20:13.185930864 -0800 +++ linux-2.6/include/linux/rmap.h 2008-03-03 22:55:13.341569687 -0800 @@ -133,4 +133,55 @@ static inline int page_mkclean(struct pa #define SWAP_AGAIN 1 #define SWAP_FAIL 2 +/* + * Notifier for devices establishing their own references to Linux + * kernel pages in addition to the regular mapping via page + * table and rmap. The notifier allows the device to drop the mapping + * when the VM removes references to pages. + */ +enum emm_operations { + emm_release, /* Process existing, */ + emm_invalidate_start, /* Before the VM unmaps pages */ + emm_invalidate_end, /* After the VM unmapped pages */ + emm_referenced /* Check if a range was referenced */ +}; + +struct emm_notifier { + int (*func)(struct emm_notifier *e, struct mm_struct *mm, + enum emm_operations op, + unsigned long start, unsigned long end); + struct emm_notifier *next; +}; + +extern int __emm_notify(struct mm_struct *mm, enum emm_operations op, + unsigned long start, unsigned long end); + +static inline int mm_has_emm_notifier(struct mm_struct *mm) +{ +#ifdef CONFIG_EMM_NOTIFIER + return unlikely(mm->emm_notifier); +#else + return 0; +#endif +} + +static inline int emm_notify(struct mm_struct *mm, enum emm_operations op, + unsigned long start, unsigned long end) +{ +#ifdef CONFIG_EMM_NOTIFIER + if (mm_has_emm_notifier(mm)) + return __emm_notify(mm, op, start, end); +#endif + return 0; +} + +/* + * Register a notifier with an mm struct. Release occurs when the process + * terminates by calling the notifier function with emm_release. + * + * Must hold the mmap_sem for write. + */ +extern void emm_notifier_register(struct emm_notifier *e, + struct mm_struct *mm); + #endif /* _LINUX_RMAP_H */ Index: linux-2.6/kernel/fork.c =================================================================== --- linux-2.6.orig/kernel/fork.c 2008-03-03 22:54:11.985264714 -0800 +++ linux-2.6/kernel/fork.c 2008-03-03 22:59:27.230858013 -0800 @@ -362,6 +362,9 @@ static struct mm_struct * mm_init(struct if (likely(!mm_alloc_pgd(mm))) { mm->def_flags = 0; +#ifdef CONFIG_EMM_NOTIFIER + mm->emm_notifier = NULL; +#endif return mm; } Index: linux-2.6/mm/filemap_xip.c =================================================================== --- linux-2.6.orig/mm/filemap_xip.c 2008-03-03 22:54:12.013264644 -0800 +++ linux-2.6/mm/filemap_xip.c 2008-03-03 22:59:25.474848348 -0800 @@ -190,6 +190,9 @@ __xip_unmap (struct address_space * mapp address = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT); BUG_ON(address < vma->vm_start || address >= vma->vm_end); + /* i_mmap_lock held */ + emm_notify(mm, emm_invalidate_start, + address, address + PAGE_SIZE); pte = page_check_address(page, mm, address, &ptl); if (pte) { /* Nuke the page table entry. */ @@ -201,6 +204,8 @@ __xip_unmap (struct address_space * mapp pte_unmap_unlock(pte, ptl); page_cache_release(page); } + emm_notify(mm, emm_invalidate_end, + address, address + PAGE_SIZE); } spin_unlock(&mapping->i_mmap_lock); } Index: linux-2.6/mm/fremap.c =================================================================== --- linux-2.6.orig/mm/fremap.c 2008-03-03 22:54:12.021264688 -0800 +++ linux-2.6/mm/fremap.c 2008-03-03 22:59:25.482848555 -0800 @@ -214,7 +214,9 @@ asmlinkage long sys_remap_file_pages(uns spin_unlock(&mapping->i_mmap_lock); } + emm_notify(mm, emm_invalidate_start, start, end); err = populate_range(mm, vma, start, size, pgoff); + emm_notify(mm, emm_invalidate_end, start, end); if (!err && !(flags & MAP_NONBLOCK)) { if (unlikely(has_write_lock)) { downgrade_write(&mm->mmap_sem); Index: linux-2.6/mm/hugetlb.c =================================================================== --- linux-2.6.orig/mm/hugetlb.c 2008-03-03 22:54:12.033264769 -0800 +++ linux-2.6/mm/hugetlb.c 2008-03-03 22:59:27.230858013 -0800 @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -755,6 +756,8 @@ void __unmap_hugepage_range(struct vm_ar BUG_ON(start & ~HPAGE_MASK); BUG_ON(end & ~HPAGE_MASK); + /* i_mmap_lock held */ + emm_notify(mm, emm_invalidate_start, start, end); spin_lock(&mm->page_table_lock); for (address = start; address < end; address += HPAGE_SIZE) { ptep = huge_pte_offset(mm, address); @@ -775,6 +778,7 @@ void __unmap_hugepage_range(struct vm_ar } spin_unlock(&mm->page_table_lock); flush_tlb_range(vma, start, end); + emm_notify(mm, emm_invalidate_end, start, end); list_for_each_entry_safe(page, tmp, &page_list, lru) { list_del(&page->lru); put_page(page); From clameter at sgi.com Mon Mar 3 23:34:20 2008 From: clameter at sgi.com (Christoph Lameter) Date: Mon, 3 Mar 2008 23:34:20 -0800 (PST) Subject: [ofa-general] [Early draft] Conversion of i_mmap_lock to semaphore In-Reply-To: References: <20080219135851.GI7128@v2.random> <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> Message-ID: Not there but the system boots and is usable. Complains about atomic contexts because the tlb functions use a get_cpu() and thus disable preempt. Not sure yet what to do about the cond_resched_lock stuff etc. Convert i_mmap_lock to i_mmap_sem The conversion to a rwsemaphore allows callbacks during rmap traversal for files in a non atomic context. A rw style lock allows concurrent walking of the reverse map. Signed-off-by: Christoph Lameter --- arch/x86/mm/hugetlbpage.c | 4 ++-- fs/hugetlbfs/inode.c | 4 ++-- fs/inode.c | 2 +- include/linux/fs.h | 2 +- include/linux/mm.h | 2 +- kernel/fork.c | 4 ++-- mm/filemap.c | 8 ++++---- mm/filemap_xip.c | 4 ++-- mm/fremap.c | 4 ++-- mm/hugetlb.c | 11 +++++------ mm/memory.c | 28 ++++++++-------------------- mm/migrate.c | 4 ++-- mm/mmap.c | 16 ++++++++-------- mm/mremap.c | 4 ++-- mm/rmap.c | 20 +++++++++----------- 15 files changed, 51 insertions(+), 66 deletions(-) Index: linux-2.6/arch/x86/mm/hugetlbpage.c =================================================================== --- linux-2.6.orig/arch/x86/mm/hugetlbpage.c 2008-03-03 22:59:25.386848427 -0800 +++ linux-2.6/arch/x86/mm/hugetlbpage.c 2008-03-03 22:59:31.174878038 -0800 @@ -69,7 +69,7 @@ static void huge_pmd_share(struct mm_str if (!vma_shareable(vma, addr)) return; - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(svma, &iter, &mapping->i_mmap, idx, idx) { if (svma == vma) continue; @@ -94,7 +94,7 @@ static void huge_pmd_share(struct mm_str put_page(virt_to_page(spte)); spin_unlock(&mm->page_table_lock); out: - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); } /* Index: linux-2.6/fs/hugetlbfs/inode.c =================================================================== --- linux-2.6.orig/fs/hugetlbfs/inode.c 2008-03-03 22:59:25.410848010 -0800 +++ linux-2.6/fs/hugetlbfs/inode.c 2008-03-03 22:59:31.174878038 -0800 @@ -454,10 +454,10 @@ static int hugetlb_vmtruncate(struct ino pgoff = offset >> PAGE_SHIFT; i_size_write(inode, offset); - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); if (!prio_tree_empty(&mapping->i_mmap)) hugetlb_vmtruncate_list(&mapping->i_mmap, pgoff); - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); truncate_hugepages(inode, offset); return 0; } Index: linux-2.6/fs/inode.c =================================================================== --- linux-2.6.orig/fs/inode.c 2008-03-03 22:59:25.418848099 -0800 +++ linux-2.6/fs/inode.c 2008-03-03 22:59:31.202878206 -0800 @@ -210,7 +210,7 @@ void inode_init_once(struct inode *inode INIT_LIST_HEAD(&inode->i_devices); INIT_RADIX_TREE(&inode->i_data.page_tree, GFP_ATOMIC); rwlock_init(&inode->i_data.tree_lock); - spin_lock_init(&inode->i_data.i_mmap_lock); + init_rwsem(&inode->i_data.i_mmap_sem); INIT_LIST_HEAD(&inode->i_data.private_list); spin_lock_init(&inode->i_data.private_lock); INIT_RAW_PRIO_TREE_ROOT(&inode->i_data.i_mmap); Index: linux-2.6/include/linux/fs.h =================================================================== --- linux-2.6.orig/include/linux/fs.h 2008-03-03 22:59:25.430848089 -0800 +++ linux-2.6/include/linux/fs.h 2008-03-03 22:59:31.202878206 -0800 @@ -503,7 +503,7 @@ struct address_space { unsigned int i_mmap_writable;/* count VM_SHARED mappings */ struct prio_tree_root i_mmap; /* tree of private and shared mappings */ struct list_head i_mmap_nonlinear;/*list VM_NONLINEAR mappings */ - spinlock_t i_mmap_lock; /* protect tree, count, list */ + struct rw_semaphore i_mmap_sem; /* protect tree, count, list */ unsigned int truncate_count; /* Cover race condition with truncate */ unsigned long nrpages; /* number of total pages */ pgoff_t writeback_index;/* writeback starts here */ Index: linux-2.6/include/linux/mm.h =================================================================== --- linux-2.6.orig/include/linux/mm.h 2008-03-03 22:59:25.442848167 -0800 +++ linux-2.6/include/linux/mm.h 2008-03-03 22:59:31.202878206 -0800 @@ -709,7 +709,7 @@ struct zap_details { struct address_space *check_mapping; /* Check page->mapping if set */ pgoff_t first_index; /* Lowest page->index to unmap */ pgoff_t last_index; /* Highest page->index to unmap */ - spinlock_t *i_mmap_lock; /* For unmap_mapping_range: */ + struct rw_semaphore *i_mmap_sem; /* For unmap_mapping_range: */ unsigned long truncate_count; /* Compare vm_truncate_count */ }; Index: linux-2.6/kernel/fork.c =================================================================== --- linux-2.6.orig/kernel/fork.c 2008-03-03 22:59:27.230858013 -0800 +++ linux-2.6/kernel/fork.c 2008-03-03 22:59:31.202878206 -0800 @@ -273,12 +273,12 @@ static int dup_mmap(struct mm_struct *mm atomic_dec(&inode->i_writecount); /* insert tmp into the share list, just after mpnt */ - spin_lock(&file->f_mapping->i_mmap_lock); + down_write(&file->f_mapping->i_mmap_sem); tmp->vm_truncate_count = mpnt->vm_truncate_count; flush_dcache_mmap_lock(file->f_mapping); vma_prio_tree_add(tmp, mpnt); flush_dcache_mmap_unlock(file->f_mapping); - spin_unlock(&file->f_mapping->i_mmap_lock); + up_write(&file->f_mapping->i_mmap_sem); } /* Index: linux-2.6/mm/filemap.c =================================================================== --- linux-2.6.orig/mm/filemap.c 2008-03-03 22:59:25.462848256 -0800 +++ linux-2.6/mm/filemap.c 2008-03-03 22:59:31.206878010 -0800 @@ -62,16 +62,16 @@ generic_file_direct_IO(int rw, struct ki /* * Lock ordering: * - * ->i_mmap_lock (vmtruncate) + * ->i_mmap_sem (vmtruncate) * ->private_lock (__free_pte->__set_page_dirty_buffers) * ->swap_lock (exclusive_swap_page, others) * ->mapping->tree_lock * * ->i_mutex - * ->i_mmap_lock (truncate->unmap_mapping_range) + * ->i_mmap_sem (truncate->unmap_mapping_range) * * ->mmap_sem - * ->i_mmap_lock + * ->i_mmap_sem * ->page_table_lock or pte_lock (various, mainly in memory.c) * ->mapping->tree_lock (arch-dependent flush_dcache_mmap_lock) * @@ -88,7 +88,7 @@ generic_file_direct_IO(int rw, struct ki * ->sb_lock (fs/fs-writeback.c) * ->mapping->tree_lock (__sync_single_inode) * - * ->i_mmap_lock + * ->i_mmap_sem * ->anon_vma.lock (vma_adjust) * * ->anon_vma.lock Index: linux-2.6/mm/filemap_xip.c =================================================================== --- linux-2.6.orig/mm/filemap_xip.c 2008-03-03 22:59:25.474848348 -0800 +++ linux-2.6/mm/filemap_xip.c 2008-03-03 22:59:31.206878010 -0800 @@ -184,7 +184,7 @@ __xip_unmap (struct address_space * mapp if (!page) return; - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) { mm = vma->vm_mm; address = vma->vm_start + @@ -207,7 +207,7 @@ __xip_unmap (struct address_space * mapp emm_notify(mm, emm_invalidate_end, address, address + PAGE_SIZE); } - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); } /* Index: linux-2.6/mm/fremap.c =================================================================== --- linux-2.6.orig/mm/fremap.c 2008-03-03 22:59:25.482848555 -0800 +++ linux-2.6/mm/fremap.c 2008-03-03 22:59:31.206878010 -0800 @@ -205,13 +205,13 @@ asmlinkage long sys_remap_file_pages(uns } goto out; } - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); flush_dcache_mmap_lock(mapping); vma->vm_flags |= VM_NONLINEAR; vma_prio_tree_remove(vma, &mapping->i_mmap); vma_nonlinear_insert(vma, &mapping->i_mmap_nonlinear); flush_dcache_mmap_unlock(mapping); - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); } emm_notify(mm, emm_invalidate_start, start, end); Index: linux-2.6/mm/hugetlb.c =================================================================== --- linux-2.6.orig/mm/hugetlb.c 2008-03-03 22:59:27.230858013 -0800 +++ linux-2.6/mm/hugetlb.c 2008-03-03 22:59:31.206878010 -0800 @@ -746,7 +746,7 @@ void __unmap_hugepage_range(struct vm_ar struct page *page; struct page *tmp; /* - * A page gathering list, protected by per file i_mmap_lock. The + * A page gathering list, protected by per file i_mmap_sem. The * lock is used to avoid list corruption from multiple unmapping * of the same page since we are using page->lru. */ @@ -756,7 +756,6 @@ void __unmap_hugepage_range(struct vm_ar BUG_ON(start & ~HPAGE_MASK); BUG_ON(end & ~HPAGE_MASK); - /* i_mmap_lock held */ emm_notify(mm, emm_invalidate_start, start, end); spin_lock(&mm->page_table_lock); for (address = start; address < end; address += HPAGE_SIZE) { @@ -797,9 +796,9 @@ void unmap_hugepage_range(struct vm_area * do nothing in this case. */ if (vma->vm_file) { - spin_lock(&vma->vm_file->f_mapping->i_mmap_lock); + down_write(&vma->vm_file->f_mapping->i_mmap_sem); __unmap_hugepage_range(vma, start, end); - spin_unlock(&vma->vm_file->f_mapping->i_mmap_lock); + up_write(&vma->vm_file->f_mapping->i_mmap_sem); } } @@ -1042,7 +1041,7 @@ void hugetlb_change_protection(struct vm BUG_ON(address >= end); flush_cache_range(vma, address, end); - spin_lock(&vma->vm_file->f_mapping->i_mmap_lock); + down_read(&vma->vm_file->f_mapping->i_mmap_sem); spin_lock(&mm->page_table_lock); for (; address < end; address += HPAGE_SIZE) { ptep = huge_pte_offset(mm, address); @@ -1057,7 +1056,7 @@ void hugetlb_change_protection(struct vm } } spin_unlock(&mm->page_table_lock); - spin_unlock(&vma->vm_file->f_mapping->i_mmap_lock); + up_read(&vma->vm_file->f_mapping->i_mmap_sem); flush_tlb_range(vma, start, end); } Index: linux-2.6/mm/memory.c =================================================================== --- linux-2.6.orig/mm/memory.c 2008-03-03 22:59:25.502849006 -0800 +++ linux-2.6/mm/memory.c 2008-03-03 22:59:31.206878010 -0800 @@ -830,7 +830,6 @@ unsigned long unmap_vmas(struct mmu_gath unsigned long tlb_start = 0; /* For tlb_finish_mmu */ int tlb_start_valid = 0; unsigned long start = start_addr; - spinlock_t *i_mmap_lock = details? details->i_mmap_lock: NULL; int fullmm = (*tlbp)->fullmm; for ( ; vma && vma->vm_start < end_addr; vma = vma->vm_next) { @@ -868,21 +867,11 @@ unsigned long unmap_vmas(struct mmu_gath tlb_finish_mmu(*tlbp, tlb_start, start); - if (need_resched() || - (i_mmap_lock && spin_needbreak(i_mmap_lock))) { - if (i_mmap_lock) { - *tlbp = NULL; - goto out; - } - cond_resched(); - } - *tlbp = tlb_gather_mmu(vma->vm_mm, fullmm); tlb_start_valid = 0; zap_work = ZAP_BLOCK_SIZE; } } -out: return start; /* which is now the end (or restart) address */ } @@ -905,7 +894,6 @@ unsigned long zap_page_range(struct vm_a tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); - /* i_mmap_lock may be held */ emm_notify(mm, emm_invalidate_start, address, end); end = unmap_vmas(&tlb, vma, address, end, &nr_accounted, details); emm_notify(mm, emm_invalidate_end, address, end); @@ -1749,7 +1737,7 @@ unwritable_page: /* * Helper functions for unmap_mapping_range(). * - * __ Notes on dropping i_mmap_lock to reduce latency while unmapping __ + * __ Notes on dropping i_mmap_sem to reduce latency while unmapping __ * * We have to restart searching the prio_tree whenever we drop the lock, * since the iterator is only valid while the lock is held, and anyway @@ -1768,7 +1756,7 @@ unwritable_page: * can't efficiently keep all vmas in step with mapping->truncate_count: * so instead reset them all whenever it wraps back to 0 (then go to 1). * mapping->truncate_count and vma->vm_truncate_count are protected by - * i_mmap_lock. + * i_mmap_sem. * * In order to make forward progress despite repeatedly restarting some * large vma, note the restart_addr from unmap_vmas when it breaks out: @@ -1818,7 +1806,7 @@ again: restart_addr = zap_page_range(vma, start_addr, end_addr - start_addr, details); - need_break = need_resched() || spin_needbreak(details->i_mmap_lock); + need_break = need_resched(); if (restart_addr >= end_addr) { /* We have now completed this vma: mark it so */ @@ -1832,9 +1820,9 @@ again: goto again; } - spin_unlock(details->i_mmap_lock); + up_write(details->i_mmap_sem); cond_resched(); - spin_lock(details->i_mmap_lock); + down_write(details->i_mmap_sem); return -EINTR; } @@ -1928,9 +1916,9 @@ void unmap_mapping_range(struct address_ details.last_index = hba + hlen - 1; if (details.last_index < details.first_index) details.last_index = ULONG_MAX; - details.i_mmap_lock = &mapping->i_mmap_lock; + details.i_mmap_sem = &mapping->i_mmap_sem; - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); /* Protect against endless unmapping loops */ mapping->truncate_count++; @@ -1945,7 +1933,7 @@ void unmap_mapping_range(struct address_ unmap_mapping_range_tree(&mapping->i_mmap, &details); if (unlikely(!list_empty(&mapping->i_mmap_nonlinear))) unmap_mapping_range_list(&mapping->i_mmap_nonlinear, &details); - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); } EXPORT_SYMBOL(unmap_mapping_range); Index: linux-2.6/mm/migrate.c =================================================================== --- linux-2.6.orig/mm/migrate.c 2008-03-03 22:59:25.510849324 -0800 +++ linux-2.6/mm/migrate.c 2008-03-03 22:59:31.206878010 -0800 @@ -202,12 +202,12 @@ static void remove_file_migration_ptes(s if (!mapping) return; - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) remove_migration_pte(vma, old, new); - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); } /* Index: linux-2.6/mm/mmap.c =================================================================== --- linux-2.6.orig/mm/mmap.c 2008-03-03 22:59:25.522848812 -0800 +++ linux-2.6/mm/mmap.c 2008-03-03 22:59:31.210878368 -0800 @@ -186,7 +186,7 @@ error: } /* - * Requires inode->i_mapping->i_mmap_lock + * Requires inode->i_mapping->i_mmap_sem */ static void __remove_shared_vm_struct(struct vm_area_struct *vma, struct file *file, struct address_space *mapping) @@ -214,9 +214,9 @@ void unlink_file_vma(struct vm_area_stru if (file) { struct address_space *mapping = file->f_mapping; - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); __remove_shared_vm_struct(vma, file, mapping); - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); } } @@ -439,7 +439,7 @@ static void vma_link(struct mm_struct *m mapping = vma->vm_file->f_mapping; if (mapping) { - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); vma->vm_truncate_count = mapping->truncate_count; } anon_vma_lock(vma); @@ -449,7 +449,7 @@ static void vma_link(struct mm_struct *m anon_vma_unlock(vma); if (mapping) - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); mm->map_count++; validate_mm(mm); @@ -536,7 +536,7 @@ again: remove_next = 1 + (end > next-> mapping = file->f_mapping; if (!(vma->vm_flags & VM_NONLINEAR)) root = &mapping->i_mmap; - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); if (importer && vma->vm_truncate_count != next->vm_truncate_count) { /* @@ -620,7 +620,7 @@ again: remove_next = 1 + (end > next-> if (anon_vma) spin_unlock(&anon_vma->lock); if (mapping) - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); if (remove_next) { if (file) @@ -2064,7 +2064,7 @@ void exit_mmap(struct mm_struct *mm) /* Insert vm structure into process list sorted by address * and into the inode's i_mmap tree. If vm_file is non-NULL - * then i_mmap_lock is taken here. + * then i_mmap_sem is taken here. */ int insert_vm_struct(struct mm_struct * mm, struct vm_area_struct * vma) { Index: linux-2.6/mm/mremap.c =================================================================== --- linux-2.6.orig/mm/mremap.c 2008-03-03 22:59:25.530848880 -0800 +++ linux-2.6/mm/mremap.c 2008-03-03 22:59:31.210878368 -0800 @@ -86,7 +86,7 @@ static void move_ptes(struct vm_area_str * and we propagate stale pages into the dst afterward. */ mapping = vma->vm_file->f_mapping; - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); if (new_vma->vm_truncate_count && new_vma->vm_truncate_count != vma->vm_truncate_count) new_vma->vm_truncate_count = 0; @@ -119,7 +119,7 @@ static void move_ptes(struct vm_area_str pte_unmap_nested(new_pte - 1); pte_unmap_unlock(old_pte - 1, old_ptl); if (mapping) - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); emm_notify(mm, emm_invalidate_end, old_start, old_end); } Index: linux-2.6/mm/rmap.c =================================================================== --- linux-2.6.orig/mm/rmap.c 2008-03-03 22:59:25.542848702 -0800 +++ linux-2.6/mm/rmap.c 2008-03-03 22:59:31.210878368 -0800 @@ -24,7 +24,7 @@ * inode->i_alloc_sem (vmtruncate_range) * mm->mmap_sem * page->flags PG_locked (lock_page) - * mapping->i_mmap_lock + * mapping->i_mmap_sem * anon_vma->lock * mm->page_table_lock or pte_lock * zone->lru_lock (in mark_page_accessed, isolate_lru_page) @@ -368,14 +368,14 @@ static int page_referenced_file(struct p * The page lock not only makes sure that page->mapping cannot * suddenly be NULLified by truncation, it makes sure that the * structure at mapping cannot be freed and reused yet, - * so we can safely take mapping->i_mmap_lock. + * so we can safely take mapping->i_mmap_sem. */ BUG_ON(!PageLocked(page)); - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); /* - * i_mmap_lock does not stabilize mapcount at all, but mapcount + * i_mmap_sem does not stabilize mapcount at all, but mapcount * is more likely to be accurate if we note it after spinning. */ mapcount = page_mapcount(page); @@ -398,7 +398,7 @@ static int page_referenced_file(struct p break; } - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); return referenced; } @@ -482,12 +482,12 @@ static int page_mkclean_file(struct addr BUG_ON(PageAnon(page)); - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) { if (vma->vm_flags & VM_SHARED) ret += page_mkclean_one(page, vma); } - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); return ret; } @@ -923,7 +923,7 @@ static int try_to_unmap_file(struct page unsigned long max_nl_size = 0; unsigned int mapcount; - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) { ret = try_to_unmap_one(page, vma, migration); if (ret == SWAP_FAIL || !page_mapped(page)) @@ -960,7 +960,6 @@ static int try_to_unmap_file(struct page mapcount = page_mapcount(page); if (!mapcount) goto out; - cond_resched_lock(&mapping->i_mmap_lock); max_nl_size = (max_nl_size + CLUSTER_SIZE - 1) & CLUSTER_MASK; if (max_nl_cursor == 0) @@ -982,7 +981,6 @@ static int try_to_unmap_file(struct page } vma->vm_private_data = (void *) max_nl_cursor; } - cond_resched_lock(&mapping->i_mmap_lock); max_nl_cursor += CLUSTER_SIZE; } while (max_nl_cursor <= max_nl_size); @@ -994,7 +992,7 @@ static int try_to_unmap_file(struct page list_for_each_entry(vma, &mapping->i_mmap_nonlinear, shared.vm_set.list) vma->vm_private_data = NULL; out: - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); return ret; } From tgree at relay.phys.ualberta.ca Mon Mar 3 23:59:18 2008 From: tgree at relay.phys.ualberta.ca (Terry Greeniaus) Date: Tue, 4 Mar 2008 00:59:18 -0700 (MST) Subject: [ofa-general] [OpenSM] Traps 64 and 65 sent too early. Message-ID: Hi, This is my first post to the list, so hopefully I get the netiquette right. We are using OpenSM 3.1.5 on our fabric. Our device is subscribing for the in and out of service traps (64 and 65) described in section 14.4.9 of the IBA. Immediately after receive an "in service" trap, we issue a SubnAdmGet(PortInfoRecord) query to the SA. In the response, we see that PortState is still set to Initialize. Section 14.4.9 p 880 states: "The events reported by these Notice()s indicate that an endport's reachability from a subscribing endport has changed. Endport A is reachable from subscriber endport B if there is a PathRecord with source B and destination A." Section 15.5.2.16 (PathRecord) p 899 states: "For unicast communication, a PathRecord specifying a path from SGID A to DGID B exists if all the following conditions hold: - there is a LID-routed path from port A to port B, - PortInfo:PortState of port A is Active, - PortInfo:PortState is Armed or Active for all intermediate switch and router ports in the path from port A to port B, and - PortInfo:PortState of port B is Armed or Active." I believe this implies that the port that has come into service and been reported via an SM Notice should at least be in the Armed state. Even better for us would be if the port was in the Active state, but I can't make a case for that by the IBA. I think this could be fixed by moving the __osm_state_mgr_report_new_ports() call in osm_state_mgr.c to follow the call that transitions the subnet to Armed or Active, however I am not very familiar with the OpenSM code so am appealing to the list to help me out. Also, perhaps there is a bug tracker somewhere against which I should file a bug report? Thanks, TG From erezz at Voltaire.COM Tue Mar 4 00:51:17 2008 From: erezz at Voltaire.COM (Erez Zilber) Date: Tue, 04 Mar 2008 10:51:17 +0200 Subject: [ofa-general] Regarding the branches in infiniband.git Message-ID: <47CD0D85.9070602@Voltaire.COM> Roland, I've noticed that your infiniband.git tree contains some branches. I understand that "for-2.6.25" is used for 2.6.25 work etc. What is the purpose of the "master" branch? Thanks, Erez From marian at mobilpost.com Mon Mar 3 23:43:52 2008 From: marian at mobilpost.com (frederick kerry) Date: Tue, 04 Mar 2008 07:43:52 +0000 Subject: [ofa-general] Time Limited Offer! Buy CIALIS Today And Get 25% OFF! Message-ID: <000601c87dda$06dc228a$dc650dba@gikdxy> the antibiotic erythromycin (E-Mycin, Eryc, Ery-Tab, others); an antifungal medication such as itraconazole (Sporanox) or ketoconazole (Nizoral); or HIV medications such as indinavir (Crixivan), ritonavir (Norvir), and others. Buy Ci-al-is Online. LOWEST GENERIC CI-AL-IS PR-I-CE Ind1an cheeap meedss -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Tue Mar 4 03:08:32 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Tue, 4 Mar 2008 03:08:32 -0800 (PST) Subject: [ofa-general] ofa_1_3_kernel 20080304-0200 daily build status Message-ID: <20080304110832.34F5DE609F8@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From erezz at voltaire.com Tue Mar 4 04:07:22 2008 From: erezz at voltaire.com (Erez Zilber) Date: Tue, 04 Mar 2008 14:07:22 +0200 Subject: [ofa-general] Re: [PATCH 1/2] IB/iSER: fix list iteration bug In-Reply-To: <877igkxffl.fsf@confield.dd.xiranet.com> References: <877igkxffl.fsf@confield.dd.xiranet.com> Message-ID: <47CD3B7A.2010106@voltaire.com> Arne Redlich wrote: > The iteration through the list of "iser_device"s during device > lookup/creation is broken - it might result in an infinite loop if more > than 1 HCA is used with iSER. Use list_for_each_entry() instead of the > custom, flawed list iteration code. > > Signed-off-by: Arne Redlich > --- > drivers/infiniband/ulp/iser/iser_verbs.c | 36 ++++++++++++----------------- > 1 files changed, 15 insertions(+), 21 deletions(-) > > diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c > index 714b8db..1c0f968 100644 > --- a/drivers/infiniband/ulp/iser/iser_verbs.c > +++ b/drivers/infiniband/ulp/iser/iser_verbs.c > @@ -237,33 +237,27 @@ static int iser_free_ib_conn_res(struct iser_conn *ib_conn) > static > struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) > { > - struct list_head *p_list; > - struct iser_device *device = NULL; > + struct iser_device *device; > > mutex_lock(&ig.device_list_mutex); > > - p_list = ig.device_list.next; > - while (p_list != &ig.device_list) { > - device = list_entry(p_list, struct iser_device, ig_list); > - /* find if there's a match using the node GUID */ > + list_for_each_entry(device, &ig.device_list, ig_list) I've just added the original comments that are missing in your patch. > if (device->ib_device->node_guid == cma_id->device->node_guid) > - break; > - } > - > - if (device == NULL) { > - device = kzalloc(sizeof *device, GFP_KERNEL); > - if (device == NULL) > goto out; > - /* assign this device to the device */ > - device->ib_device = cma_id->device; > - /* init the device and link it into ig device list */ > - if (iser_create_device_ib_res(device)) { > - kfree(device); > - device = NULL; > - goto out; > - } > - list_add(&device->ig_list, &ig.device_list); > + > + device = kzalloc(sizeof *device, GFP_KERNEL); > + if (device == NULL) > + goto out; > + > + device->ib_device = cma_id->device; > + /* init the device and link it into ig device list */ > + if (iser_create_device_ib_res(device)) { > + kfree(device); > + device = NULL; > + goto out; > } > + list_add(&device->ig_list, &ig.device_list); > + > out: > BUG_ON(device == NULL); > device->refcount++; The iteration through the list of "iser_device"s during device lookup/creation is broken - it might result in an infinite loop if more than 1 HCA is used with iSER. Use list_for_each_entry() instead of the custom, flawed list iteration code. Signed-off-by: Arne Redlich Signed-off-by: Erez Zilber --- drivers/infiniband/ulp/iser/iser_verbs.c | 36 +++++++++++++---------------- 1 files changed, 16 insertions(+), 20 deletions(-) diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c index 714b8db..768ba69 100644 --- a/drivers/infiniband/ulp/iser/iser_verbs.c +++ b/drivers/infiniband/ulp/iser/iser_verbs.c @@ -237,33 +237,29 @@ static int iser_free_ib_conn_res(struct iser_conn *ib_conn) static struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) { - struct list_head *p_list; - struct iser_device *device = NULL; + struct iser_device *device; mutex_lock(&ig.device_list_mutex); - p_list = ig.device_list.next; - while (p_list != &ig.device_list) { - device = list_entry(p_list, struct iser_device, ig_list); + list_for_each_entry(device, &ig.device_list, ig_list) /* find if there's a match using the node GUID */ if (device->ib_device->node_guid == cma_id->device->node_guid) - break; - } - - if (device == NULL) { - device = kzalloc(sizeof *device, GFP_KERNEL); - if (device == NULL) goto out; - /* assign this device to the device */ - device->ib_device = cma_id->device; - /* init the device and link it into ig device list */ - if (iser_create_device_ib_res(device)) { - kfree(device); - device = NULL; - goto out; - } - list_add(&device->ig_list, &ig.device_list); + + device = kzalloc(sizeof *device, GFP_KERNEL); + if (device == NULL) + goto out; + + /* assign this device to the device */ + device->ib_device = cma_id->device; + /* init the device and link it into ig device list */ + if (iser_create_device_ib_res(device)) { + kfree(device); + device = NULL; + goto out; } + list_add(&device->ig_list, &ig.device_list); + out: BUG_ON(device == NULL); device->refcount++; -- 1.5.3.6 I agree with your patch. It seems that we forgot to add something like p_list=p_list->next. Anyway, using list_for_each_entry is better than what we had. From erezz at voltaire.com Tue Mar 4 04:11:54 2008 From: erezz at voltaire.com (Erez Zilber) Date: Tue, 04 Mar 2008 14:11:54 +0200 Subject: [ofa-general] Re: [PATCH 2/2] IB/iSER: handle iser_device allocation error gracefully In-Reply-To: <87zltgw0ut.fsf@confield.dd.xiranet.com> References: <87zltgw0ut.fsf@confield.dd.xiranet.com> Message-ID: <47CD3C8A.20405@voltaire.com> Arne Redlich wrote: > "iser_device" allocation failure is "handled" with a BUG_ON() right > before dereferencing the NULL-pointer - fix this! > I agree with this patch. I'm resending it because it depends on the 1st patch that has changed, so the 2nd patch can be applied over the new 1st patch. > Signed-off-by: Arne Redlich > --- > drivers/infiniband/ulp/iser/iser_verbs.c | 13 +++++++++---- > 1 files changed, 9 insertions(+), 4 deletions(-) > > diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c > index 1c0f968..cf70d15 100644 > --- a/drivers/infiniband/ulp/iser/iser_verbs.c > +++ b/drivers/infiniband/ulp/iser/iser_verbs.c > @@ -243,7 +243,7 @@ struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) > > list_for_each_entry(device, &ig.device_list, ig_list) > if (device->ib_device->node_guid == cma_id->device->node_guid) > - goto out; > + goto inc_refcnt; > > device = kzalloc(sizeof *device, GFP_KERNEL); > if (device == NULL) > @@ -258,9 +258,9 @@ struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) > } > list_add(&device->ig_list, &ig.device_list); > > -out: > - BUG_ON(device == NULL); > +inc_refcnt: > device->refcount++; > +out: > mutex_unlock(&ig.device_list_mutex); > return device; > } > @@ -366,6 +366,12 @@ static void iser_addr_handler(struct rdma_cm_id *cma_id) > int ret; > > device = iser_device_find_by_ib_device(cma_id); > + if (!device) { > + iser_err("device lookup/creation failed\n"); > + iser_connect_error(cma_id); > + return; > + } > + > ib_conn = (struct iser_conn *)cma_id->context; > ib_conn->device = device; > > @@ -374,7 +380,6 @@ static void iser_addr_handler(struct rdma_cm_id *cma_id) > iser_err("resolve route failed: %d\n", ret); > iser_connect_error(cma_id); > } > - return; > } > > static void iser_route_handler(struct rdma_cm_id *cma_id) iser_device" allocation failure is "handled" with a BUG_ON() right before dereferencing the NULL-pointer - fix this! Signed-off-by: Arne Redlich Signed-off-by: Erez Zilber --- drivers/infiniband/ulp/iser/iser_verbs.c | 13 +++++++++---- 1 files changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c index 768ba69..993f0a8 100644 --- a/drivers/infiniband/ulp/iser/iser_verbs.c +++ b/drivers/infiniband/ulp/iser/iser_verbs.c @@ -244,7 +244,7 @@ struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) list_for_each_entry(device, &ig.device_list, ig_list) /* find if there's a match using the node GUID */ if (device->ib_device->node_guid == cma_id->device->node_guid) - goto out; + goto inc_refcnt; device = kzalloc(sizeof *device, GFP_KERNEL); if (device == NULL) @@ -260,9 +260,9 @@ struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) } list_add(&device->ig_list, &ig.device_list); -out: - BUG_ON(device == NULL); +inc_refcnt: device->refcount++; +out: mutex_unlock(&ig.device_list_mutex); return device; } @@ -368,6 +368,12 @@ static void iser_addr_handler(struct rdma_cm_id *cma_id) int ret; device = iser_device_find_by_ib_device(cma_id); + if (!device) { + iser_err("device lookup/creation failed\n"); + iser_connect_error(cma_id); + return; + } + ib_conn = (struct iser_conn *)cma_id->context; ib_conn->device = device; @@ -376,7 +382,6 @@ static void iser_addr_handler(struct rdma_cm_id *cma_id) iser_err("resolve route failed: %d\n", ret); iser_connect_error(cma_id); } - return; } static void iser_route_handler(struct rdma_cm_id *cma_id) -- 1.5.3.6 From nickpiggin at yahoo.com.au Mon Mar 3 11:50:10 2008 From: nickpiggin at yahoo.com.au (Nick Piggin) Date: Tue, 4 Mar 2008 06:50:10 +1100 Subject: [ofa-general] Re: [patch 2/6] mmu_notifier: Callbacks to invalidate address ranges In-Reply-To: References: <20080215064859.384203497@sgi.com> <200803031611.10275.nickpiggin@yahoo.com.au> Message-ID: <200803040650.11942.nickpiggin@yahoo.com.au> On Tuesday 04 March 2008 06:28, Christoph Lameter wrote: > On Mon, 3 Mar 2008, Nick Piggin wrote: > > Your skeleton is just registering notifiers and saying > > > > /* you fill the hard part in */ > > > > If somebody needs a skeleton in order just to register the notifiers, > > then almost by definition they are unqualified to write the hard > > part ;) > > Its also providing a locking scheme. Not the full locking scheme. If you have a look at the real code required to do it, it is non trivial. > > OK, there are ways to solve it or hack around it. But this is exactly > > why I think the implementations should be kept seperate. Andrea's > > notifiers are coherent, work on all types of mappings, and will > > hopefully match closely the regular TLB invalidation sequence in the > > Linux VM (at the moment it is quite close, but I hope to make it a > > bit closer) so that it requires almost no changes to the mm. > > Then put it into the arch code for TLB invalidation. Paravirt ops gives > good examples on how to do that. Put what into arch code? > > What about a completely different approach... XPmem runs over NUMAlink, > > right? Why not provide some non-sleeping way to basically IPI remote > > nodes over the NUMAlink where they can process the invalidation? If you > > intra-node cache coherency has to run over this link anyway, then > > presumably it is capable. > > There is another Linux instance at the remote end that first has to > remove its own ptes. Yeah, what's the problem? > Also would not work for Inifiniband and other > solutions. infiniband doesn't want it. Other solutions is just handwaving, because if we don't know what the other soloutions are, then we can't make any sort of informed choices. > All the approaches that require evictions in an atomic context > are limiting the approach and do not allow the generic functionality that > we want in order to not add alternate APIs for this. The only generic way to do this that I have seen (and the only proposed way that doesn't add alternate APIs for that matter) is turning VM locks into sleeping locks. In which case, Andrea's notifiers will work just fine (except for relatively minor details like rcu list scanning). So I don't see what you're arguing for. There is no requirement that we support sleeping notifiers in the same patch as non-sleeping ones. Considering the simplicity of the non-sleeping notifiers and the problems with sleeping ones, I think it is pretty clear that they are different beasts (unless VM locking is changed). > > Or another idea, why don't you LD_PRELOAD in the MPT library to also > > intercept munmap, mprotect, mremap etc as well as just fork()? That > > would give you similarly "good enough" coherency as the mmu notifier > > patches except that you can't swap (which Robin said was not a big > > problem). > > The good enough solution right now is to pin pages by elevating > refcounts. Which kind of leads to the question of why do you need any further kernel patches if that is good enough? From andrea at qumranet.com Tue Mar 4 05:21:18 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Tue, 4 Mar 2008 14:21:18 +0100 Subject: [ofa-general] Re: [PATCH] KVM swapping with mmu notifiers #v9 In-Reply-To: <47CC9B57.5050402@qumranet.com> References: <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> Message-ID: <20080304132118.GB5301@v2.random> Hello Izik, On Tue, Mar 04, 2008 at 02:44:07AM +0200, Izik Eidus wrote: > i wrote to you about this before (i didnt get answer for this so i write Ouch I must have lost your previous comment with a too-fast pgdown in the full quoting of the patch sorry. > again) > with large pages support i think we need to use here put_page Right, thanks!! From andrea at qumranet.com Tue Mar 4 05:30:20 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Tue, 4 Mar 2008 14:30:20 +0100 Subject: [ofa-general] Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: References: <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> Message-ID: <20080304133020.GC5301@v2.random> On Mon, Mar 03, 2008 at 11:31:15PM -0800, Christoph Lameter wrote: > @@ -446,6 +450,8 @@ static int page_mkclean_one(struct page > if (address == -EFAULT) > goto out; > > + /* rmap lock held */ > + emm_notify(mm, emm_invalidate_start, address, address + PAGE_SIZE); > pte = page_check_address(page, mm, address, &ptl); > if (!pte) > goto out; > @@ -462,6 +468,7 @@ static int page_mkclean_one(struct page > } > > pte_unmap_unlock(pte, ptl); > + emm_notify(mm, emm_invalidate_end, address, address + PAGE_SIZE); > out: > return ret; > } I could have ripped invalidate_page from my patch too, except I didn't want to slow down those paths for the known-common-users when not even GRU would get any benefit from two hooks when only one is needed. When working with single pages it's more efficient and preferable to call invalidate_page and only later release the VM reference on the page. From a.p.zijlstra at chello.nl Tue Mar 4 02:35:32 2008 From: a.p.zijlstra at chello.nl (Peter Zijlstra) Date: Tue, 04 Mar 2008 11:35:32 +0100 Subject: [ofa-general] ***SPAM*** Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080303191540.GB11156@sgi.com> References: <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> <20080303165910.GA23998@wotan.suse.de> <20080303180605.GA3552@sgi.com> <20080303184517.GA4951@wotan.suse.de> <20080303191540.GB11156@sgi.com> Message-ID: <1204626932.6241.41.camel@lappy> On Mon, 2008-03-03 at 13:15 -0600, Jack Steiner wrote: > I haven't thought about locking requirements for the radix tree. Most accesses > would be read-only & updates infrequent. Any chance of an RCU-based radix > implementation? Otherwise, don't we add the potential for hot locks/cachelines > for threaded applications ??? The current radix tree implementation in the kernel is RCU capable. We just don't have many RCU users yet. From steiner at sgi.com Tue Mar 4 06:44:06 2008 From: steiner at sgi.com (Jack Steiner) Date: Tue, 4 Mar 2008 08:44:06 -0600 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <1204626932.6241.41.camel@lappy> References: <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080303125152.GS8091@v2.random> <20080303131017.GC13138@wotan.suse.de> <20080303151859.GA19374@sgi.com> <20080303165910.GA23998@wotan.suse.de> <20080303180605.GA3552@sgi.com> <20080303184517.GA4951@wotan.suse.de> <20080303191540.GB11156@sgi.com> <1204626932.6241.41.camel@lappy> Message-ID: <20080304144406.GA25467@sgi.com> On Tue, Mar 04, 2008 at 11:35:32AM +0100, Peter Zijlstra wrote: > > On Mon, 2008-03-03 at 13:15 -0600, Jack Steiner wrote: > > > I haven't thought about locking requirements for the radix tree. Most accesses > > would be read-only & updates infrequent. Any chance of an RCU-based radix > > implementation? Otherwise, don't we add the potential for hot locks/cachelines > > for threaded applications ??? > > The current radix tree implementation in the kernel is RCU capable. We > just don't have many RCU users yet. Ahhh. You are right. I thought I looked but obviously missed it. From arne.redlich at xiranet.com Tue Mar 4 07:41:20 2008 From: arne.redlich at xiranet.com (Arne Redlich) Date: Tue, 04 Mar 2008 16:41:20 +0100 Subject: [ofa-general] Re: [PATCH 1/2] IB/iSER: fix list iteration bug In-Reply-To: <47CD3B7A.2010106@voltaire.com> (Erez Zilber's message of "Tue, 04 Mar 2008 14:07:22 +0200") References: <877igkxffl.fsf@confield.dd.xiranet.com> <47CD3B7A.2010106@voltaire.com> Message-ID: <878x0yfqe7.fsf@confield.dd.xiranet.com> Erez Zilber writes: > Arne Redlich wrote: >> The iteration through the list of "iser_device"s during device >> lookup/creation is broken - it might result in an infinite loop if more >> than 1 HCA is used with iSER. Use list_for_each_entry() instead of the >> custom, flawed list iteration code. >> >> Signed-off-by: Arne Redlich >> --- >> drivers/infiniband/ulp/iser/iser_verbs.c | 36 ++++++++++++----------------- >> 1 files changed, 15 insertions(+), 21 deletions(-) >> >> diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c >> index 714b8db..1c0f968 100644 >> --- a/drivers/infiniband/ulp/iser/iser_verbs.c >> +++ b/drivers/infiniband/ulp/iser/iser_verbs.c >> @@ -237,33 +237,27 @@ static int iser_free_ib_conn_res(struct iser_conn *ib_conn) >> static >> struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) >> { >> - struct list_head *p_list; >> - struct iser_device *device = NULL; >> + struct iser_device *device; >> >> mutex_lock(&ig.device_list_mutex); >> >> - p_list = ig.device_list.next; >> - while (p_list != &ig.device_list) { >> - device = list_entry(p_list, struct iser_device, ig_list); >> - /* find if there's a match using the node GUID */ >> + list_for_each_entry(device, &ig.device_list, ig_list) > > I've just added the original comments that are missing in your patch. Ah well, I probably should've mentioned in the patch description that I intentionally removed those comments as I think they're really redundant, stating the obvious. But of course I won't insist. :) Thanks, Arne From swise at opengridcomputing.com Tue Mar 4 08:27:55 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 04 Mar 2008 10:27:55 -0600 Subject: [ofa-general] Re: [RFC PATCH 05/14] RFC RDMA/cxgb3: IDR IDs are signed In-Reply-To: <20082292026.bgCw0jLHUAh50ca1@cisco.com> References: <20082292026.bgCw0jLHUAh50ca1@cisco.com> Message-ID: <47CD788B.1020605@opengridcomputing.com> Acked-by: Steve Wise Roland Dreier wrote: > Fix sparse warnings about pointer signedness by using a signed int when > calling idr_get_new_above(). > > Signed-off-by: Roland Dreier > --- > drivers/infiniband/hw/cxgb3/iwch.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/drivers/infiniband/hw/cxgb3/iwch.h b/drivers/infiniband/hw/cxgb3/iwch.h > index caf4e60..9ad9b1e 100644 > --- a/drivers/infiniband/hw/cxgb3/iwch.h > +++ b/drivers/infiniband/hw/cxgb3/iwch.h > @@ -147,7 +147,7 @@ static inline int insert_handle(struct iwch_dev *rhp, struct idr *idr, > void *handle, u32 id) > { > int ret; > - u32 newid; > + int newid; > > do { > if (!idr_pre_get(idr, GFP_KERNEL)) { From swise at opengridcomputing.com Tue Mar 4 08:29:37 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 04 Mar 2008 10:29:37 -0600 Subject: [ofa-general] Re: [PATCH] iwch_create_cq off by one error In-Reply-To: <47C83798.3090702@opengridcomputing.com> References: <1204158583-22858-1-git-send-email-jon@opengridcomputing.com> <47C83798.3090702@opengridcomputing.com> Message-ID: <47CD78F1.2070309@opengridcomputing.com> Hey Roland, Are you going to pull this one in? Thanks, Steve. Steve Wise wrote: > Acked-by: Steve Wise > > > Jon Mason wrote: >> The cxbg3 driver is unnecessarily decreasing the number of cq entries >> by one when creating a cq. This will cause an error of not having as >> many cqs as requested by the user, if the user requests a power of 2 >> cq length. >> >> Thanks, >> Jon >> >> Signed-off-by: Jon Mason >> --- >> drivers/infiniband/hw/cxgb3/iwch_provider.c | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c >> b/drivers/infiniband/hw/cxgb3/iwch_provider.c >> index f0c7775..800ef6d 100644 >> --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c >> +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c >> @@ -188,7 +188,7 @@ static struct ib_cq *iwch_create_cq(struct >> ib_device *ibdev, int entries, int ve >> return ERR_PTR(-ENOMEM); >> } >> chp->rhp = rhp; >> - chp->ibcq.cqe = (1 << chp->cq.size_log2) - 1; >> + chp->ibcq.cqe = 1 << chp->cq.size_log2; >> spin_lock_init(&chp->lock); >> atomic_set(&chp->refcnt, 1); >> init_waitqueue_head(&chp->wait); > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Tue Mar 4 09:05:35 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 04 Mar 2008 09:05:35 -0800 Subject: [ofa-general] OpenSM building error In-Reply-To: <1204554728.6469.237.camel@hrosenstock-ws.xsigo.com> References: <1204554728.6469.237.camel@hrosenstock-ws.xsigo.com> Message-ID: <1204650335.6469.403.camel@hrosenstock-ws.xsigo.com> On Mon, 2008-03-03 at 06:32 -0800, Hal Rosenstock wrote: > Hi Sasha, > > When I now build OpenSM, I get the following error at link time: > > opensm-osm_console_io.o(.text+0x245): In function `is_authorized': > osm_console_io.c:134: undefined reference to `hosts_ctl' > > I think hosts_ctl comes from libwrap but I don't see that as part of > what the link includes. What am I missing ? I see where it's supposed to be included and defined (it's been there quite a while) but forgot this and what drives setting of HAVE_LIBWRAP in config.h. -- Hal > Thanks. > > -- Hal > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sashak at voltaire.com Tue Mar 4 09:53:25 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 4 Mar 2008 17:53:25 +0000 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <1204553616.6469.229.camel@hrosenstock-ws.xsigo.com> References: <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> <20080302141713.GC30723@sashak.voltaire.com> <1204480309.6469.201.camel@hrosenstock-ws.xsigo.com> <20080302190504.GE30723@sashak.voltaire.com> <1204553616.6469.229.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080304175325.GO11788@sashak.voltaire.com> On 06:13 Mon 03 Mar , Hal Rosenstock wrote: > On Sun, 2008-03-02 at 19:05 +0000, Sasha Khapyorsky wrote: > > On 09:51 Sun 02 Mar , Hal Rosenstock wrote: > > > > > > A different routing dump reflecting balance or not and how out of > > > balance. > > > > This makes sense. Actually OpenSM has such sort of dump right now, but > > it is printed to stdout. > > Yes, there are various dumps (to stdout or opensm.fdbs) which contain > routing information but do they contain balance info ? Yes, dump_ucast_path_distribution() prints balance info to stdout. Sasha From ardavis at ichips.intel.com Tue Mar 4 09:39:52 2008 From: ardavis at ichips.intel.com (Arlin Davis) Date: Tue, 04 Mar 2008 09:39:52 -0800 Subject: [ofa-general] UDAPL with OFED1.2 Howto In-Reply-To: <47CCA95C.2070102@penguincomputing.com> References: <47CCA95C.2070102@penguincomputing.com> Message-ID: <47CD8968.3070801@ichips.intel.com> Joshua Bernstein wrote: > Hi Everyone, > > I caught this post some time ago: > > http://www.mail-archive.com/openib-general at openib.org/msg07877.html > > In this post, it was asked if there was a UDAPL OFED HOWTO guide. Since > this post was quite old, and nothing more recent seemed to come up in my > searches, I was wondering if anybody had written any more of a howto > other then whats covered in a OFED Wiki. Any pointers? > Can you be more specific... How to setup/test uDAPL with OFED? How to write a DAPL application? How to implement a new provider? You can start by looking in the OFA downloads directory: http://www.openfabrics.org/downloads/dapl/ under documentation there are several documents including a new uDAPL OFED setup/test bkm that may answer your questions. Let me know if you cannot find the information you need and I will attempt to update accordingly. -arlin From okir at lst.de Tue Mar 4 09:44:36 2008 From: okir at lst.de (Olaf Kirch) Date: Tue, 4 Mar 2008 18:44:36 +0100 Subject: [ofa-general] Some RDS performance data Message-ID: <200803041844.37260.okir@lst.de> For those interested, I uploaded some charts on RDSv3 performance today. A comparison of bcopy mode and the RDMA support introduced in OFED 1.3 can be found at http://oss.oracle.com/~okir/rds/2008-Feb-29/rdma Some charts looking into scalability are at http://oss.oracle.com/~okir/rds/2008-Feb-29/scalability Enjoy, Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play olaf.kirch at oracle.com | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax From ralph.campbell at qlogic.com Tue Mar 4 10:00:10 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 04 Mar 2008 10:00:10 -0800 Subject: [ofa-general] Does QLogic have their own HCA? In-Reply-To: References: Message-ID: <1204653610.5109.557.camel@brick.pathscale.com> See the following for the list of IB HCA products. http://www.qlogic.com/Products/HPC_products_infipathhcas.aspx QLogic sells its own HCA as well as reselling Mellanox HCAs. The main advantage of the QLE7280 is that it has a 16x PCIe gen1 bus so that the full 4xDDR rate is not limited by the PCIe bus. On Mon, 2008-03-03 at 20:24 -0600, Tim Hadeen wrote: > Hello, > I know that we have ipath driver which is for QLogic IB part. Can > somebody let us know more information on QLogic's IB HCA (Chip) > offerings? > > Thanks > Tim > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sashak at voltaire.com Tue Mar 4 10:34:29 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 4 Mar 2008 18:34:29 +0000 Subject: [ofa-general] OpenSM building error In-Reply-To: <1204650335.6469.403.camel@hrosenstock-ws.xsigo.com> References: <1204554728.6469.237.camel@hrosenstock-ws.xsigo.com> <1204650335.6469.403.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080304183429.GP11788@sashak.voltaire.com> On 09:05 Tue 04 Mar , Hal Rosenstock wrote: > On Mon, 2008-03-03 at 06:32 -0800, Hal Rosenstock wrote: > > Hi Sasha, > > > > When I now build OpenSM, I get the following error at link time: > > > > opensm-osm_console_io.o(.text+0x245): In function `is_authorized': > > osm_console_io.c:134: undefined reference to `hosts_ctl' > > > > I think hosts_ctl comes from libwrap but I don't see that as part of > > what the link includes. What am I missing ? > > I see where it's supposed to be included and defined (it's been there > quite a while) but forgot this and what drives setting of HAVE_LIBWRAP > in config.h. There is AC_CHECK_LIB(wrap, request_init, [] in config/osmvsel.m4, it is turned on as part of console selector. Sasha From hrosenstock at xsigo.com Tue Mar 4 10:22:11 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 04 Mar 2008 10:22:11 -0800 Subject: [ofa-general] OFED 1.3 OpenSM and QoS Annex Support Message-ID: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> Hi, As I recall, the QoS manager (at OFED 1.3) is experimental. If so, in the release notes, QoS manager should be indicated as experimental in the release notes. Additionally, there should be some statement of what HCAs and firmware versions this works with (and any switch firmware dependencies if any). It also brings on new tools requirements for building (versions of yacc and lex) which should also be indicated there (and perhaps checked somehow in the build process). Can this get added/updated to the release notes ? Also, it would also be nice if there were a way to not include this in the OpenSM build if not wanted (an --enable-qos-annex configure option or the like). Is that a straightforward change ? Thanks. -- Hal From sashak at voltaire.com Tue Mar 4 10:39:12 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 4 Mar 2008 18:39:12 +0000 Subject: [ofa-general] Re: [PATCH] OpenSM: Set packet life time to subnet timeout option rather than default In-Reply-To: <1204553517.6469.225.camel@hrosenstock-ws.xsigo.com> References: <1204553517.6469.225.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080304183912.GQ11788@sashak.voltaire.com> On 06:11 Mon 03 Mar , Hal Rosenstock wrote: > OpenSM: Set packet life time to subnet timeout option rather than > default > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Tue Mar 4 10:53:52 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 4 Mar 2008 18:53:52 +0000 Subject: [ofa-general] Re: [PATCH] Fix bug which prevented some GUIDs from being found due to formating issues. In-Reply-To: <20080303105833.35c8d01c.weiny2@llnl.gov> References: <20080229112509.64f1bbb4.weiny2@llnl.gov> <20080301020441.GI1485@sashak.voltaire.com> <20080303105833.35c8d01c.weiny2@llnl.gov> Message-ID: <20080304185352.GR11788@sashak.voltaire.com> On 10:58 Mon 03 Mar , Ira Weiny wrote: > From 8bc9c94ae20fffcd4eff5d557ce6efb4ef1f7d38 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 28 Feb 2008 15:03:23 -0800 > Subject: [PATCH] Fix bug which prevented some GUIDs from being found due to formating issues. > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From chu11 at llnl.gov Tue Mar 4 10:44:36 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 04 Mar 2008 10:44:36 -0800 Subject: [ofa-general] Re: [OpenSM] updn routing performance fix??? In-Reply-To: <20080304175325.GO11788@sashak.voltaire.com> References: <20080229172135.GD1485@sashak.voltaire.com> <1204308601.6469.74.camel@hrosenstock-ws.xsigo.com> <20080301014716.GG1485@sashak.voltaire.com> <1204343987.6469.140.camel@hrosenstock-ws.xsigo.com> <20080301225347.GE11788@sashak.voltaire.com> <1204463050.6469.198.camel@hrosenstock-ws.xsigo.com> <20080302141713.GC30723@sashak.voltaire.com> <1204480309.6469.201.camel@hrosenstock-ws.xsigo.com> <20080302190504.GE30723@sashak.voltaire.com> <1204553616.6469.229.camel@hrosenstock-ws.xsigo.com> <20080304175325.GO11788@sashak.voltaire.com> Message-ID: <1204656276.761.89.camel@cardanus.llnl.gov> On Tue, 2008-03-04 at 17:53 +0000, Sasha Khapyorsky wrote: > On 06:13 Mon 03 Mar , Hal Rosenstock wrote: > > On Sun, 2008-03-02 at 19:05 +0000, Sasha Khapyorsky wrote: > > > On 09:51 Sun 02 Mar , Hal Rosenstock wrote: > > > > > > > > A different routing dump reflecting balance or not and how out of > > > > balance. > > > > > > This makes sense. Actually OpenSM has such sort of dump right now, but > > > it is printed to stdout. > > > > Yes, there are various dumps (to stdout or opensm.fdbs) which contain > > routing information but do they contain balance info ? > > Yes, dump_ucast_path_distribution() prints balance info to stdout. Although it outputs the number of paths going through switch ports, it doesn't seem to output balance information. You'd have to visually look at all of the dump output to figure it out. All my script does is condense the output to output only unbalanced info and handle "corner cases" in the output (i.e. don't count switch ports directly connected to channel adapters in the "balance" calculation). This function provides a reasonable starting point. A little bit of editing and it should provide what we need for a console command. The console command would be way better/faster than my script. Al > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From sashak at voltaire.com Tue Mar 4 10:59:27 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 4 Mar 2008 18:59:27 +0000 Subject: [ofa-general] Re: [PATCH] infiniband-diags/scripts/ib[linkinfo][queryerrors].pl: report switch not found In-Reply-To: <20080303105923.7ba71092.weiny2@llnl.gov> References: <20080303105923.7ba71092.weiny2@llnl.gov> Message-ID: <20080304185927.GS11788@sashak.voltaire.com> On 10:59 Mon 03 Mar , Ira Weiny wrote: > From 2580633c3842aac4257c6b6a330444ff7fe4a6bf Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Mon, 3 Mar 2008 10:44:56 -0800 > Subject: [PATCH] infiniband-diags/scripts/ib[linkinfo][queryerrors].pl: report switch not found > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Tue Mar 4 11:00:46 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 4 Mar 2008 19:00:46 +0000 Subject: [ofa-general] Re: [PATCH] Update documentation for guid format In-Reply-To: <20080303110206.0d9f8e0e.weiny2@llnl.gov> References: <20080303110206.0d9f8e0e.weiny2@llnl.gov> Message-ID: <20080304190046.GT11788@sashak.voltaire.com> On 11:02 Mon 03 Mar , Ira Weiny wrote: > From b7581a29d6ad58de01c2dc6c2bce420f300210ab Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Mon, 3 Mar 2008 11:01:00 -0800 > Subject: [PATCH] Update documentation for guid format > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From clameter at sgi.com Tue Mar 4 10:58:41 2008 From: clameter at sgi.com (Christoph Lameter) Date: Tue, 4 Mar 2008 10:58:41 -0800 (PST) Subject: [ofa-general] Re: [patch 2/6] mmu_notifier: Callbacks to invalidate address ranges In-Reply-To: <200803040650.11942.nickpiggin@yahoo.com.au> References: <20080215064859.384203497@sgi.com> <200803031611.10275.nickpiggin@yahoo.com.au> <200803040650.11942.nickpiggin@yahoo.com.au> Message-ID: On Tue, 4 Mar 2008, Nick Piggin wrote: > > Then put it into the arch code for TLB invalidation. Paravirt ops gives > > good examples on how to do that. > > Put what into arch code? The mmu notifier code. > > > What about a completely different approach... XPmem runs over NUMAlink, > > > right? Why not provide some non-sleeping way to basically IPI remote > > > nodes over the NUMAlink where they can process the invalidation? If you > > > intra-node cache coherency has to run over this link anyway, then > > > presumably it is capable. > > > > There is another Linux instance at the remote end that first has to > > remove its own ptes. > > Yeah, what's the problem? The remote end has to invalidate the page which involves locking etc. > > Also would not work for Inifiniband and other > > solutions. > > infiniband doesn't want it. Other solutions is just handwaving, > because if we don't know what the other soloutions are, then we can't > make any sort of informed choices. We need a solution in general to avoid the pinning problems. Infiniband has those too. > > All the approaches that require evictions in an atomic context > > are limiting the approach and do not allow the generic functionality that > > we want in order to not add alternate APIs for this. > > The only generic way to do this that I have seen (and the only proposed > way that doesn't add alternate APIs for that matter) is turning VM locks > into sleeping locks. In which case, Andrea's notifiers will work just > fine (except for relatively minor details like rcu list scanning). No they wont. As you pointed out the callback need RCU locking. > > The good enough solution right now is to pin pages by elevating > > refcounts. > > Which kind of leads to the question of why do you need any further > kernel patches if that is good enough? Well its good enough with severe problems during reclaim, livelocks etc. One could improve on that scheme through Rik's work trying to add a new page flag that mark pinned pages and then keep them off the LRUs and limiting their number. Having pinned page would limit the ability to reclaim by the VM and make page migration, memory unplug etc impossible. It is better to have notifier scheme that allows to tell a device driver to free up the memory it has mapped. From clameter at sgi.com Tue Mar 4 11:00:31 2008 From: clameter at sgi.com (Christoph Lameter) Date: Tue, 4 Mar 2008 11:00:31 -0800 (PST) Subject: [ofa-general] Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <20080304133020.GC5301@v2.random> References: <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> Message-ID: On Tue, 4 Mar 2008, Andrea Arcangeli wrote: > When working with single pages it's more efficient and preferable to > call invalidate_page and only later release the VM reference on the > page. But as you pointed out before that path is a slow path anyways. Its rarely taken. Having a single eviction callback simplifies design. Plus the device driver can still check if the mapping was of PAGE_SIZE and then implement its own optimization. From chu11 at llnl.gov Tue Mar 4 11:01:48 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 04 Mar 2008 11:01:48 -0800 Subject: [ofa-general] [PATCH] opensm: multi lid routing balancing for updn/minhop In-Reply-To: <1204586117.761.79.camel@cardanus.llnl.gov> References: <1204585724.761.77.camel@cardanus.llnl.gov> <1204586117.761.79.camel@cardanus.llnl.gov> Message-ID: <1204657308.761.94.camel@cardanus.llnl.gov> Fixed up one more typo .. Al On Mon, 2008-03-03 at 15:15 -0800, Al Chu wrote: > Hey Sasha, > > I just noticed a comment typo. Here's a fixed patch :P > > Al > > On Mon, 2008-03-03 at 15:08 -0800, Al Chu wrote: > > Hey Sasha, > > > > I was originally going to submit this later on, but given the recent > > "rebalancing switch connections" threads, I figure perhaps was now a > > good time to post this patch. > > > > When we turn on lmc > 0, we noticed that sometimes extra lids from a > > port would be forwarded through one parent switch than another. For > > example, suppose LMC = 2 and we are trying to route lids (1,2,3,4). The > > lids can be forwarded out of 8 ports, which go to two different > > switches. We would see something like this: > > > > switch port 1 (to switch A): 1 > > switch port 2 (to switch A): 3 > > switch port 3 (to switch A): 4 > > switch port 4 (to switch A): > > switch port 5 (to switch B): 2 > > switch port 6 (to switch B) > > switch port 7 (to switch B): > > switch port 8 (to switch B): > > > > This occurs because the routing for LMC only favors those sys_guids and > > node_guids that have not been seen before. But it does not consider how > > many times we have routed through a sys_guid/node_guid before. > > > > The patch is fairly straight forward. We just count how many times we > > have forwarded to a sys_guid/node_guid before. If there is a port that > > has an equal number of paths to another port, but has not been forwarded > > out as much, we pick that port. Most of the patch is architectural > > changes. I stuff the sys_guid, node_guid, and a counter inside one > > struct and array, because we can't count properly using the multiple > > uint64_t arrays from before. > > > > Thanks, > > Al > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-support-balanced-multi-lid-routing.patch Type: text/x-patch Size: 15261 bytes Desc: not available URL: From roland.list at gmail.com Tue Mar 4 11:21:18 2008 From: roland.list at gmail.com (Roland Dreier) Date: Tue, 4 Mar 2008 11:21:18 -0800 Subject: [ofa-general] Re: [PATCH] iwch_create_cq off by one error In-Reply-To: <47CD78F1.2070309@opengridcomputing.com> References: <1204158583-22858-1-git-send-email-jon@opengridcomputing.com> <47C83798.3090702@opengridcomputing.com> <47CD78F1.2070309@opengridcomputing.com> Message-ID: > Are you going to pull this one in? Yeah... I'm on vacation for the week but I'll get it next week. From hrosenstock at xsigo.com Tue Mar 4 11:34:13 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 04 Mar 2008 11:34:13 -0800 Subject: [ofa-general] OFED 1.3 OpenSM and QoS Annex Building Message-ID: <1204659253.6469.441.camel@hrosenstock-ws.xsigo.com> Hi, When I configure OFED 1.3 OpenSM, I get: checking for bison... no checking for byacc... byacc checking for flex... flex checking for yywrap in -lfl... yes checking lex output file root... lex.yy checking whether yytext is a pointer... yes When I make, I get the following error: byacc -d -o ./osm_qos_parser_y.c -p__qos_parser_ ./osm_qos_parser.y usage: byacc [-dlrtv] [-b file_prefix] [-p symbol_prefix] filename make[2]: *** [osm_qos_parser_y.c] Error 1 Apparently, my version does not support -o and it is preferred not to be change yacc if possible. So I tried to change opensm/Makefile.am as follows: diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 50fdae1..f85a6d6 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -63,8 +63,11 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_ - $(YACC) -d -o $(srcdir)/osm_qos_parser_y.c -p__qos_parser_ $(srcdir)/osm - mv -f $(srcdir)/osm_qos_parser_y.h $(srcdir)/../include/opensm/osm_qos_p + $(YACC) -d -b osm_qos_parser -p__qos_parser_ $(srcdir)/osm_qos_parser.y + mv -f $(srcdir)/osm_qos_parser.tab.c $(srcdir)/osm_qos_parser_y.c + mv -f $(srcdir)/osm_qos_parser.tab.h $(srcdir)/../include/opensm/osm_qos +# $(YACC) -d -o $(srcdir)/osm_qos_parser_y.c -p__qos_parser_ $(srcdir)/osm +# mv -f $(srcdir)/osm_qos_parser_y.h $(srcdir)/../include/opensm/osm_qos_p osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_ but get: if gcc -DHAVE_CONFIG_H -I. -I. -I../include -I./../include -I./../../libibcommon/include -I./../../libibumad/include -I/opt/ofed/include -Wall -DOSM_VENDOR_INTF_OPENIB -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP -g -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 -I/opt/ofed/include -MT opensm-osm_qos_parser_y.o -MD -MP -MF ".deps/opensm-osm_qos_parser_y.Tpo" \ -c -o opensm-osm_qos_parser_y.o `test -f 'osm_qos_parser_y.c' || echo './'`osm_qos_parser_y.c; \ then mv -f ".deps/opensm-osm_qos_parser_y.Tpo" ".deps/opensm-osm_qos_parser_y.Po"; \ else rm -f ".deps/opensm-osm_qos_parser_y.Tpo"; exit 1; \ fi ./osm_qos_parser.y: In function `osm_qos_parse_policy_file': ./osm_qos_parser.y:2324: warning: implicit declaration of function `__qos_parser_parse' osm_qos_parser.tab.c: In function `__qos_parser_parse': osm_qos_parser.tab.c:1722: warning: suggest parentheses around assignment used as truth value osm_qos_parser.tab.c:1770: warning: label `yyerrlab' defined but not used osm_qos_parser.tab.c:1765: warning: label `yynewerror' defined but not used ../include/complib/cl_qcomppool.h: At top level: osm_qos_parser_y.c:2: warning: `yysccsid' defined but not used flex -P__qos_parser_ -o./osm_qos_parser_l.c ./osm_qos_parser.l if gcc -DHAVE_CONFIG_H -I. -I. -I../include -I./../include -I./../../libibcommon/include -I./../../libibumad/include -I/opt/ofed/include -Wall -DOSM_VENDOR_INTF_OPENIB -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP -g -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 -I/opt/ofed/include -MT opensm-osm_qos_parser_l.o -MD -MP -MF ".deps/opensm-osm_qos_parser_l.Tpo" \ -c -o opensm-osm_qos_parser_l.o `test -f 'osm_qos_parser_l.c' || echo './'`osm_qos_parser_l.c; \ then mv -f ".deps/opensm-osm_qos_parser_l.Tpo" ".deps/opensm-osm_qos_parser_l.Po"; \ else rm -f ".deps/opensm-osm_qos_parser_l.Tpo"; exit 1; \ fi ./osm_qos_parser.l: In function `__qos_parser_lex': ./osm_qos_parser.l:205: `__qos_parser_lval' undeclared (first use in this function) ./osm_qos_parser.l:205: (Each undeclared identifier is reported only once ./osm_qos_parser.l:205: for each function it appears in.) make[2]: *** [opensm-osm_qos_parser_l.o] Error 1 Is there a way around this without updating yacc ? -- Hal From meier3 at llnl.gov Tue Mar 4 11:39:29 2008 From: meier3 at llnl.gov (Timothy A. Meier) Date: Tue, 04 Mar 2008 11:39:29 -0800 Subject: [ofa-general] OpenSM building error In-Reply-To: <1204650335.6469.403.camel@hrosenstock-ws.xsigo.com> References: <1204554728.6469.237.camel@hrosenstock-ws.xsigo.com> <1204650335.6469.403.camel@hrosenstock-ws.xsigo.com> Message-ID: <47CDA571.6020105@llnl.gov> Hi Hal, I added that new module (containing existing code tho), so looked at it to see if I messed up some conditional compile option. I tried a variety of configurations, and can't seem to reproduce the build error. When you compile with remote console, libwrap is required (the tcp wrapper stuff). We have the latest version (0.7.6), which is located in /usr/lib64 for our environment. This is not a new requirement/dependency for opensm, but it is for osm_console_io, since that module didn't exist before. Fresh rebuild (autogen, configure....)????? Tim Hal Rosenstock wrote: > On Mon, 2008-03-03 at 06:32 -0800, Hal Rosenstock wrote: >> Hi Sasha, >> >> When I now build OpenSM, I get the following error at link time: >> >> opensm-osm_console_io.o(.text+0x245): In function `is_authorized': >> osm_console_io.c:134: undefined reference to `hosts_ctl' >> >> I think hosts_ctl comes from libwrap but I don't see that as part of >> what the link includes. What am I missing ? > > I see where it's supposed to be included and defined (it's been there > quite a while) but forgot this and what drives setting of HAVE_LIBWRAP > in config.h. > > -- Hal > >> Thanks. >> >> -- Hal >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > -- Timothy A. Meier Computer Scientist ICCD/High Performance Computing 925.422.3341 meier3 at llnl.gov From hrosenstock at xsigo.com Tue Mar 4 11:45:13 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 04 Mar 2008 11:45:13 -0800 Subject: [ofa-general] OpenSM building error In-Reply-To: <47CDA571.6020105@llnl.gov> References: <1204554728.6469.237.camel@hrosenstock-ws.xsigo.com> <1204650335.6469.403.camel@hrosenstock-ws.xsigo.com> <47CDA571.6020105@llnl.gov> Message-ID: <1204659913.6469.449.camel@hrosenstock-ws.xsigo.com> Hi Tim, On Tue, 2008-03-04 at 11:39 -0800, Timothy A. Meier wrote: > Hi Hal, > > I added that new module (containing existing code tho), so looked at > it to see if I messed up some conditional compile option. > > I tried a variety of configurations, and can't seem to reproduce the > build error. > > When you compile with remote console, libwrap is required (the tcp wrapper stuff). > We have the latest version (0.7.6), which is located in /usr/lib64 for our > environment. > > This is not a new requirement/dependency for opensm, Right; this has been the case for quite a while now. > but it is for osm_console_io, since that module didn't exist before. What is new (aside from this module) is the need for hosts_ctl. I wonder if that is a libwrap version issue. -- Hal > Fresh rebuild (autogen, configure....)????? > > Tim > > Hal Rosenstock wrote: > > On Mon, 2008-03-03 at 06:32 -0800, Hal Rosenstock wrote: > >> Hi Sasha, > >> > >> When I now build OpenSM, I get the following error at link time: > >> > >> opensm-osm_console_io.o(.text+0x245): In function `is_authorized': > >> osm_console_io.c:134: undefined reference to `hosts_ctl' > >> > >> I think hosts_ctl comes from libwrap but I don't see that as part of > >> what the link includes. What am I missing ? > > > > I see where it's supposed to be included and defined (it's been there > > quite a while) but forgot this and what drives setting of HAVE_LIBWRAP > > in config.h. > > > > -- Hal > > > >> Thanks. > >> > >> -- Hal > >> _______________________________________________ > >> general mailing list > >> general at lists.openfabrics.org > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >> > >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > From meier3 at llnl.gov Tue Mar 4 12:03:47 2008 From: meier3 at llnl.gov (Timothy A. Meier) Date: Tue, 04 Mar 2008 12:03:47 -0800 Subject: [ofa-general] OpenSM building error In-Reply-To: <1204659913.6469.449.camel@hrosenstock-ws.xsigo.com> References: <1204554728.6469.237.camel@hrosenstock-ws.xsigo.com> <1204650335.6469.403.camel@hrosenstock-ws.xsigo.com> <47CDA571.6020105@llnl.gov> <1204659913.6469.449.camel@hrosenstock-ws.xsigo.com> Message-ID: <47CDAB23.1050003@llnl.gov> Hi Hal, The patch that created the new osm_console_io module didn't contain any new code. It was just existing code moved from osm_console. Hal Rosenstock wrote: > Hi Tim, > > On Tue, 2008-03-04 at 11:39 -0800, Timothy A. Meier wrote: >> Hi Hal, >> >> I added that new module (containing existing code tho), so looked at >> it to see if I messed up some conditional compile option. >> >> I tried a variety of configurations, and can't seem to reproduce the >> build error. >> >> When you compile with remote console, libwrap is required (the tcp wrapper stuff). >> We have the latest version (0.7.6), which is located in /usr/lib64 for our >> environment. >> >> This is not a new requirement/dependency for opensm, > > Right; this has been the case for quite a while now. > >> but it is for osm_console_io, since that module didn't exist before. > > What is new (aside from this module) is the need for hosts_ctl. I wonder > if that is a libwrap version issue. "hosts_ctl()" has been in use since at least opensm3.0.3 (I can't go back very far). It was in the "connection_ok()" method, which I moved and renamed to "is_authorized()". Tim > > -- Hal > >> Fresh rebuild (autogen, configure....)????? >> >> Tim >> -- Timothy A. Meier Computer Scientist ICCD/High Performance Computing 925.422.3341 meier3 at llnl.gov From sashak at voltaire.com Tue Mar 4 12:20:36 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 4 Mar 2008 20:20:36 +0000 Subject: [ofa-general] [infiniband-diags] check_lft_balance script In-Reply-To: <57972.128.15.244.53.1204484021.squirrel@127.0.0.1> References: <52160.128.15.244.18.1204474250.squirrel@127.0.0.1> <57972.128.15.244.53.1204484021.squirrel@127.0.0.1> Message-ID: <20080304202036.GU11788@sashak.voltaire.com> On 10:53 Sun 02 Mar , Albert Chu wrote: > From 656ba8ee8103fd27f3041570d8c44d823848abf4 Mon Sep 17 00:00:00 2001 > From: Albert L. Chu > Date: Sat, 1 Mar 2008 20:02:03 -0800 > Subject: [PATCH] add check_lft_balance script > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From hrosenstock at xsigo.com Tue Mar 4 12:08:40 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 04 Mar 2008 12:08:40 -0800 Subject: [ofa-general] OpenSM building error In-Reply-To: <47CDAB23.1050003@llnl.gov> References: <1204554728.6469.237.camel@hrosenstock-ws.xsigo.com> <1204650335.6469.403.camel@hrosenstock-ws.xsigo.com> <47CDA571.6020105@llnl.gov> <1204659913.6469.449.camel@hrosenstock-ws.xsigo.com> <47CDAB23.1050003@llnl.gov> Message-ID: <1204661320.6469.464.camel@hrosenstock-ws.xsigo.com> Hi again Tim, On Tue, 2008-03-04 at 12:03 -0800, Timothy A. Meier wrote: > Hi Hal, > > The patch that created the new osm_console_io module didn't contain > any new code. It was just existing code moved from osm_console. Thanks; it's a new requirement v. OFED 1.2 and I'm just in the process of trying various newer management things out. -- Hal > > Hal Rosenstock wrote: > > Hi Tim, > > > > On Tue, 2008-03-04 at 11:39 -0800, Timothy A. Meier wrote: > >> Hi Hal, > >> > >> I added that new module (containing existing code tho), so looked at > >> it to see if I messed up some conditional compile option. > >> > >> I tried a variety of configurations, and can't seem to reproduce the > >> build error. > >> > >> When you compile with remote console, libwrap is required (the tcp wrapper stuff). > >> We have the latest version (0.7.6), which is located in /usr/lib64 for our > >> environment. > >> > >> This is not a new requirement/dependency for opensm, > > > > Right; this has been the case for quite a while now. > > > >> but it is for osm_console_io, since that module didn't exist before. > > > > What is new (aside from this module) is the need for hosts_ctl. I wonder > > if that is a libwrap version issue. > > "hosts_ctl()" has been in use since at least opensm3.0.3 (I can't go back very far). > It was in the "connection_ok()" method, which I moved and renamed to "is_authorized()". > > Tim > > > > -- Hal > > > >> Fresh rebuild (autogen, configure....)????? > >> > >> Tim > >> > From swise at opengridcomputing.com Tue Mar 4 13:25:21 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 04 Mar 2008 15:25:21 -0600 Subject: [ofa-general] detecting "recv buffer not available" events Message-ID: <47CDBE41.1060600@opengridcomputing.com> All, This question is regarding SENDs arriving when no RECV buffer is posted. For IB, somehow the transport can deal with this and force the sender transport to retransmit. First, is my statement above correct? If so, How can I detect that this is happening on an IB RC? IE is there some stat I can query? Thanks, Steve. From chu11 at llnl.gov Tue Mar 4 13:34:04 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 04 Mar 2008 13:34:04 -0800 Subject: [ofa-general] [OpenSM]: renaming of opensm init script to opensmd Message-ID: <1204666444.761.101.camel@cardanus.llnl.gov> Hey Sasha, One of the sys-admins was wondering why the script name was changed from opensm to opensmd (which requires him to go change some internal scripts he uses). Our local Redhat guy told me in the past rhel/fedora convention is the following (which is a comment stolen from rpmlint). 'incoherent-init-script-name', '''The init script name should be the same as the package name in lower case, or one with 'd' appended if it invokes a process by that name.''', Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From CFGGZIELABGG at yahoo.com Tue Mar 4 13:49:01 2008 From: CFGGZIELABGG at yahoo.com (Mervin Hutton) Date: Tue, 04 Mar 2008 22:49:01 +0100 Subject: [ofa-general] It's Here Now Message-ID: <68RS87FE.0H24.CFGGZIELABGG@yahoo.com> Sorry to drop in on you, but you were referred to us by a friend / working associate. As of January 2006 our University has started a work experience degree program. We can offer you 3 of the following choices: - Associate Degree - Bachelor's Degree - Master's Degree Our work experience / life experience degrees are the same degrees we give our full time students, but we base them upon your past knowledge and therefore require no studying. Due to back logs we will need 1-2 weeks to verify your information and send your degree with transcripts in the mail. Our Education office has someone available 24 hours a day, 7 Days a week. If you are still interested then call us at: 1-309-401-0721 From CFGGZIELABGG at yahoo.com Tue Mar 4 13:49:01 2008 From: CFGGZIELABGG at yahoo.com (Mervin Hutton) Date: Tue, 04 Mar 2008 22:49:01 +0100 Subject: [ofa-general] It's Here Now Message-ID: <68RS87FE.0H24.CFGGZIELABGG@yahoo.com> Sorry to drop in on you, but you were referred to us by a friend / working associate. As of January 2006 our University has started a work experience degree program. We can offer you 3 of the following choices: - Associate Degree - Bachelor's Degree - Master's Degree Our work experience / life experience degrees are the same degrees we give our full time students, but we base them upon your past knowledge and therefore require no studying. Due to back logs we will need 1-2 weeks to verify your information and send your degree with transcripts in the mail. Our Education office has someone available 24 hours a day, 7 Days a week. If you are still interested then call us at: 1-309-401-0721 From p2s2-chairs at mcs.anl.gov Tue Mar 4 13:03:37 2008 From: p2s2-chairs at mcs.anl.gov (p2s2-chairs at mcs.anl.gov) Date: Tue, 4 Mar 2008 15:03:37 -0600 (CST) Subject: [ofa-general] [p2s2-announce] CFP: Workshop on Parallel Programming Models and Systems Software for High-end Computing (P2S2) Message-ID: <20080304210337.BAC4246218@shakey.mcs.anl.gov> CALL FOR PAPERS =============== First International Workshop on Parallel Programming Models and Systems Software for High-end Computing (P2S2) (http://www.mcs.anl.gov/events/workshops/p2s2) Sep. 8th, 2008 To be held in conjunction with ICPP-08: The 27th International Conference on Parallel Processing Sep. 8-12, 2008 Portland, Oregon, USA SCOPE ----- The goal of this workshop is to bring together researchers and practitioners in parallel programming models and systems software for high-end computing systems. Please join us in a discussion of new ideas, experiences, and the latest trends in these areas at the workshop. TOPICS OF INTEREST ------------------ The focus areas for this workshop include, but are not limited to: * Programming models and their high-performance implementations o MPI, Sockets, OpenMP, Global Arrays, X10, UPC, Chapel o Other Hybrid Programming Models * Systems software for scientific and enterprise computing o Communication sub-subsystems for high-end computing o High-performance File and storage systems o Fault-tolerance techniques and implementations o Efficient and high-performance virtualization and other management mechanisms * Tools for Management, Maintenance, Coordination and Synchronization o Software for Enterprise Data-centers using Modern Architectures o Job scheduling libraries o Management libraries for large-scale system o Toolkits for process and task coordination on modern platforms * Performance evaluation, analysis and modeling of emerging computing platforms SUBMISSION INSTRUCTIONS ----------------------- Submissions should be in PDF format in U.S. Letter size paper. They should not exceed 8 pages (all inclusive). Submissions will be judged based on relevance, significance, originality, correctness and clarity. DATES AND DEADLINES ------------------- Paper Submission: April 11th, 2008 Author Notification: May 20th, 2008 Camera Ready: June 2nd, 2008 PROGRAM CHAIRS -------------- * Pavan Balaji (Argonne National Laboratory) * Sayantan Sur (IBM Research) STEERING COMMITTEE ------------------ * William D. Gropp (University of Illinois Urbana-Champaign) * Dhabaleswar K. Panda (Ohio State University) * Vijay Saraswat (IBM Research) PROGRAM COMMITTEE ----------------- * David Bernholdt (Oak Ridge National Laboratory) * Ron Brightwell (Sandia National Laboratory) * Wu-chun Feng (Virginia Tech) * Richard Graham (Oak Ridge National Laboratory) * Hyun-wook Jin (Konkuk University, South Korea) * Sameer Kumar (IBM Research) * Doug Lea (State University of New York at Oswego) * Jarek Nieplocha (Pacific Northwest National Laboratory) * Scott Pakin (Los Alamos National Laboratory) * Vivek Sarkar (Rice University) * Rajeev Thakur (Argonne National Laboratory) * Pete Wyckoff (Ohio Supercomputing Center) If you have any questions, please contact us at p2s2-chairs at mcs.anl.gov ------------------------------------------------------------------------- If you do not want to receive any more announcements about the P2S2 workshop, please send an email to majordomo at mcs.anl.gov with the email body "unsubscribe p2s2-announce". ------------------------------------------------------------------------- From andrea at qumranet.com Tue Mar 4 14:20:30 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Tue, 4 Mar 2008 23:20:30 +0100 Subject: [ofa-general] Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: References: <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> Message-ID: <20080304222030.GB8951@v2.random> On Tue, Mar 04, 2008 at 11:00:31AM -0800, Christoph Lameter wrote: > But as you pointed out before that path is a slow path anyways. Its rarely It's a slow path but I don't see why you think two hooks are better than one, when only one is necessary. I once ripped invalidate_page while working on #v8 but then I reintroduced it because I thought reducing the total number of hooks was beneficial to the core linux VM (even if only a microoptimization, I sure agree about that, but it's trivial to add one hook instead of two hooks there, so a microoptimization was worth it IMHO). Your API is also too restrictive, if we'll happen to need one more method that doesn't take just (start,end) we'll have to cause all drivers to have significant changes instead of one-liners to use whatever new feature. > taken. Having a single eviction callback simplifies design. IMHO the design is actually same and I don't understand why you rewrote it once more time in a less flexibile way (on a style side you're not even using hlist), dropping RCU (not sure how you replace it with), etc.... Your implementation has the same bug you had in your first V1, see how you're not clearing the spte young bits if the pte young bit is set. Once you fix that, your change in the ptep_clear_flush_young path will look remarkably similar to the patch I posted incremental with #v8 to make ->clear_flush_young sleep capable... Converging in a single design is great, but it'd be nice if we could converge into a single implementation, and my last patch doesn't have any bug and I think it's quite nicer too (also including Nick cleanup work) but then I may be biased ;). But as usual I'm entirely satisfied by your brand new EMM Notifier to be merged and all perfecting work done on my MMU notifier patch over the weeks by multiple developers (including you) to be dropped for good, as long as we can enable the new advanced KVM features in 2.6.25. From jlentini at netapp.com Tue Mar 4 14:21:23 2008 From: jlentini at netapp.com (James Lentini) Date: Tue, 4 Mar 2008 17:21:23 -0500 (EST) Subject: [ofa-general] detecting "recv buffer not available" events In-Reply-To: <47CDBE41.1060600@opengridcomputing.com> References: <47CDBE41.1060600@opengridcomputing.com> Message-ID: On Tue, 4 Mar 2008, Steve Wise wrote: > All, > > This question is regarding SENDs arriving when no RECV buffer is > posted. For IB, somehow the transport can deal with this and force > the sender transport to retransmit. > > First, is my statement above correct? This is correct for IB's RC and RD transports. The IBTA spec has more info (see RNR retry and RNR timer). The struct ib_qp_attr's min_rnr_timer and rnr_retry fields allow fine grained control of these values (rnr_retry == 7 means infinite). ULPs generaly interact with these values through either the IB CM's struct ib_cm_req_param/struct ib_cm_rep_param (if the ULP is IB specific) or the RDMA CM's struct rdma_conn_param. > If so, How can I detect that this is happening on an IB RC? IE is > there some stat I can query? Good question. I don't see an obvious stat in /sys/class/infiniband. From ralph.campbell at qlogic.com Tue Mar 4 14:21:59 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 04 Mar 2008 14:21:59 -0800 Subject: [ofa-general] detecting "recv buffer not available" events In-Reply-To: <47CDBE41.1060600@opengridcomputing.com> References: <47CDBE41.1060600@opengridcomputing.com> Message-ID: <1204669319.5109.579.camel@brick.pathscale.com> There is no generic counter you can query to find the number of RNR retires. The best you can do is set the IBV_QP_RNR_RETRY parameter to zero and see if you get any IBV_WC_RNR_RETRY_EXC_ERR error completions. If IBV_QP_RNR_RETRY is N, the HCA will do up to N=0..6 retries for you (or if it is 7, an infinite number of times). On Tue, 2008-03-04 at 15:25 -0600, Steve Wise wrote: > All, > > This question is regarding SENDs arriving when no RECV buffer is posted. > For IB, somehow the transport can deal with this and force the sender > transport to retransmit. > > First, is my statement above correct? > > If so, How can I detect that this is happening on an IB RC? IE is there > some stat I can query? > > Thanks, > > Steve. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From clameter at sgi.com Tue Mar 4 14:35:21 2008 From: clameter at sgi.com (Christoph Lameter) Date: Tue, 4 Mar 2008 14:35:21 -0800 (PST) Subject: [ofa-general] Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <20080304222030.GB8951@v2.random> References: <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> Message-ID: On Tue, 4 Mar 2008, Andrea Arcangeli wrote: > I once ripped invalidate_page while working on #v8 but then I > reintroduced it because I thought reducing the total number of hooks > was beneficial to the core linux VM (even if only a > microoptimization, I sure agree about that, but it's trivial to add > one hook instead of two hooks there, so a microoptimization was worth > it IMHO). Well the problem is if one does not have the begin/end hooks then reliable clearing of the mapping may not be possible. begin/end allow holding off new references and that avoids the issue that would come with an single callback that could race with something else. > Your API is also too restrictive, if we'll happen to need one more > method that doesn't take just (start,end) we'll have to cause all > drivers to have significant changes instead of one-liners to use > whatever new feature. What would that be? I think the API need to stay as simple as possible. And this set is pretty minimal and easy to understand. Not having the invalidate_page() removes a troublespot from the API. > IMHO the design is actually same and I don't understand why you > rewrote it once more time in a less flexibile way (on a style side > you're not even using hlist), dropping RCU (not sure how you replace > it with), etc.... All of that is needed in order to allow sleeping in the future. Your version locks us into atomic callbacks. It also makes the API needlessly complex. RCU means that the callbacks occur in an atomic context. > Converging in a single design is great, but it'd be nice if we could > converge into a single implementation, and my last patch doesn't have > any bug and I think it's quite nicer too (also including Nick cleanup > work) but then I may be biased ;). It is the atomic dead end that we want to avoid. And your patch is exactly that. Both the invalidate_page and the RCU locks us into this. > But as usual I'm entirely satisfied by your brand new EMM Notifier to > be merged and all perfecting work done on my MMU notifier patch over > the weeks by multiple developers (including you) to be dropped for > good, as long as we can enable the new advanced KVM features in > 2.6.25. Well I really want us to have one API that is suitable for multiple purposes and that allows a generic use by device drivers for multiple purposes. The discussion in the last month have made that possible. I am glad that you do not see any major issues with the patch. I sure wish I would not have to post a competing patchset because I want things to be merged ASAP and get this over with. But we need to have at minimum clear way to support sleeping with the existing API in the future. From swise at opengridcomputing.com Tue Mar 4 14:44:52 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 04 Mar 2008 16:44:52 -0600 Subject: [ofa-general] [PATCH 2.6.25] RDMA/IWCM: don't access the cm_id after dereferencing it. Message-ID: <20080304224451.8548.33855.stgit@dell3.ogc.int> RDMA/IWCM: don't access the cm_id after dereferencing it. cm_work_handler() can access cm_id_priv after it was potentially freed by iwch_deref_id(). The fix is to note whether IWCM_F_CALLBACK_DESTROY is set before dereferencing. Then if it was set, free the cm_id on this thread. Signed-off-by: Steve Wise --- drivers/infiniband/core/iwcm.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c index 223b1aa..81c9195 100644 --- a/drivers/infiniband/core/iwcm.c +++ b/drivers/infiniband/core/iwcm.c @@ -839,6 +839,7 @@ static void cm_work_handler(struct work_struct *_work) unsigned long flags; int empty; int ret = 0; + int destroy_id; spin_lock_irqsave(&cm_id_priv->lock, flags); empty = list_empty(&cm_id_priv->work_list); @@ -857,9 +858,9 @@ static void cm_work_handler(struct work_struct *_work) destroy_cm_id(&cm_id_priv->id); } BUG_ON(atomic_read(&cm_id_priv->refcount)==0); + destroy_id = test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags); if (iwcm_deref_id(cm_id_priv)) { - if (test_bit(IWCM_F_CALLBACK_DESTROY, - &cm_id_priv->flags)) { + if (destroy_id) { BUG_ON(!list_empty(&cm_id_priv->work_list)); free_cm_id(cm_id_priv); } From swise at opengridcomputing.com Tue Mar 4 14:49:50 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 04 Mar 2008 16:49:50 -0600 Subject: [ofa-general] [RFC PATCH 04/14] RFC RDMA/amso1100: Don't use 0UL as a NULL pointer In-Reply-To: <20082292026.payxI6A60JrUgBpz@cisco.com> References: <20082292026.payxI6A60JrUgBpz@cisco.com> Message-ID: <47CDD20E.1000503@opengridcomputing.com> Acked-by: Steve Wise Roland Dreier wrote: > Write tests for NULL pointers as > > if (!ptr) > > instead of > > if (ptr == 0UL) > > to fix sparse warnings. > > Signed-off-by: Roland Dreier > --- > drivers/infiniband/hw/amso1100/c2.c | 10 +++++----- > 1 files changed, 5 insertions(+), 5 deletions(-) > > diff --git a/drivers/infiniband/hw/amso1100/c2.c b/drivers/infiniband/hw/amso1100/c2.c > index f283a9f..c50533b 100644 > --- a/drivers/infiniband/hw/amso1100/c2.c > +++ b/drivers/infiniband/hw/amso1100/c2.c > @@ -1005,7 +1005,7 @@ static int __devinit c2_probe(struct pci_dev *pcidev, > /* Remap the adapter PCI registers in BAR4 */ > mmio_regs = ioremap_nocache(reg4_start + C2_PCI_REGS_OFFSET, > sizeof(struct c2_adapter_pci_regs)); > - if (mmio_regs == 0UL) { > + if (!mmio_regs) { > printk(KERN_ERR PFX > "Unable to remap adapter PCI registers in BAR4\n"); > ret = -EIO; > @@ -1109,7 +1109,7 @@ static int __devinit c2_probe(struct pci_dev *pcidev, > /* Remap the adapter HRXDQ PA space to kernel VA space */ > c2dev->mmio_rxp_ring = ioremap_nocache(reg4_start + C2_RXP_HRXDQ_OFFSET, > C2_RXP_HRXDQ_SIZE); > - if (c2dev->mmio_rxp_ring == 0UL) { > + if (!c2dev->mmio_rxp_ring) { > printk(KERN_ERR PFX "Unable to remap MMIO HRXDQ region\n"); > ret = -EIO; > goto bail6; > @@ -1118,7 +1118,7 @@ static int __devinit c2_probe(struct pci_dev *pcidev, > /* Remap the adapter HTXDQ PA space to kernel VA space */ > c2dev->mmio_txp_ring = ioremap_nocache(reg4_start + C2_TXP_HTXDQ_OFFSET, > C2_TXP_HTXDQ_SIZE); > - if (c2dev->mmio_txp_ring == 0UL) { > + if (!c2dev->mmio_txp_ring) { > printk(KERN_ERR PFX "Unable to remap MMIO HTXDQ region\n"); > ret = -EIO; > goto bail7; > @@ -1129,7 +1129,7 @@ static int __devinit c2_probe(struct pci_dev *pcidev, > > /* Remap the PCI registers in adapter BAR0 to kernel VA space */ > c2dev->regs = ioremap_nocache(reg0_start, reg0_len); > - if (c2dev->regs == 0UL) { > + if (!c2dev->regs) { > printk(KERN_ERR PFX "Unable to remap BAR0\n"); > ret = -EIO; > goto bail8; > @@ -1139,7 +1139,7 @@ static int __devinit c2_probe(struct pci_dev *pcidev, > c2dev->pa = reg4_start + C2_PCI_REGS_OFFSET; > c2dev->kva = ioremap_nocache(reg4_start + C2_PCI_REGS_OFFSET, > kva_map_size); > - if (c2dev->kva == 0UL) { > + if (!c2dev->kva) { > printk(KERN_ERR PFX "Unable to remap BAR4\n"); > ret = -EIO; > goto bail9; From swise at opengridcomputing.com Tue Mar 4 14:52:51 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 04 Mar 2008 16:52:51 -0600 Subject: [ofa-general] [RFC PATCH 15/14] RDMA/amso1100: Start of endianness annotation In-Reply-To: References: <20082292026.yXkK6fEAjkkUN7jE@cisco.com> Message-ID: <47CDD2C3.2060707@opengridcomputing.com> Acked-by: Steve Wise Roland Dreier wrote: > Signed-off-by: Roland Dreier > --- > A bonus patch... cuts down on the sparse noise considerably, although > there are still more related to the mqsp stuff, which I haven't > understood sufficiently to annotate properly. > From clameter at sgi.com Tue Mar 4 15:14:42 2008 From: clameter at sgi.com (Christoph Lameter) Date: Tue, 4 Mar 2008 15:14:42 -0800 (PST) Subject: [ofa-general] Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <1204670529.6241.52.camel@lappy> References: <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <1204670529.6241.52.camel@lappy> Message-ID: On Tue, 4 Mar 2008, Peter Zijlstra wrote: > > On Tue, 2008-03-04 at 14:35 -0800, Christoph Lameter wrote: > > > RCU means that the callbacks occur in an atomic context. > > Not really, if it requires moving the VM locks to sleepable locks under > a .config option, I think its also fair to require PREEMPT_RCU. Which would make the patchset pretty complex. RCU is not needed with a single linked list. Linked list operations can exploit atomic pointer updates and we only tear down the list when a single execution thread remains. Having said that: Here a couple of updates to address Andrea's complaint that we not check the referenced bit from the external mapper when the rerferences bit is set on an OS pte. Plus two barriers to ensure that a new emm notifier object becomes visible before the base pointer is updated. Signed-off-by: Christoph Lameter --- mm/rmap.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) Index: linux-2.6/mm/rmap.c =================================================================== --- linux-2.6.orig/mm/rmap.c 2008-03-04 14:36:36.321922321 -0800 +++ linux-2.6/mm/rmap.c 2008-03-04 15:10:46.159429369 -0800 @@ -298,10 +298,10 @@ static int page_referenced_one(struct pa (*mapcount)--; pte_unmap_unlock(pte, ptl); - if (!referenced) - /* rmap lock held */ - referenced = emm_notify(mm, emm_referenced, - address, address + PAGE_SIZE); + + /* rmap lock held */ + if (emm_notify(mm, emm_referenced, address, address + PAGE_SIZE)) + referenced = 1; out: return referenced; } @@ -1057,6 +1057,7 @@ EXPORT_SYMBOL_GPL(emm_notifier_release); void emm_notifier_register(struct emm_notifier *e, struct mm_struct *mm) { e->next = mm->emm_notifier; + smp_wmb(); mm->emm_notifier = e; } EXPORT_SYMBOL_GPL(emm_notifier_register); @@ -1069,6 +1070,7 @@ int __emm_notify(struct mm_struct *mm, e int x; while (e) { + smp_rmb(); if (e->func) { x = e->func(e, mm, op, start, end); if (x) From a.p.zijlstra at chello.nl Tue Mar 4 14:42:09 2008 From: a.p.zijlstra at chello.nl (Peter Zijlstra) Date: Tue, 04 Mar 2008 23:42:09 +0100 Subject: [ofa-general] ***SPAM*** Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: References: <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> Message-ID: <1204670529.6241.52.camel@lappy> On Tue, 2008-03-04 at 14:35 -0800, Christoph Lameter wrote: > RCU means that the callbacks occur in an atomic context. Not really, if it requires moving the VM locks to sleepable locks under a .config option, I think its also fair to require PREEMPT_RCU. OTOH, if you want to unconditionally move the VM locks to sleepable locks you have a point. From a.p.zijlstra at chello.nl Tue Mar 4 15:25:00 2008 From: a.p.zijlstra at chello.nl (Peter Zijlstra) Date: Wed, 05 Mar 2008 00:25:00 +0100 Subject: [ofa-general] ***SPAM*** Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: References: <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <1204670529.6241.52.camel@lappy> Message-ID: <1204673100.6241.59.camel@lappy> On Tue, 2008-03-04 at 15:14 -0800, Christoph Lameter wrote: > On Tue, 4 Mar 2008, Peter Zijlstra wrote: > > > > > On Tue, 2008-03-04 at 14:35 -0800, Christoph Lameter wrote: > > > > > RCU means that the callbacks occur in an atomic context. > > > > Not really, if it requires moving the VM locks to sleepable locks under > > a .config option, I think its also fair to require PREEMPT_RCU. > > Which would make the patchset pretty complex. RCU is not needed with a > single linked list. Linked list operations can exploit atomic pointer > updates and we only tear down the list when a single execution thread > remains. OK, that constraint on removal makes it work. > Having said that: Here a couple of updates to address Andrea's complaint > that we not check the referenced bit from the external mapper when the > rerferences bit is set on an OS pte. > > Plus two barriers to ensure that a new emm notifier object becomes > visible before the base pointer is updated. > > Signed-off-by: Christoph Lameter > > --- > mm/rmap.c | 10 ++++++---- > 1 file changed, 6 insertions(+), 4 deletions(-) > > Index: linux-2.6/mm/rmap.c > =================================================================== > --- linux-2.6.orig/mm/rmap.c 2008-03-04 14:36:36.321922321 -0800 > +++ linux-2.6/mm/rmap.c 2008-03-04 15:10:46.159429369 -0800 > @@ -298,10 +298,10 @@ static int page_referenced_one(struct pa > > (*mapcount)--; > pte_unmap_unlock(pte, ptl); > - if (!referenced) > - /* rmap lock held */ > - referenced = emm_notify(mm, emm_referenced, > - address, address + PAGE_SIZE); > + > + /* rmap lock held */ > + if (emm_notify(mm, emm_referenced, address, address + PAGE_SIZE)) > + referenced = 1; referenced++; seems more in-style with the rest of that code.. > out: > return referenced; > } > @@ -1057,6 +1057,7 @@ EXPORT_SYMBOL_GPL(emm_notifier_release); > void emm_notifier_register(struct emm_notifier *e, struct mm_struct *mm) > { > e->next = mm->emm_notifier; > + smp_wmb(); > mm->emm_notifier = e; > } > EXPORT_SYMBOL_GPL(emm_notifier_register); > @@ -1069,6 +1070,7 @@ int __emm_notify(struct mm_struct *mm, e > int x; > > while (e) { > + smp_rmb(); > if (e->func) { > x = e->func(e, mm, op, start, end); > if (x) We generally require comments around barriers.. From a.p.zijlstra at chello.nl Tue Mar 4 15:30:40 2008 From: a.p.zijlstra at chello.nl (Peter Zijlstra) Date: Wed, 05 Mar 2008 00:30:40 +0100 Subject: [ofa-general] ***SPAM*** Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <1204673100.6241.59.camel@lappy> References: <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <1204670529.6241.52.camel@lappy> <1204673100.6241.59.camel@lappy> Message-ID: <1204673440.6241.61.camel@lappy> FWIW, I'll cut the kvm and openfabrics lists from any future posts. I'm getting tired of the bounces. From npiggin at suse.de Tue Mar 4 16:37:22 2008 From: npiggin at suse.de (Nick Piggin) Date: Wed, 5 Mar 2008 01:37:22 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: References: <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> Message-ID: <20080305003722.GG1510@wotan.suse.de> On Mon, Mar 03, 2008 at 11:01:22AM -0800, Christoph Lameter wrote: > On Mon, 3 Mar 2008, Nick Piggin wrote: > > > I'm still not completely happy with this. I had a very quick look > > at the GRU driver, but I don't see why it can't be implemented > > more like the regular TLB model, and have TLB insertions depend on > > the linux pte, and do invalidates _after_ restricting permissions > > to the pte. > > > > Ie. I'd still like to get rid of invalidate_range_begin, and get > > rid of invalidate calls from places where permissions are relaxed. > > Isnt this more a job for paravirt ops if it is so tightly bound to page > tables? Are we not adding another similar API? Um, it's bound to the *Linux page tables*, yes. And I have no idea why you would use the paravirt ops for this. From nickpiggin at yahoo.com.au Tue Mar 4 16:52:13 2008 From: nickpiggin at yahoo.com.au (Nick Piggin) Date: Wed, 5 Mar 2008 11:52:13 +1100 Subject: [ofa-general] Re: [patch 2/6] mmu_notifier: Callbacks to invalidate address ranges In-Reply-To: References: <20080215064859.384203497@sgi.com> <200803040650.11942.nickpiggin@yahoo.com.au> Message-ID: <200803051152.15160.nickpiggin@yahoo.com.au> On Wednesday 05 March 2008 05:58, Christoph Lameter wrote: > On Tue, 4 Mar 2008, Nick Piggin wrote: > > > Then put it into the arch code for TLB invalidation. Paravirt ops gives > > > good examples on how to do that. > > > > Put what into arch code? > > The mmu notifier code. It isn't arch specific. > > > > What about a completely different approach... XPmem runs over > > > > NUMAlink, right? Why not provide some non-sleeping way to basically > > > > IPI remote nodes over the NUMAlink where they can process the > > > > invalidation? If you intra-node cache coherency has to run over this > > > > link anyway, then presumably it is capable. > > > > > > There is another Linux instance at the remote end that first has to > > > remove its own ptes. > > > > Yeah, what's the problem? > > The remote end has to invalidate the page which involves locking etc. I don't see what the problem is. > > > Also would not work for Inifiniband and other > > > solutions. > > > > infiniband doesn't want it. Other solutions is just handwaving, > > because if we don't know what the other soloutions are, then we can't > > make any sort of informed choices. > > We need a solution in general to avoid the pinning problems. Infiniband > has those too. > > > > All the approaches that require evictions in an atomic context > > > are limiting the approach and do not allow the generic functionality > > > that we want in order to not add alternate APIs for this. > > > > The only generic way to do this that I have seen (and the only proposed > > way that doesn't add alternate APIs for that matter) is turning VM locks > > into sleeping locks. In which case, Andrea's notifiers will work just > > fine (except for relatively minor details like rcu list scanning). > > No they wont. As you pointed out the callback need RCU locking. That can be fixed easily. > > > The good enough solution right now is to pin pages by elevating > > > refcounts. > > > > Which kind of leads to the question of why do you need any further > > kernel patches if that is good enough? > > Well its good enough with severe problems during reclaim, livelocks etc. > One could improve on that scheme through Rik's work trying to add a new > page flag that mark pinned pages and then keep them off the LRUs and > limiting their number. Having pinned page would limit the ability to > reclaim by the VM and make page migration, memory unplug etc impossible. Well not impossible. You could have a callback to invalidate the remote TLB and drop the pin on a given page. > It is better to have notifier scheme that allows to tell a device driver > to free up the memory it has mapped. Yeah, it would be nice for those people with clusters of Altixes. Doesn't mean it has to go upstream, though. From meier3 at llnl.gov Tue Mar 4 17:21:36 2008 From: meier3 at llnl.gov (Timothy A. Meier) Date: Tue, 04 Mar 2008 17:21:36 -0800 Subject: [ofa-general] [PATCH] opensm: provide methods for getting the OpenSM and Log Message-ID: <47CDF5A0.1080509@llnl.gov> Hi Sasha, I wrote this patch so I could avoid passing around a pointer to the opensm, just to get access to the log. Also, it provides a method to get a pointer to the opensm as well, because there are several instances where this is needed for a trivial reason, and many functions include it as one of their arguments. Since the opensm is a singleton, it could just be accessed directly by the function when its needed. Ultimately, this can lead to simplification of header files. -- Timothy A. Meier Computer Scientist ICCD/High Performance Computing 925.422.3341 meier3 at llnl.gov -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-opensm-provide-methods-for-getting-the-OpenSM-and.patch URL: From esywv at aos.every1.net Tue Mar 4 17:22:50 2008 From: esywv at aos.every1.net (danielle) Date: Tue, 04 Mar 2008 17:22:50 -0800 Subject: [ofa-general] it`s danielle Message-ID: <4564376_@TLZ5421860_@TLZ> hello, I am pretty russian girl, bored tonight. would you like to chat with me and see my pics? if so then email me at edanielle4 at golovaonline.com From dladqwuwnvggfc at planetstorm.every1.net Tue Mar 4 17:22:59 2008 From: dladqwuwnvggfc at planetstorm.every1.net (kendra) Date: Tue, 04 Mar 2008 17:22:59 -0800 Subject: [ofa-general] hi from kendra Message-ID: <18401122944625.0024.qmail@zmusic.net> hello, I am pretty russian girl, bored tonight. would you like to chat with me and see my pics? if so then email me at ekendra862 at golovaonline.com From troy at scl.ameslab.gov Tue Mar 4 18:48:38 2008 From: troy at scl.ameslab.gov (Troy Benjegerdes) Date: Tue, 04 Mar 2008 20:48:38 -0600 Subject: [ofa-general] valgrind warnings in libibverbs & PVFS Message-ID: <47CE0A06.7060509@scl.ameslab.gov> I am trying to track down some issues with PVFS using IB with valgrind, and I'm trying to make a run of 'valgrind pvfs2-ls' come out with no errors. The first thing I managed to figure out is this change to libibverbs: p4l4:/usr/src/ib/ofed-1_3_git/libibverbs/Bppc32# git diff diff --git a/src/cmd.c b/src/cmd.c diff --git a/src/verbs.c b/src/verbs.c index 11d3c4c..bdfe723 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -226,6 +226,8 @@ struct ibv_comp_channel *ibv_create_comp_channel(struct ibv_context *context) return NULL; } + VALGRIND_MAKE_MEM_DEFINED(&resp.fd, sizeof(resp.fd)); + channel->context = context; channel->fd = resp.fd; channel->refcnt = 0; However, I am still getting two errors, and I can't seem to figure out if it's a PVFS issue, an ibverbs issue, or a libmthca issue, and I'm wondering how to track this down. troy at p4l4:/usr/src/pvfs2-hg/Bppc32$ valgrind src/apps/admin/pvfs2-ls ==2541== Memcheck, a memory error detector. ==2541== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. ==2541== Using LibVEX rev 1658, a library for dynamic binary translation. ==2541== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==2541== Using valgrind-3.2.1-Debian, a dynamic binary instrumentation framework. ==2541== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==2541== For more details, rerun with: -v ==2541== ==2541== Syscall param write(buf) points to uninitialised byte(s) ==2541== at 0xFEB2B14: write (in /usr/lib/debug/libpthread-0.10.so) ==2541== by 0xFE7DD3C: ibv_cmd_create_cq (cmd.c:320) ==2541== by 0xFE84080: ibv_create_cq@@IBVERBS_1.1 (verbs.c:281) ==2541== by 0x10055C0C: openib_ib_initialize (openib.c:956) ==2541== by 0x10051458: BMI_ib_initialize (ib.c:2001) ==2541== by 0x1004CA00: activate_method (bmi.c:2008) ==2541== by 0x1004CEE4: BMI_addr_lookup (bmi.c:1555) ==2541== by 0x10032D20: PVFS_isys_fs_add (fs-add.sm:127) ==2541== by 0x10032F50: PVFS_sys_fs_add (fs-add.sm:194) ==2541== by 0x1000B698: main (pvfs2-ls.c:766) ==2541== Address 0xFEB9D550 is on thread 1's stack ==2541== ==2541== Conditional jump or move depends on uninitialised value(s) ==2541== at 0xFB19A80: mthca_cq_clean (cq.c:576) ==2541== by 0xFB1D688: mthca_destroy_qp (verbs.c:674) ==2541== by 0xFE83758: ibv_destroy_qp@@IBVERBS_1.1 (verbs.c:490) ==2541== by 0x1005693C: openib_close_connection (openib.c:368) ==2541== by 0x1004F6E4: ib_close_connection (ib.c:1695) ==2541== by 0x10051800: BMI_ib_finalize (ib.c:2082) ==2541== by 0x1004DFA8: BMI_finalize (bmi.c:474) ==2541== by 0x100112B0: PVFS_sys_finalize (finalize.c:57) ==2541== by 0x1000BDF0: main (pvfs2-ls.c:850) ==2541== ==2541== ERROR SUMMARY: 7 errors from 2 contexts (suppressed: 9 from 5) ==2541== malloc/free: in use at exit: 1,943 bytes in 12 blocks. ==2541== malloc/free: 820 allocs, 808 frees, 67,920,532 bytes allocated. ==2541== For counts of detected errors, rerun with: -v ==2541== searching for pointers to 12 not-freed blocks. ==2541== checked 653,680 bytes. ==2541== ==2541== LEAK SUMMARY: ==2541== definitely lost: 0 bytes in 0 blocks. ==2541== possibly lost: 0 bytes in 0 blocks. ==2541== still reachable: 1,943 bytes in 12 blocks. ==2541== suppressed: 0 bytes in 0 blocks. ==2541== Reachable blocks (those to which a pointer was found) are not shown. ==2541== To see them, rerun with: --show-reachable=yes From avi at qumranet.com Tue Mar 4 21:09:55 2008 From: avi at qumranet.com (Avi Kivity) Date: Wed, 05 Mar 2008 07:09:55 +0200 Subject: [ofa-general] Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <1204670529.6241.52.camel@lappy> References: <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <1204670529.6241.52.camel@lappy> Message-ID: <47CE2B23.6010505@qumranet.com> Peter Zijlstra wrote: > On Tue, 2008-03-04 at 14:35 -0800, Christoph Lameter wrote: > > >> RCU means that the callbacks occur in an atomic context. >> > > Not really, if it requires moving the VM locks to sleepable locks under > a .config option, I think its also fair to require PREEMPT_RCU. > > OTOH, if you want to unconditionally move the VM locks to sleepable > locks you have a point. > Isn't that out of the question for .25? I really wish we can get the atomic variant in now, and add on sleepability in .26, updating users if necessary. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. From erezz at voltaire.com Tue Mar 4 22:26:59 2008 From: erezz at voltaire.com (Erez Zilber) Date: Wed, 05 Mar 2008 08:26:59 +0200 Subject: [ofa-general] Re: [PATCH 1/2] IB/iSER: fix list iteration bug In-Reply-To: <878x0yfqe7.fsf@confield.dd.xiranet.com> References: <877igkxffl.fsf@confield.dd.xiranet.com> <47CD3B7A.2010106@voltaire.com> <878x0yfqe7.fsf@confield.dd.xiranet.com> Message-ID: <47CE3D33.7060805@voltaire.com> Arne Redlich wrote: > Erez Zilber writes: > > >> Arne Redlich wrote: >> >>> The iteration through the list of "iser_device"s during device >>> lookup/creation is broken - it might result in an infinite loop if more >>> than 1 HCA is used with iSER. Use list_for_each_entry() instead of the >>> custom, flawed list iteration code. >>> >>> Signed-off-by: Arne Redlich >>> --- >>> drivers/infiniband/ulp/iser/iser_verbs.c | 36 ++++++++++++----------------- >>> 1 files changed, 15 insertions(+), 21 deletions(-) >>> >>> diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c >>> index 714b8db..1c0f968 100644 >>> --- a/drivers/infiniband/ulp/iser/iser_verbs.c >>> +++ b/drivers/infiniband/ulp/iser/iser_verbs.c >>> @@ -237,33 +237,27 @@ static int iser_free_ib_conn_res(struct iser_conn *ib_conn) >>> static >>> struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) >>> { >>> - struct list_head *p_list; >>> - struct iser_device *device = NULL; >>> + struct iser_device *device; >>> >>> mutex_lock(&ig.device_list_mutex); >>> >>> - p_list = ig.device_list.next; >>> - while (p_list != &ig.device_list) { >>> - device = list_entry(p_list, struct iser_device, ig_list); >>> - /* find if there's a match using the node GUID */ >>> + list_for_each_entry(device, &ig.device_list, ig_list) >>> >> I've just added the original comments that are missing in your patch. >> > > Ah well, I probably should've mentioned in the patch description that I > intentionally removed those comments as I think they're really > redundant, stating the obvious. But of course I won't insist. :) > > Thanks, > Arne > OK and thanks for catching the bugs. Roland - will you be able to merge these 2 patches for 2.6.25? Both are bug fixes. Thanks, Erez From ogerlitz at voltaire.com Tue Mar 4 23:57:52 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 05 Mar 2008 09:57:52 +0200 Subject: [ofa-general] detecting "recv buffer not available" events In-Reply-To: <1204669319.5109.579.camel@brick.pathscale.com> References: <47CDBE41.1060600@opengridcomputing.com> <1204669319.5109.579.camel@brick.pathscale.com> Message-ID: <47CE5280.3040501@voltaire.com> Ralph Campbell wrote: > There is no generic counter you can query to find the number > of RNR retires. The best you can do is set the IBV_QP_RNR_RETRY > parameter to zero and see if you get any IBV_WC_RNR_RETRY_EXC_ERR > error completions. Applicatively, if you are using the RDMA-CM, what Ralph suggests is to set the rnr_retry_count conn param field when calling rdam_connnect/accept Or. From crawfordfunds52 at yahoo.dk Tue Mar 4 22:46:30 2008 From: crawfordfunds52 at yahoo.dk (Lady Helen Crawford) Date: Wed, 5 Mar 2008 08:46:30 +0200 Subject: [ofa-general] MY WISH....."Crawford Charity Foundation" Message-ID: <1204699590.47ce41c64dfe3@webmail.altecnet.gr> Lady Helen Crawford. 38 Old Church Street, Manchester, England. Here writes Lady Helen Crawford, suffering from cancerous ailment. I was married to Sir William Crawford an English man (Deceased), who died after a protracted illness. When my husband was alive, he was into private practice all his life. Our life together as husband and wife lasted for about three decades without a child. My late husband and I made a vow to uplift the down- trodden and the less-privileged individuals in the society, as we both had passion for those people who can not help themselves due to physical disability or financial predicament. I can adduce this to the fact that we both needed a Child from this relationship, which never came. When my late husband was alive he deposited the sum of Twenty Five Million Pounds Sterling(25,000,000.00 GBP) which were derived from his vast estates and investment in capital market with his bank here in UK(all records are kept with my family lawyer). Presently, the money is still with the Bank. Recently, my Doctor told me that, I have limited days to live, due to the cancerous problems that i have been suffering from. Though what bothers me most is the stroke that I have in addition with the cancer. Hence, With this hard reality that has befallen my family and me, I have decided to donate this fund to you and i want you to use this gift which comes from my Late husbands effort to establish a charity home called "Crawford Charity Foundation" for the upkeep of the down-trodden and persons who prove to be genuinely handicapped financially. I took this decision because I do not have any child that will inherit this money and my late husband relatives are bourgeois and very wealthy persons and I do not want my husband hard earned money to be misused or invested into ill perceived ventures. Hence the reason for taking this bold decision. I do not need any telephone communication in this regard due to my deteriorating health and also because of the presence of my husband relatives around me. I do not want them to know about this development, that is why I cannot give this fund to anybody here in England, because my Husband relatives are people who are highly placed here in England, and will do very thing possible to frustrate all attempts within this country. Moreso they do not know about this Twenty Five Million Pounds sterling deposit that my late husband made with his bank. Presently, this deposit is only known by me, my family lawyer and now you. So, i want you to Contact my family lawyer Barrister Paul Westley (ESQ) with this specified email addressemail: attnypaul_westleychambesr at yahoo.co.uk and Tell him that I have WILLED the £25,000,000 to you by quoting my personal reference number Law/chamber/solicitors/rt/osb/WILL/9834520012, as I have also notified him that I will be WILLING that amount to you for a specific purpose and good work. Please, assure me that you will act just as I have stated herein. as i Hope to hear from you soon that you have contacted my family lawyer with the email address that i have given to you above, as he is presently waiting to receive your reply. Remain blessed your Sister, Lady Helen Crawford From holt at sgi.com Wed Mar 5 01:47:37 2008 From: holt at sgi.com (Robin Holt) Date: Wed, 5 Mar 2008 03:47:37 -0600 Subject: [ofa-general] Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <47CE2B23.6010505@qumranet.com> References: <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <1204670529.6241.52.camel@lappy> <47CE2B23.6010505@qumranet.com> Message-ID: <20080305094736.GA2013@sgi.com> On Wed, Mar 05, 2008 at 07:09:55AM +0200, Avi Kivity wrote: > Isn't that out of the question for .25? I keep hearing this mantra. What is so compelling about the .25 release? When seems to be more important than what. While I understand product release cycles, etc. and can certainly agree with them. I would like to know with what I am being asked to agree. That said, I agree we should probably finish getting the comments on Andrea's most recent patch, if any, cleared up and put that one in. Robin From avi at qumranet.com Wed Mar 5 01:53:17 2008 From: avi at qumranet.com (Avi Kivity) Date: Wed, 05 Mar 2008 11:53:17 +0200 Subject: [ofa-general] Re: [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <20080305094736.GA2013@sgi.com> References: <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <1204670529.6241.52.camel@lappy> <47CE2B23.6010505@qumranet.com> <20080305094736.GA2013@sgi.com> Message-ID: <47CE6D8D.9060306@qumranet.com> Robin Holt wrote: > On Wed, Mar 05, 2008 at 07:09:55AM +0200, Avi Kivity wrote: > >> Isn't that out of the question for .25? >> > > I keep hearing this mantra. What is so compelling about the .25 > release? When seems to be more important than what. While I understand > product release cycles, etc. and can certainly agree with them. I would > like to know with what I am being asked to agree. > > kvm gained the ability to swap in 2.6.25. Without mmu notifiers, though, the guest can still easily pin all of its memory. > That said, I agree we should probably finish getting the comments on > Andrea's most recent patch, if any, cleared up and put that one in. > Great. Thanks. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. From dor.laor at gmail.com Wed Mar 5 02:02:57 2008 From: dor.laor at gmail.com (Dor Laor) Date: Wed, 05 Mar 2008 12:02:57 +0200 Subject: [ofa-general] Re: [kvm-devel] [RFC] Notifier for Externally Mapped Memory (EMM) In-Reply-To: <20080305094736.GA2013@sgi.com> References: <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <1204670529.6241.52.camel@lappy> <47CE2B23.6010505@qumranet.com> <20080305094736.GA2013@sgi.com> Message-ID: <1204711377.31109.19.camel@localhost.localdomain> On Wed, 2008-03-05 at 03:47 -0600, Robin Holt wrote: > On Wed, Mar 05, 2008 at 07:09:55AM +0200, Avi Kivity wrote: > > Isn't that out of the question for .25? > > I keep hearing this mantra. What is so compelling about the .25 > release? When seems to be more important than what. While I understand > product release cycles, etc. and can certainly agree with them. I would > like to know with what I am being asked to agree. > The main reason is that several kvm exciting features are dependent on mmu notifiers: - It enables full guest swapping (as opposed to partial today) - It enables memory ballooning - It enables running Izik Eidus's Kernel Shared Pages module that unify guest pages together. The patchset is kernel-internal, stable and reviewed. Even if the interface will be changed in .26 it won't have noticeable effect. So since its stable, internal, reviewed, needed to enable important kvm features we like to see it in for .25. Regards, Dor > That said, I agree we should probably finish getting the comments on > Andrea's most recent patch, if any, cleared up and put that one in. > > Robin > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > kvm-devel mailing list > kvm-devel at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/kvm-devel From Rivera at rcgarage.com Wed Mar 5 02:08:07 2008 From: Rivera at rcgarage.com (Andy Robinson) Date: Wed, 05 Mar 2008 11:08:07 +0100 Subject: [ofa-general] Ssoftware At Loww Prrice Message-ID: <000a01c87ea8$946f8080$0100007f@mugujeq> Go to: sunosoftx. com (w/o space) ----------------- Microsoft Windows Vista Ultimate $79 Macromedia Flash Professional 8 $49 Adobe Premiere 2.0 $59 ----------------- For Mac: Adobe Acrobat Pro 7 $69 Adobe After Effects $49 Macromedia Flash Pro 8 $49 ----------------- Ramsey walked inside, saw Brod No, but God knows I tried. Its No, it isnt. How about, Gillia From sashak at voltaire.com Wed Mar 5 02:43:53 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 10:43:53 +0000 Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> <20080301161023.GM1485@sashak.voltaire.com> <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> Message-ID: <20080305104353.GA27256@sashak.voltaire.com> Hi Al, On 07:46 Sun 02 Mar , Albert Chu wrote: > > In order to make things work, I also had to add this patch. Seems like a > corner case that needs to be handled since we never fall into > __osm_pi_rcv_process_switch_port(). Hmm, it is strange. After this light sweep cycle OpenSM should continue with heavy sweep where __osm_pi_rcv_process_switch_port() should be reissued. Do you see any errors during discovery? Sasha From sashak at voltaire.com Wed Mar 5 03:01:34 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 11:01:34 +0000 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <47CB2870.50602@mellanox.co.il> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47CB2870.50602@mellanox.co.il> Message-ID: <20080305110134.GB27256@sashak.voltaire.com> On 00:21 Mon 03 Mar , Yevgeny Kliteynik wrote: > > BTW, are you aware of any other IBA 1.2.1 - related issues that need to be > fixed? > I mean, is OpenSM fully IBA 1.2.1 compliant? At least it is a target, we have "Unsupported IB Compliance Statements" list in release notes. Of course there could be more issues. Sasha From tstormde at 4-all-theme-websites.com Wed Mar 5 02:51:42 2008 From: tstormde at 4-all-theme-websites.com (Ebrahim Westrick) Date: Wed, 5 Mar 2008 12:51:42 +0200 Subject: [ofa-general] Regain your lost inches now Message-ID: <000d01c87eae$e5c2b050$81ca4955@STATION02> See the passion in her eyes with your newly acquired member -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Wed Mar 5 03:10:45 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 11:10:45 +0000 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080305111045.GC27256@sashak.voltaire.com> Hi Hal, On 10:22 Tue 04 Mar , Hal Rosenstock wrote: > > As I recall, the QoS manager (at OFED 1.3) is experimental. If so, in > the release notes, QoS manager should be indicated as experimental in > the release notes. Yes, it seems correct. > Additionally, there should be some statement of what > HCAs and firmware versions this works with (and any switch firmware > dependencies if any). Would be nice to have. Yevgeny, could you provide such info? > It also brings on new tools requirements for building (versions of yacc > and lex) which should also be indicated there (and perhaps checked > somehow in the build process). > > Can this get added/updated to the release notes ? The only technical problem I see with this is that 1.3 was released already. We could update release notes in the master - better than nothing. > Also, it would also be nice if there were a way to not include this in > the OpenSM build if not wanted (an --enable-qos-annex configure option > or the like). Is that a straightforward change ? Seems fine for me. And I think it was discussed already, but somehow forgotten :( . We can keep this enabled by default. Sasha From sashak at voltaire.com Wed Mar 5 03:17:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 11:17:08 +0000 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Building In-Reply-To: <1204659253.6469.441.camel@hrosenstock-ws.xsigo.com> References: <1204659253.6469.441.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080305111708.GD27256@sashak.voltaire.com> On 11:34 Tue 04 Mar , Hal Rosenstock wrote: > > When I configure OFED 1.3 OpenSM, I get: > checking for bison... no > checking for byacc... byacc There is a bug about this: https://bugs.openfabrics.org/show_bug.cgi?id=932 Not solved yet :( . Sasha From vlad at lists.openfabrics.org Wed Mar 5 03:09:05 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Wed, 5 Mar 2008 03:09:05 -0800 (PST) Subject: [ofa-general] ofa_1_3_kernel 20080305-0200 daily build status Message-ID: <20080305110905.C7F59E60BA5@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From sashak at voltaire.com Wed Mar 5 03:30:32 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 11:30:32 +0000 Subject: [ofa-general] Re: [OpenSM]: renaming of opensm init script to opensmd In-Reply-To: <1204666444.761.101.camel@cardanus.llnl.gov> References: <1204666444.761.101.camel@cardanus.llnl.gov> Message-ID: <20080305113032.GE27256@sashak.voltaire.com> Hi, On 13:34 Tue 04 Mar , Al Chu wrote: > > One of the sys-admins was wondering why the script name was changed from > opensm to opensmd (which requires him to go change some internal scripts > he uses). I changed this in order to keep backward compatibility with OFED-1.2 (I've got several complains about this). > Our local Redhat guy told me in the past rhel/fedora > convention is the following (which is a comment stolen from rpmlint). > > 'incoherent-init-script-name', > '''The init script name should be the same as the package name in lower > case, > or one with 'd' appended if it invokes a process by that name.''', Seems that it is against the current OFED rules, there 'd' prefix was explicitly requested. Probably OFED should follow a distro conventions? Vlad? Sasha From ogerlitz at voltaire.com Wed Mar 5 03:22:57 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 05 Mar 2008 13:22:57 +0200 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <20080305111045.GC27256@sashak.voltaire.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> Message-ID: <47CE8291.9050303@voltaire.com> Sasha Khapyorsky wrote: >> Additionally, there should be some statement of what >> HCAs and firmware versions this works with (and any switch firmware >> dependencies if any). >> > > Would be nice to have. > > Yevgeny, could you provide such info? > Indeed, Yevgeny, is there any update on the FW issues pointed by you in https://bugs.openfabrics.org/show_bug.cgi?id=821#c11 Or. From sashak at voltaire.com Wed Mar 5 03:48:46 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 11:48:46 +0000 Subject: [ofa-general] [OpenSM] Traps 64 and 65 sent too early. In-Reply-To: References: Message-ID: <20080305114846.GB28730@sashak.voltaire.com> Hi Terry, On 00:59 Tue 04 Mar , Terry Greeniaus wrote: > > I think this could be fixed by moving the > __osm_state_mgr_report_new_ports() call in osm_state_mgr.c to follow the > call that transitions the subnet to Armed or Active, however I am not > very familiar with the OpenSM code so am appealing to the list to help > me out. This seems correct to me. Could you try the patch (against master)? >From 1dfc192236c47edfa24d70967a3027af01aaa28d Mon Sep 17 00:00:00 2001 From: Sasha Khapyorsky Date: Wed, 5 Mar 2008 13:28:37 +0200 Subject: [PATCH] opensm: send trap 64 only after new ports are in ACTIVE state. Send trap 64 only when new ports were moved to ACTIVE state and routing is configured. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_state_mgr.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index 38b2c4e..9b03314 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -1169,10 +1169,7 @@ _repeat_discovery: /* * Proceed with unicast forwarding table configuration. - * First - send trap 64 on newly discovered endports */ - __osm_state_mgr_report_new_ports(sm); - osm_ucast_mgr_process(&sm->ucast_mgr); if (wait_for_pending_transactions(&sm->p_subn->p_osm->stats)) return; @@ -1223,6 +1220,11 @@ _repeat_discovery: * The sweep completed! */ + /* + * Send trap 64 on newly discovered endports + */ + __osm_state_mgr_report_new_ports(sm); + /* in any case we zero this flag */ sm->p_subn->coming_out_of_standby = FALSE; -- 1.5.4.1.122.gaa8d > Also, perhaps there is a bug tracker somewhere against which I should > file a bug report? Sure, it is https://bugs.openfabrics.org . Sasha From partneredib28 at lazarducot.com Thu Mar 6 06:11:00 2008 From: partneredib28 at lazarducot.com (Tania Locke) Date: Wed, 6 Mar 2008 09:11:00 -0500 Subject: [ofa-general] Outlook 2007 Message-ID: <01c87f69$fe9c2a00$745241be@partneredib28> Microsoft Office Enterprise 2007 includes: • Access 2007 • Communicator 2007 • Excel 2007 • Groove 2007 • InfoPath 2007 • OneNote 2007 • Outlook 2007 • PowerPoint 2007 • Publisher 2007 • Word 2007 http://yvonneplatzcd483.blogspot.com System Requirements • Intel® Pentium® or AMD® 500 MHz processor • Microsoft Windows® XP Professional or Home Edition with Service Pack 2, Windows Server® 2003 with SP1 , Microsoft Windows Vista. • 256 Mb of RAM • 2GB of available hard-disk space. • 1024x768 or higher resolution monitor • Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From hrosenstock at xsigo.com Wed Mar 5 06:25:34 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 06:25:34 -0800 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <20080305111045.GC27256@sashak.voltaire.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> Message-ID: <1204727134.6469.563.camel@hrosenstock-ws.xsigo.com> Hi Sasha, On Wed, 2008-03-05 at 11:10 +0000, Sasha Khapyorsky wrote: > Hi Hal, > > On 10:22 Tue 04 Mar , Hal Rosenstock wrote: > > > > As I recall, the QoS manager (at OFED 1.3) is experimental. If so, in > > the release notes, QoS manager should be indicated as experimental in > > the release notes. > > Yes, it seems correct. > > > Additionally, there should be some statement of what > > HCAs and firmware versions this works with (and any switch firmware > > dependencies if any). > > Would be nice to have. > > Yevgeny, could you provide such info? > > > It also brings on new tools requirements for building (versions of yacc > > and lex) which should also be indicated there (and perhaps checked > > somehow in the build process). > > > > Can this get added/updated to the release notes ? > > The only technical problem I see with this is that 1.3 was released > already. We could update release notes in the master - better than > nothing. I expect 1.3 fix releases so it could be done there as well. > > Also, it would also be nice if there were a way to not include this in > > the OpenSM build if not wanted (an --enable-qos-annex configure option > > or the like). Is that a straightforward change ? > > Seems fine for me. And I think it was discussed already, but somehow > forgotten :( . 'Fraid so. That's my recollection as well. -- Hal > We can keep this enabled by default. > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Wed Mar 5 06:27:00 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 06:27:00 -0800 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Building In-Reply-To: <20080305111708.GD27256@sashak.voltaire.com> References: <1204659253.6469.441.camel@hrosenstock-ws.xsigo.com> <20080305111708.GD27256@sashak.voltaire.com> Message-ID: <1204727220.6469.566.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-05 at 11:17 +0000, Sasha Khapyorsky wrote: > On 11:34 Tue 04 Mar , Hal Rosenstock wrote: > > > > When I configure OFED 1.3 OpenSM, I get: > > checking for bison... no > > checking for byacc... byacc > > There is a bug about this: > https://bugs.openfabrics.org/show_bug.cgi?id=932 > > Not solved yet :( . Is this more properly assigned to Yevgeny ? -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Wed Mar 5 06:37:34 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 06:37:34 -0800 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <20080305110134.GB27256@sashak.voltaire.com> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47CB2870.50602@mellanox.co.il> <20080305110134.GB27256@sashak.voltaire.com> Message-ID: <1204727854.6469.576.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-05 at 11:01 +0000, Sasha Khapyorsky wrote: > On 00:21 Mon 03 Mar , Yevgeny Kliteynik wrote: > > > > BTW, are you aware of any other IBA 1.2.1 - related issues that need to be > > fixed? > > I mean, is OpenSM fully IBA 1.2.1 compliant? > At least it is a target, we have "Unsupported IB Compliance Statements" > list in release notes. Of course there could be more issues. Not sure what you mean exactly by fully 1.2.1 compliant. There are two aspects of 1.2.1 changes: 1. Subtle changes in IBA 1.2.1 behavior like this one 2. Optional features added I don't think the spec changes have been compared to OpenSM, at least not as far as I am aware. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From kliteyn at mellanox.co.il Wed Mar 5 06:39:56 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 05 Mar 2008 16:39:56 +0200 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Building In-Reply-To: <1204727220.6469.566.camel@hrosenstock-ws.xsigo.com> References: <1204659253.6469.441.camel@hrosenstock-ws.xsigo.com> <20080305111708.GD27256@sashak.voltaire.com> <1204727220.6469.566.camel@hrosenstock-ws.xsigo.com> Message-ID: <47CEB0BC.50204@mellanox.co.il> Hal Rosenstock wrote: > On Wed, 2008-03-05 at 11:17 +0000, Sasha Khapyorsky wrote: > >> On 11:34 Tue 04 Mar , Hal Rosenstock wrote: >> >>> When I configure OFED 1.3 OpenSM, I get: >>> checking for bison... no >>> checking for byacc... byacc >>> >> There is a bug about this: >> https://bugs.openfabrics.org/show_bug.cgi?id=932 >> >> Not solved yet :( . >> > > Is this more properly assigned to Yevgeny ? > Now it is. -- Yevgeny > -- Hal > > >> Sasha >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >> > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From sashak at voltaire.com Wed Mar 5 07:12:47 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 15:12:47 +0000 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <1204727854.6469.576.camel@hrosenstock-ws.xsigo.com> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47CB2870.50602@mellanox.co.il> <20080305110134.GB27256@sashak.voltaire.com> <1204727854.6469.576.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080305151247.GJ30723@sashak.voltaire.com> Hi Hal, On 06:37 Wed 05 Mar , Hal Rosenstock wrote: > On Wed, 2008-03-05 at 11:01 +0000, Sasha Khapyorsky wrote: > > On 00:21 Mon 03 Mar , Yevgeny Kliteynik wrote: > > > > > > BTW, are you aware of any other IBA 1.2.1 - related issues that need to be > > > fixed? > > > I mean, is OpenSM fully IBA 1.2.1 compliant? > > > At least it is a target, we have "Unsupported IB Compliance Statements" > > list in release notes. Of course there could be more issues. > > Not sure what you mean exactly by fully 1.2.1 compliant. > > There are two aspects of 1.2.1 changes: > 1. Subtle changes in IBA 1.2.1 behavior like this one > 2. Optional features added > > I don't think the spec changes have been compared to OpenSM, at least > not as far as I am aware. I'm not aware too. Do you know how we could prepare a list of 1.2 -> 1.2.1 spec changes? Sasha From hrosenstock at xsigo.com Wed Mar 5 07:02:24 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 07:02:24 -0800 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <20080305151247.GJ30723@sashak.voltaire.com> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47CB2870.50602@mellanox.co.il> <20080305110134.GB27256@sashak.voltaire.com> <1204727854.6469.576.camel@hrosenstock-ws.xsigo.com> <20080305151247.GJ30723@sashak.voltaire.com> Message-ID: <1204729344.6469.595.camel@hrosenstock-ws.xsigo.com> Hi Sasha, On Wed, 2008-03-05 at 15:12 +0000, Sasha Khapyorsky wrote: > Hi Hal, > > On 06:37 Wed 05 Mar , Hal Rosenstock wrote: > > On Wed, 2008-03-05 at 11:01 +0000, Sasha Khapyorsky wrote: > > > On 00:21 Mon 03 Mar , Yevgeny Kliteynik wrote: > > > > > > > > BTW, are you aware of any other IBA 1.2.1 - related issues that need to be > > > > fixed? > > > > I mean, is OpenSM fully IBA 1.2.1 compliant? > > > > > At least it is a target, we have "Unsupported IB Compliance Statements" > > > list in release notes. Of course there could be more issues. > > > > Not sure what you mean exactly by fully 1.2.1 compliant. > > > > There are two aspects of 1.2.1 changes: > > 1. Subtle changes in IBA 1.2.1 behavior like this one > > 2. Optional features added > > > > I don't think the spec changes have been compared to OpenSM, at least > > not as far as I am aware. > > I'm not aware too. Do you know how we could prepare a list of 1.2 -> > 1.2.1 spec changes? Look at the change bars. -- Hal > Sasha From kliteyn at mellanox.co.il Wed Mar 5 07:06:56 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 05 Mar 2008 17:06:56 +0200 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <20080305151247.GJ30723@sashak.voltaire.com> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47CB2870.50602@mellanox.co.il> <20080305110134.GB27256@sashak.voltaire.com> <1204727854.6469.576.camel@hrosenstock-ws.xsigo.com> <20080305151247.GJ30723@sashak.voltaire.com> Message-ID: <47CEB710.4070001@mellanox.co.il> Sasha Khapyorsky wrote: > Hi Hal, > > On 06:37 Wed 05 Mar , Hal Rosenstock wrote: > >> On Wed, 2008-03-05 at 11:01 +0000, Sasha Khapyorsky wrote: >> >>> On 00:21 Mon 03 Mar , Yevgeny Kliteynik wrote: >>> >>>> BTW, are you aware of any other IBA 1.2.1 - related issues that need to be >>>> fixed? >>>> I mean, is OpenSM fully IBA 1.2.1 compliant? >>>> >>> At least it is a target, we have "Unsupported IB Compliance Statements" >>> list in release notes. Of course there could be more issues. >>> >> Not sure what you mean exactly by fully 1.2.1 compliant. >> >> There are two aspects of 1.2.1 changes: >> 1. Subtle changes in IBA 1.2.1 behavior like this one >> 2. Optional features added >> >> I don't think the spec changes have been compared to OpenSM, at least >> not as far as I am aware. >> > > I'm not aware too. Do you know how we could prepare a list of 1.2 -> > 1.2.1 spec changes? > Actually this is what I meant in my question. I'm not aware of such list (other than the change bars in the pdf), so I was just wondering whether there are any changes that need to be taken care of in OpenSM. -- Yevgeny > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From hrosenstock at xsigo.com Wed Mar 5 07:14:39 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 07:14:39 -0800 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <47CEB710.4070001@mellanox.co.il> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47CB2870.50602@mellanox.co.il> <20080305110134.GB27256@sashak.voltaire.com> <1204727854.6469.576.camel@hrosenstock-ws.xsigo.com> <20080305151247.GJ30723@sashak.voltaire.com> <47CEB710.4070001@mellanox.co.il> Message-ID: <1204730079.6469.598.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-05 at 17:06 +0200, Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: > > Hi Hal, > > > > On 06:37 Wed 05 Mar , Hal Rosenstock wrote: > > > >> On Wed, 2008-03-05 at 11:01 +0000, Sasha Khapyorsky wrote: > >> > >>> On 00:21 Mon 03 Mar , Yevgeny Kliteynik wrote: > >>> > >>>> BTW, are you aware of any other IBA 1.2.1 - related issues that need to be > >>>> fixed? > >>>> I mean, is OpenSM fully IBA 1.2.1 compliant? > >>>> > >>> At least it is a target, we have "Unsupported IB Compliance Statements" > >>> list in release notes. Of course there could be more issues. > >>> > >> Not sure what you mean exactly by fully 1.2.1 compliant. > >> > >> There are two aspects of 1.2.1 changes: > >> 1. Subtle changes in IBA 1.2.1 behavior like this one > >> 2. Optional features added > >> > >> I don't think the spec changes have been compared to OpenSM, at least > >> not as far as I am aware. > >> > > > > I'm not aware too. Do you know how we could prepare a list of 1.2 -> > > 1.2.1 spec changes? > > > > Actually this is what I meant in my question. > I'm not aware of such list (other than the change bars in the pdf), The other way to do this is the resolved issues in the spreadsheet from the IBTA MgtWG for 1.2.1. -- Hal > so I was just wondering whether there are any changes that need > to be taken care of in OpenSM. > > -- Yevgeny > > > Sasha > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > From hrosenstock at xsigo.com Wed Mar 5 07:14:39 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 07:14:39 -0800 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <47CEB710.4070001@mellanox.co.il> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47CB2870.50602@mellanox.co.il> <20080305110134.GB27256@sashak.voltaire.com> <1204727854.6469.576.camel@hrosenstock-ws.xsigo.com> <20080305151247.GJ30723@sashak.voltaire.com> <47CEB710.4070001@mellanox.co.il> Message-ID: <1204730079.6469.598.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-05 at 17:06 +0200, Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: > > Hi Hal, > > > > On 06:37 Wed 05 Mar , Hal Rosenstock wrote: > > > >> On Wed, 2008-03-05 at 11:01 +0000, Sasha Khapyorsky wrote: > >> > >>> On 00:21 Mon 03 Mar , Yevgeny Kliteynik wrote: > >>> > >>>> BTW, are you aware of any other IBA 1.2.1 - related issues that need to be > >>>> fixed? > >>>> I mean, is OpenSM fully IBA 1.2.1 compliant? > >>>> > >>> At least it is a target, we have "Unsupported IB Compliance Statements" > >>> list in release notes. Of course there could be more issues. > >>> > >> Not sure what you mean exactly by fully 1.2.1 compliant. > >> > >> There are two aspects of 1.2.1 changes: > >> 1. Subtle changes in IBA 1.2.1 behavior like this one > >> 2. Optional features added > >> > >> I don't think the spec changes have been compared to OpenSM, at least > >> not as far as I am aware. > >> > > > > I'm not aware too. Do you know how we could prepare a list of 1.2 -> > > 1.2.1 spec changes? > > > > Actually this is what I meant in my question. > I'm not aware of such list (other than the change bars in the pdf), The other way to do this is the resolved issues in the spreadsheet from the IBTA MgtWG for 1.2.1. -- Hal > so I was just wondering whether there are any changes that need > to be taken care of in OpenSM. > > -- Yevgeny > > > Sasha > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > From hrosenstock at xsigo.com Wed Mar 5 07:14:39 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 07:14:39 -0800 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <47CEB710.4070001@mellanox.co.il> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47CB2870.50602@mellanox.co.il> <20080305110134.GB27256@sashak.voltaire.com> <1204727854.6469.576.camel@hrosenstock-ws.xsigo.com> <20080305151247.GJ30723@sashak.voltaire.com> <47CEB710.4070001@mellanox.co.il> Message-ID: <1204730079.6469.598.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-05 at 17:06 +0200, Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: > > Hi Hal, > > > > On 06:37 Wed 05 Mar , Hal Rosenstock wrote: > > > >> On Wed, 2008-03-05 at 11:01 +0000, Sasha Khapyorsky wrote: > >> > >>> On 00:21 Mon 03 Mar , Yevgeny Kliteynik wrote: > >>> > >>>> BTW, are you aware of any other IBA 1.2.1 - related issues that need to be > >>>> fixed? > >>>> I mean, is OpenSM fully IBA 1.2.1 compliant? > >>>> > >>> At least it is a target, we have "Unsupported IB Compliance Statements" > >>> list in release notes. Of course there could be more issues. > >>> > >> Not sure what you mean exactly by fully 1.2.1 compliant. > >> > >> There are two aspects of 1.2.1 changes: > >> 1. Subtle changes in IBA 1.2.1 behavior like this one > >> 2. Optional features added > >> > >> I don't think the spec changes have been compared to OpenSM, at least > >> not as far as I am aware. > >> > > > > I'm not aware too. Do you know how we could prepare a list of 1.2 -> > > 1.2.1 spec changes? > > > > Actually this is what I meant in my question. > I'm not aware of such list (other than the change bars in the pdf), The other way to do this is the resolved issues in the spreadsheet from the IBTA MgtWG for 1.2.1. -- Hal > so I was just wondering whether there are any changes that need > to be taken care of in OpenSM. > > -- Yevgeny > > > Sasha > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > From kliteyn at mellanox.co.il Wed Mar 5 07:45:22 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 05 Mar 2008 17:45:22 +0200 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <20080305111045.GC27256@sashak.voltaire.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> Message-ID: <47CEC012.5040203@mellanox.co.il> Sasha Khapyorsky wrote: > Hi Hal, > > On 10:22 Tue 04 Mar , Hal Rosenstock wrote: > >> As I recall, the QoS manager (at OFED 1.3) is experimental. If so, in >> the release notes, QoS manager should be indicated as experimental in >> the release notes. >> > > Yes, it seems correct. > > >> Additionally, there should be some statement of what >> HCAs and firmware versions this works with (and any switch firmware >> dependencies if any). >> > > Would be nice to have. > > Yevgeny, could you provide such info? > I'm not aware of any switch FW dependency. As for HCAs - I'll collect the relevant info and update the list and the doc. >> It also brings on new tools requirements for building (versions of yacc >> and lex) which should also be indicated there (and perhaps checked >> somehow in the build process). >> >> Can this get added/updated to the release notes ? >> > > The only technical problem I see with this is that 1.3 was released > already. We could update release notes in the master - better than > nothing. > We can also update the release notes in the ofed_1_3 branch, and then it would be included in OFED 1.3.1 that will be out at some point. >> Also, it would also be nice if there were a way to not include this in >> the OpenSM build if not wanted (an --enable-qos-annex configure option >> or the like). Is that a straightforward change ? >> > > Seems fine for me. And I think it was discussed already, but somehow > forgotten :( . We can keep this enabled by default. > I'll work to add a config option for this, something like --without-qos-annex. Will it go into ofed_1_3 ? -- Yevgeny > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From hrosenstock at xsigo.com Wed Mar 5 07:56:22 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 07:56:22 -0800 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <47CEC012.5040203@mellanox.co.il> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> <47CEC012.5040203@mellanox.co.il> Message-ID: <1204732582.6469.604.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-05 at 17:45 +0200, Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: > > Hi Hal, > > > > On 10:22 Tue 04 Mar , Hal Rosenstock wrote: > > > >> As I recall, the QoS manager (at OFED 1.3) is experimental. If so, in > >> the release notes, QoS manager should be indicated as experimental in > >> the release notes. > >> > > > > Yes, it seems correct. > > > > > >> Additionally, there should be some statement of what > >> HCAs and firmware versions this works with (and any switch firmware > >> dependencies if any). > > Would be nice to have. > > > > Yevgeny, could you provide such info? > > > > I'm not aware of any switch FW dependency. Does it work for Anafa (as well as Anafa-II (IS-3)) ? > As for HCAs - I'll collect the relevant info and update > the list and the doc. Thanks. > >> It also brings on new tools requirements for building (versions of yacc > >> and lex) which should also be indicated there (and perhaps checked > >> somehow in the build process). > >> > >> Can this get added/updated to the release notes ? > >> > > > > The only technical problem I see with this is that 1.3 was released > > already. We could update release notes in the master - better than > > nothing. > > > > We can also update the release notes in the ofed_1_3 branch, > and then it would be included in OFED 1.3.1 that will be out > at some point. > > >> Also, it would also be nice if there were a way to not include this in > >> the OpenSM build if not wanted (an --enable-qos-annex configure option > >> or the like). Is that a straightforward change ? > >> > > > > Seems fine for me. And I think it was discussed already, but somehow > > forgotten :( . We can keep this enabled by default. > > > > I'll work to add a config option for this, something > like --without-qos-annex. Sounds good to me; thanks. -- Hal > Will it go into ofed_1_3 ? > > -- Yevgeny > > > Sasha > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > From sashak at voltaire.com Wed Mar 5 08:10:07 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 16:10:07 +0000 Subject: [ofa-general] [ANNOUNCE] opensm-3.1.10 tarball release Message-ID: <20080305161007.GN30723@sashak.voltaire.com> Hi, There is a release of the OpenSM management tarball 3.1.10 as was released with OFED-1.3 available in: http://www.openfabrics.org/downloads/management/ md5sum: 33aa446970c8234cadd0ea20d5ad4a3a opensm-3.1.10.tar.gz Sasha From sashak at voltaire.com Wed Mar 5 08:14:31 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 16:14:31 +0000 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <47CEC012.5040203@mellanox.co.il> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> <47CEC012.5040203@mellanox.co.il> Message-ID: <20080305161431.GO30723@sashak.voltaire.com> On 17:45 Wed 05 Mar , Yevgeny Kliteynik wrote: > > I'm not aware of any switch FW dependency. > As for HCAs - I'll collect the relevant info and update > the list and the doc. Thanks. > We can also update the release notes in the ofed_1_3 branch, > and then it would be included in OFED 1.3.1 that will be out > at some point. Sure. > I'll work to add a config option for this, something > like --without-qos-annex. Or '--disable-qos-annex' (similar to PerfMgr). > Will it go into ofed_1_3 ? Personally I don't think it is necessary, but if somebody wants it in OFED-1.3.x it could be done there too. Sasha From hrosenstock at xsigo.com Wed Mar 5 08:06:57 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 05 Mar 2008 08:06:57 -0800 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <20080305161431.GO30723@sashak.voltaire.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> <47CEC012.5040203@mellanox.co.il> <20080305161431.GO30723@sashak.voltaire.com> Message-ID: <1204733217.6469.606.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-05 at 16:14 +0000, Sasha Khapyorsky wrote: > On 17:45 Wed 05 Mar , Yevgeny Kliteynik wrote: > > > > I'm not aware of any switch FW dependency. > > As for HCAs - I'll collect the relevant info and update > > the list and the doc. > > Thanks. > > > We can also update the release notes in the ofed_1_3 branch, > > and then it would be included in OFED 1.3.1 that will be out > > at some point. > > Sure. > > > I'll work to add a config option for this, something > > like --without-qos-annex. > > Or '--disable-qos-annex' (similar to PerfMgr). > > > Will it go into ofed_1_3 ? > > Personally I don't think it is necessary, but if somebody wants it in > OFED-1.3.x it could be done there too. I would like it there (at least until a solution for bug 932 is found). -- Hal > Sasha From sashak at voltaire.com Wed Mar 5 08:27:29 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 16:27:29 +0000 Subject: [ofa-general] [PATCH] opensm: release notes update In-Reply-To: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080305162729.GP30723@sashak.voltaire.com> On 10:22 Tue 04 Mar , Hal Rosenstock wrote: > > As I recall, the QoS manager (at OFED 1.3) is experimental. If so, in > the release notes, QoS manager should be indicated as experimental in > the release notes. Additionally, there should be some statement of what > HCAs and firmware versions this works with (and any switch firmware > dependencies if any). > > It also brings on new tools requirements for building (versions of yacc > and lex) which should also be indicated there (and perhaps checked > somehow in the build process). > > Can this get added/updated to the release notes ? State experimental status of QoS manager, add flex and bison dependencies. Signed-off-by: Sasha Khapyorsky --- opensm/doc/opensm_release_notes-3.1.10.txt | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/opensm/doc/opensm_release_notes-3.1.10.txt b/opensm/doc/opensm_release_notes-3.1.10.txt index d92c58d..825d758 100644 --- a/opensm/doc/opensm_release_notes-3.1.10.txt +++ b/opensm/doc/opensm_release_notes-3.1.10.txt @@ -24,7 +24,7 @@ This document includes the following sections: 1.1 Major New Features -* QoS manager +* QoS manager (experimental) This QoS manager implementation is in accordance with IBA QoS Annex. Highly configurable QoS Policy is parsed from OpenSM QoS policy file. Valid QoS parameters will be reported in SA PathRecord and @@ -128,6 +128,9 @@ OFED 1.0, OpenIB gen2 (e.g. IBG2 distribution), OpenIB gen1 (e.g. IBGD distribution), or Mellanox VAPI stacks. The qualified driver versions are provided in Table 2, "Qualified IB Stacks". +Also building of QoS manager policy file parser requires flex and bison +installed. + 1.5 Supported Devices Firmware The main task of OpenSM is to initialize InfiniBand devices. The -- 1.5.4.rc2.60.gb2e62 From chu11 at llnl.gov Wed Mar 5 09:10:08 2008 From: chu11 at llnl.gov (Al Chu) Date: Wed, 05 Mar 2008 09:10:08 -0800 Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <20080305104353.GA27256@sashak.voltaire.com> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> <20080301161023.GM1485@sashak.voltaire.com> <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> <20080305104353.GA27256@sashak.voltaire.com> Message-ID: <1204737008.761.121.camel@cardanus.llnl.gov> On Wed, 2008-03-05 at 10:43 +0000, Sasha Khapyorsky wrote: > Hi Al, > > On 07:46 Sun 02 Mar , Albert Chu wrote: > > > > In order to make things work, I also had to add this patch. Seems like a > > corner case that needs to be handled since we never fall into > > __osm_pi_rcv_process_switch_port(). > > Hmm, it is strange. After this light sweep cycle OpenSM should continue > with heavy sweep where __osm_pi_rcv_process_switch_port() should be > reissued. Do you see any errors during discovery? I can't restart opensm on that cluster at this time. I don't recall any port errors. However, I do recall seeing this output from __osm_state_mgr_light_sweep_start(): OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0108: " "Unknown remote side for node 0x%016" PRIx64 "(%s) port %u. Adding to light sweep sampling list\n", cl_ntoh64(osm_node_get_node_guid (p_node)), p_node->print_desc, port_num); leading to a call to __osm_state_mgr_get_remote_port_info(), leading to what I fixed in osm_pi_rcv_process(). My original assumption was that the remote side for some ports wasn't known b/c the remote side ports were down. Is it possible for opensm to not know about a remote side even if that remote side port is up/active? Al > Sasha -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From sashak at voltaire.com Wed Mar 5 10:22:12 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 5 Mar 2008 18:22:12 +0000 Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <1204737008.761.121.camel@cardanus.llnl.gov> References: <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> <20080301161023.GM1485@sashak.voltaire.com> <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> <20080305104353.GA27256@sashak.voltaire.com> <1204737008.761.121.camel@cardanus.llnl.gov> Message-ID: <20080305182212.GR30723@sashak.voltaire.com> On 09:10 Wed 05 Mar , Al Chu wrote: > > I can't restart opensm on that cluster at this time. I don't recall any > port errors. However, I do recall seeing this output from > __osm_state_mgr_light_sweep_start(): > > OSM_LOG(sm->p_log, OSM_LOG_ERROR, > "ERR 0108: " > "Unknown remote side for node 0x%016" > PRIx64 > "(%s) port %u. Adding to light sweep sampling list\n", > cl_ntoh64(osm_node_get_node_guid > (p_node)), > p_node->print_desc, port_num); > > leading to a call to __osm_state_mgr_get_remote_port_info(), leading to > what I fixed in osm_pi_rcv_process(). Yes, this is valid (handled) scenario. What I cannot understand is why it doesn't reach __osm_pi_rcv_process_switch_port() (where ignore_existing_lfts flag should be enforced in accordance with port state) after querying port with "unknown" remotes during a light sweep. I did some experiments with ibsim and still not be able to reproduce this. I'm afraid there could be some hidden bug which I'm not able to catch yet. > My original assumption was that the remote side for some ports wasn't > known b/c the remote side ports were down. Is it possible for opensm to > not know about a remote side even if that remote side port is up/active? I think yes, some ports could be DOWN during initial discovery and become INIT later during LID assignment and/or link state setup. Normally (as in your scenario) next light sweep catches this and enforce heavy sweep. Sasha From clameter at sgi.com Wed Mar 5 10:48:24 2008 From: clameter at sgi.com (Christoph Lameter) Date: Wed, 5 Mar 2008 10:48:24 -0800 (PST) Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: <20080305003722.GG1510@wotan.suse.de> References: <20080219231157.GC18912@wotan.suse.de> <20080220010941.GR7128@v2.random> <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080305003722.GG1510@wotan.suse.de> Message-ID: On Wed, 5 Mar 2008, Nick Piggin wrote: > Um, it's bound to the *Linux page tables*, yes. And I have no idea why > you would use the paravirt ops for this. paravirt ops allows interception of page table operations? From chu11 at llnl.gov Wed Mar 5 11:20:12 2008 From: chu11 at llnl.gov (Al Chu) Date: Wed, 05 Mar 2008 11:20:12 -0800 Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <20080305182212.GR30723@sashak.voltaire.com> References: <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> <20080301161023.GM1485@sashak.voltaire.com> <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> <20080305104353.GA27256@sashak.voltaire.com> <1204737008.761.121.camel@cardanus.llnl.gov> <20080305182212.GR30723@sashak.voltaire.com> Message-ID: <1204744812.761.146.camel@cardanus.llnl.gov> On Wed, 2008-03-05 at 18:22 +0000, Sasha Khapyorsky wrote: > On 09:10 Wed 05 Mar , Al Chu wrote: > > > > I can't restart opensm on that cluster at this time. I don't recall any > > port errors. However, I do recall seeing this output from > > __osm_state_mgr_light_sweep_start(): > > > > OSM_LOG(sm->p_log, OSM_LOG_ERROR, > > "ERR 0108: " > > "Unknown remote side for node 0x%016" > > PRIx64 > > "(%s) port %u. Adding to light sweep sampling list\n", > > cl_ntoh64(osm_node_get_node_guid > > (p_node)), > > p_node->print_desc, port_num); > > > > leading to a call to __osm_state_mgr_get_remote_port_info(), leading to > > what I fixed in osm_pi_rcv_process(). > > Yes, this is valid (handled) scenario. > > What I cannot understand is why it doesn't reach > __osm_pi_rcv_process_switch_port() (where ignore_existing_lfts flag > should be enforced in accordance with port state) after querying port > with "unknown" remotes during a light sweep. > > I did some experiments with ibsim and still not be able to reproduce > this. I'm afraid there could be some hidden bug which I'm not able to > catch yet. > > > My original assumption was that the remote side for some ports wasn't > > known b/c the remote side ports were down. Is it possible for opensm to > > not know about a remote side even if that remote side port is up/active? > > I think yes, some ports could be DOWN during initial discovery and become > INIT later during LID assignment and/or link state setup. Normally (as in > your scenario) next light sweep catches this and enforce heavy sweep. Perhaps it does "reach __osm_pi_rcv_process_switch_port", but the need_update flag is just not set? Is it possible for those remote side ports to be at ARMED or ACTIVE before the 2nd heavy sweep? If so, then that remote side port would have their need_update flag cleared, and thus ignore_existing_lfts wouldn't be set in __osm_pi_rcv_process_switch_port(). Al > Sasha -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From a-alexsl at adavisglobal.com Thu Mar 6 11:24:29 2008 From: a-alexsl at adavisglobal.com (Liz Ragland) Date: Wed, 6 Mar 2008 20:24:29 +0100 Subject: [ofa-general] You told me that you will reply back Message-ID: <01c87fc8$14403c80$d94c0359@a-alexsl> Hello! I am bored this evening. I am nice girl that would like to chat with you. Email me at Inga at GreatGolovaSite.com only, because I am using my friend's email to write this. I will show you some great pictures of me. From weiny2 at llnl.gov Wed Mar 5 13:37:53 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 5 Mar 2008 13:37:53 -0800 Subject: [ofa-general] OpenSM Management BOF at Sonoma Message-ID: <20080305133753.1334f34c.weiny2@llnl.gov> I would like to set up an OpenSM/Management BOF at Sonoma. I have spoken with Sasha, Hal, and Yevgeny; they like the idea. Who would be interested in taking some time, say 1 hour, to attending a BOF? The idea is 3 fold, A) A chance to meet each other and users face to face. B) people can share their use and experience with OpenSM. C) Ideas for the future. Thanks, Ira Weiny Comp Sci/Math Prog. LLNL From clameter at sgi.com Wed Mar 5 16:22:11 2008 From: clameter at sgi.com (Christoph Lameter) Date: Wed, 5 Mar 2008 16:22:11 -0800 (PST) Subject: [ofa-general] Notifier for Externally Mapped Memory (EMM) V1 Message-ID: This patch implements a simple callback for device drivers that establish their own references to pages (KVM, GRU, XPmem, RDMA/Infiniband, DMA engines etc). These references are unknown to the VM (therefore external). With these callbacks it is possible for the device driver to release external references when the VM requests it. This enables swapping, page migration and allows support of remapping, permission changes etc etc for externally mapped memory. With this functionality it becomes also possible to avoid pinning or mlocking pages (commonly done to stop the VM from unmapping device mapped pages). A device driver must subscribe to a process using emm_register_notifier(struct emm_notifier *, struct mm_struct *) The VM will then perform callbacks for operations that unmap or change permissions of pages in that address space. When the process terminates the callback function is called with emm_release. Callbacks are performed before and after the unmapping action of the VM. emm_invalidate_start before emm_invalidate_end after The device driver must hold off establishing new references to pages in the range specified between a callback with emm_invalidate_start and the subsequent call with emm_invalidate_end set. This allows the VM to ensure that no concurrent driver actions are performed on an address range while performing remapping or unmapping operations. Callbacks are mostly performed in a non atomic context. However, in various places spinlocks are held to traverse rmaps. So this patch here is only useful for those devices that can remove mappings in an atomic context (f.e. KVM/GRU). If the rmap spinlocks are converted to semaphores then all callbacks will be performed in a nonatomic context. No additional changes will be necessary to this patchset. Signed-off-by: Christoph Lameter --- include/linux/mm_types.h | 3 + include/linux/rmap.h | 50 ++++++++++++++++++++++++++++++ kernel/fork.c | 3 + mm/Kconfig | 5 +++ mm/filemap_xip.c | 4 ++ mm/fremap.c | 2 + mm/hugetlb.c | 3 + mm/memory.c | 42 ++++++++++++++++++++----- mm/mmap.c | 3 + mm/mprotect.c | 3 + mm/mremap.c | 4 ++ mm/rmap.c | 78 +++++++++++++++++++++++++++++++++++++++++++++-- 12 files changed, 190 insertions(+), 10 deletions(-) Index: linux-2.6.25-rc3-mm1/include/linux/mm_types.h =================================================================== --- linux-2.6.25-rc3-mm1.orig/include/linux/mm_types.h 2008-03-04 11:28:59.852322222 -0800 +++ linux-2.6.25-rc3-mm1/include/linux/mm_types.h 2008-03-05 15:59:26.872868541 -0800 @@ -236,6 +236,9 @@ struct mm_struct { struct file *exe_file; unsigned long num_exe_file_vmas; #endif +#ifdef CONFIG_EMM_NOTIFIER + struct emm_notifier *emm_notifier; +#endif }; #endif /* _LINUX_MM_TYPES_H */ Index: linux-2.6.25-rc3-mm1/mm/Kconfig =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/Kconfig 2008-02-24 13:25:54.000000000 -0800 +++ linux-2.6.25-rc3-mm1/mm/Kconfig 2008-03-05 15:58:51.246302285 -0800 @@ -193,3 +193,8 @@ config NR_QUICK config VIRT_TO_BUS def_bool y depends on !ARCH_NO_VIRT_TO_BUS + +config EMM_NOTIFIER + def_bool n + bool "External Mapped Memory Notifier for drivers directly mapping memory" + Index: linux-2.6.25-rc3-mm1/include/linux/rmap.h =================================================================== --- linux-2.6.25-rc3-mm1.orig/include/linux/rmap.h 2008-02-24 13:25:54.000000000 -0800 +++ linux-2.6.25-rc3-mm1/include/linux/rmap.h 2008-03-05 15:58:51.246302285 -0800 @@ -85,6 +85,56 @@ static inline void page_dup_rmap(struct #endif /* + * Notifier for devices establishing their own references to Linux + * kernel pages in addition to the regular mapping via page + * table and rmap. The notifier allows the device to drop the mapping + * when the VM removes references to pages. + */ +enum emm_operation { + emm_release, /* Process existing, */ + emm_invalidate_start, /* Before the VM unmaps pages */ + emm_invalidate_end, /* After the VM unmapped pages */ + emm_referenced /* Check if a range was referenced */ +}; + +struct emm_notifier { + int (*callback)(struct emm_notifier *e, struct mm_struct *mm, + enum emm_operation op, + unsigned long start, unsigned long end); + struct emm_notifier *next; +}; + +extern int __emm_notify(struct mm_struct *mm, enum emm_operation op, + unsigned long start, unsigned long end); + +/* + * Callback to the device driver for an externally memory mapped section + * of memory. + * + * start Address of first byte of the range + * end Address of first byte after range. + */ +static inline int emm_notify(struct mm_struct *mm, enum emm_operation op, + unsigned long start, unsigned long end) +{ +#ifdef CONFIG_EMM_NOTIFIER + if (unlikely(mm->emm_notifier)) + return __emm_notify(mm, op, start, end); +#endif + return 0; +} + +/* + * Register a notifier with an mm struct. Release occurs when the process + * terminates by calling the notifier function with emm_release. + * + * Must hold the mmap_sem for write. + */ +extern void emm_notifier_register(struct emm_notifier *e, + struct mm_struct *mm); + + +/* * Called from mm/vmscan.c to handle paging out */ int page_referenced(struct page *, int is_locked, struct mem_cgroup *cnt); Index: linux-2.6.25-rc3-mm1/mm/rmap.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/rmap.c 2008-03-04 11:29:00.608332241 -0800 +++ linux-2.6.25-rc3-mm1/mm/rmap.c 2008-03-05 16:19:49.863651354 -0800 @@ -275,6 +275,66 @@ pte_t *page_check_address(struct page *p return NULL; } +#ifdef CONFIG_EMM_NOTIFIER +/* + * Notifier for devices establishing their own references to Linux + * kernel pages in addition to the regular mapping via page + * table and rmap. The notifier allows the device to drop the mapping + * when the VM removes references to pages. + */ + +/* + * This function is only called when a single process remains that performs + * teardown when the last process is exiting. + */ +void emm_notifier_release(struct mm_struct *mm) +{ + struct emm_notifier *e; + + while (mm->emm_notifier) { + e = mm->emm_notifier; + mm->emm_notifier = e->next; + e->callback(e, mm, emm_release, 0, 0); + } +} + +/* Register a notifier */ +void emm_notifier_register(struct emm_notifier *e, struct mm_struct *mm) +{ + e->next = mm->emm_notifier; + /* + * The update to emm_notifier must be visible + * before the pointer becomes visible. + */ + smp_wmb(); + mm->emm_notifier = e; +} +EXPORT_SYMBOL_GPL(emm_notifier_register); + +/* Perform a callback */ +int __emm_notify(struct mm_struct *mm, enum emm_operation op, + unsigned long start, unsigned long end) +{ + struct emm_notifier *e = mm->emm_notifier; + int x; + + while (e) { + /* + * emm_notifier contents must be fetched after + * the retrival of the pointer to the notifier. + */ + smp_rmb(); + if (e->callback) { + x = e->callback(e, mm, op, start, end); + if (x) + return x; + } + e = e->next; + } + return 0; +} +EXPORT_SYMBOL_GPL(__emm_notify); +#endif /* * Subfunctions of page_referenced: page_referenced_one called * repeatedly from either page_referenced_anon or page_referenced_file. @@ -310,6 +370,9 @@ static int page_referenced_one(struct pa (*mapcount)--; pte_unmap_unlock(pte, ptl); + + if (emm_notify(mm, emm_referenced, address, address + PAGE_SIZE)) + referenced++; out: return referenced; } @@ -461,9 +524,10 @@ static int page_mkclean_one(struct page if (address == -EFAULT) goto out; + emm_notify(mm, emm_invalidate_start, address, address + PAGE_SIZE); pte = page_check_address(page, mm, address, &ptl); if (!pte) - goto out; + goto out_notifier; if (pte_dirty(*pte) || pte_write(*pte)) { pte_t entry; @@ -477,6 +541,9 @@ static int page_mkclean_one(struct page } pte_unmap_unlock(pte, ptl); + +out_notifier: + emm_notify(mm, emm_invalidate_end, address, address + PAGE_SIZE); out: return ret; } @@ -716,9 +783,10 @@ static int try_to_unmap_one(struct page if (address == -EFAULT) goto out; + emm_notify(mm, emm_invalidate_start, address, address + PAGE_SIZE); pte = page_check_address(page, mm, address, &ptl); if (!pte) - goto out; + goto out_notify; /* * If the page is mlock()d, we cannot swap it out. @@ -788,6 +856,8 @@ static int try_to_unmap_one(struct page out_unmap: pte_unmap_unlock(pte, ptl); +out_notify: + emm_notify(mm, emm_invalidate_end, address, address + PAGE_SIZE); out: return ret; } @@ -826,6 +896,7 @@ static void try_to_unmap_cluster(unsigne spinlock_t *ptl; struct page *page; unsigned long address; + unsigned long start; unsigned long end; address = (vma->vm_start + cursor) & CLUSTER_MASK; @@ -847,6 +918,8 @@ static void try_to_unmap_cluster(unsigne if (!pmd_present(*pmd)) return; + start = address; + emm_notify(mm, emm_invalidate_start, start, end); pte = pte_offset_map_lock(mm, pmd, address, &ptl); /* Update high watermark before we lower rss */ @@ -879,6 +952,7 @@ static void try_to_unmap_cluster(unsigne (*mapcount)--; } pte_unmap_unlock(pte - 1, ptl); + emm_notify(mm, emm_invalidate_end, start, end); } static int try_to_unmap_anon(struct page *page, int migration) Index: linux-2.6.25-rc3-mm1/kernel/fork.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/kernel/fork.c 2008-03-04 11:29:00.212326992 -0800 +++ linux-2.6.25-rc3-mm1/kernel/fork.c 2008-03-05 15:58:51.246302285 -0800 @@ -362,6 +362,9 @@ static struct mm_struct * mm_init(struct if (likely(!mm_alloc_pgd(mm))) { mm->def_flags = 0; +#ifdef CONFIG_EMM_NOTIFIER + mm->emm_notifier = NULL; +#endif return mm; } Index: linux-2.6.25-rc3-mm1/mm/memory.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/memory.c 2008-03-04 11:29:00.600332133 -0800 +++ linux-2.6.25-rc3-mm1/mm/memory.c 2008-03-05 16:19:07.141370568 -0800 @@ -596,6 +596,7 @@ int copy_page_range(struct mm_struct *ds unsigned long next; unsigned long addr = vma->vm_start; unsigned long end = vma->vm_end; + int ret = 0; /* * Don't copy ptes where a page fault will fill them correctly. @@ -605,12 +606,15 @@ int copy_page_range(struct mm_struct *ds */ if (!(vma->vm_flags & (VM_HUGETLB|VM_NONLINEAR|VM_PFNMAP|VM_INSERTPAGE))) { if (!vma->anon_vma) - return 0; + goto out; } if (is_vm_hugetlb_page(vma)) return copy_hugetlb_page_range(dst_mm, src_mm, vma); + if (is_cow_mapping(vma->vm_flags)) + emm_notify(src_mm, emm_invalidate_start, addr, end); + dst_pgd = pgd_offset(dst_mm, addr); src_pgd = pgd_offset(src_mm, addr); do { @@ -618,10 +622,16 @@ int copy_page_range(struct mm_struct *ds if (pgd_none_or_clear_bad(src_pgd)) continue; if (copy_pud_range(dst_mm, src_mm, dst_pgd, src_pgd, - vma, addr, next)) - return -ENOMEM; + vma, addr, next)) { + ret = -ENOMEM; + break; + } } while (dst_pgd++, src_pgd++, addr = next, addr != end); - return 0; + + if (is_cow_mapping(vma->vm_flags)) + emm_notify(src_mm, emm_invalidate_end, addr, end); +out: + return ret; } static unsigned long zap_pte_range(struct mmu_gather *tlb, @@ -894,12 +904,15 @@ unsigned long zap_page_range(struct vm_a unsigned long end = address + size; unsigned long nr_accounted = 0; + emm_notify(mm, emm_invalidate_start, address, end); lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); + end = unmap_vmas(&tlb, vma, address, end, &nr_accounted, details); if (tlb) tlb_finish_mmu(tlb, address, end); + emm_notify(mm, emm_invalidate_end, address, end); return end; } @@ -1339,6 +1352,7 @@ int remap_pfn_range(struct vm_area_struc pgd_t *pgd; unsigned long next; unsigned long end = addr + PAGE_ALIGN(size); + unsigned long start = addr; struct mm_struct *mm = vma->vm_mm; int err; @@ -1371,6 +1385,7 @@ int remap_pfn_range(struct vm_area_struc BUG_ON(addr >= end); pfn -= addr >> PAGE_SHIFT; pgd = pgd_offset(mm, addr); + emm_notify(mm, emm_invalidate_start, start, end); flush_cache_range(vma, addr, end); do { next = pgd_addr_end(addr, end); @@ -1379,6 +1394,7 @@ int remap_pfn_range(struct vm_area_struc if (err) break; } while (pgd++, addr = next, addr != end); + emm_notify(mm, emm_invalidate_end, start, end); return err; } EXPORT_SYMBOL(remap_pfn_range); @@ -1462,10 +1478,12 @@ int apply_to_page_range(struct mm_struct { pgd_t *pgd; unsigned long next; + unsigned long start = addr; unsigned long end = addr + size; int err; BUG_ON(addr >= end); + emm_notify(mm, emm_invalidate_start, start, end); pgd = pgd_offset(mm, addr); do { next = pgd_addr_end(addr, end); @@ -1473,6 +1491,7 @@ int apply_to_page_range(struct mm_struct if (err) break; } while (pgd++, addr = next, addr != end); + emm_notify(mm, emm_invalidate_end, start, end); return err; } EXPORT_SYMBOL_GPL(apply_to_page_range); @@ -1613,8 +1632,10 @@ static int do_wp_page(struct mm_struct * page_table = pte_offset_map_lock(mm, pmd, address, &ptl); page_cache_release(old_page); - if (!pte_same(*page_table, orig_pte)) - goto unlock; + if (!pte_same(*page_table, orig_pte)) { + pte_unmap_unlock(page_table, ptl); + goto check_dirty; + } page_mkwrite = 1; } @@ -1630,7 +1651,8 @@ static int do_wp_page(struct mm_struct * if (ptep_set_access_flags(vma, address, page_table, entry,1)) update_mmu_cache(vma, address, entry); ret |= VM_FAULT_WRITE; - goto unlock; + pte_unmap_unlock(page_table, ptl); + goto check_dirty; } /* @@ -1652,6 +1674,7 @@ gotten: if (mem_cgroup_charge(new_page, mm, GFP_KERNEL)) goto oom_free_new; + emm_notify(mm, emm_invalidate_start, address, address + PAGE_SIZE); /* * Re-check the pte - we dropped the lock */ @@ -1690,8 +1713,11 @@ gotten: page_cache_release(new_page); if (old_page) page_cache_release(old_page); -unlock: + pte_unmap_unlock(page_table, ptl); + emm_notify(mm, emm_invalidate_end, address, address + PAGE_SIZE); + +check_dirty: if (dirty_page) { if (vma->vm_file) file_update_time(vma->vm_file); Index: linux-2.6.25-rc3-mm1/mm/mmap.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/mmap.c 2008-03-04 11:29:00.604332186 -0800 +++ linux-2.6.25-rc3-mm1/mm/mmap.c 2008-03-05 16:15:36.317854841 -0800 @@ -1751,6 +1751,7 @@ static void unmap_region(struct mm_struc struct mmu_gather *tlb; unsigned long nr_accounted = 0; + emm_notify(mm, emm_invalidate_start, start, end); lru_add_drain(); tlb = tlb_gather_mmu(mm, 0); update_hiwater_rss(mm); @@ -1759,6 +1760,7 @@ static void unmap_region(struct mm_struc free_pgtables(&tlb, vma, prev? prev->vm_end: FIRST_USER_ADDRESS, next? next->vm_start: 0); tlb_finish_mmu(tlb, start, end); + emm_notify(mm, emm_invalidate_end, start, end); } /* @@ -2048,6 +2050,7 @@ void exit_mmap(struct mm_struct *mm) /* mm's last user has gone, and its about to be pulled down */ arch_exit_mmap(mm); + emm_notify(mm, emm_release, 0, TASK_SIZE); lru_add_drain(); flush_cache_mm(mm); Index: linux-2.6.25-rc3-mm1/mm/mprotect.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/mprotect.c 2008-02-24 13:25:54.000000000 -0800 +++ linux-2.6.25-rc3-mm1/mm/mprotect.c 2008-03-05 16:15:53.033182172 -0800 @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -198,10 +199,12 @@ success: dirty_accountable = 1; } + emm_notify(mm, emm_invalidate_start, start, end); if (is_vm_hugetlb_page(vma)) hugetlb_change_protection(vma, start, end, vma->vm_page_prot); else change_protection(vma, start, end, vma->vm_page_prot, dirty_accountable); + emm_notify(mm, emm_invalidate_end, start, end); vm_stat_account(mm, oldflags, vma->vm_file, -nrpages); vm_stat_account(mm, newflags, vma->vm_file, nrpages); return 0; Index: linux-2.6.25-rc3-mm1/mm/mremap.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/mremap.c 2008-02-24 13:25:54.000000000 -0800 +++ linux-2.6.25-rc3-mm1/mm/mremap.c 2008-03-05 16:16:12.108414521 -0800 @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -74,7 +75,9 @@ static void move_ptes(struct vm_area_str struct mm_struct *mm = vma->vm_mm; pte_t *old_pte, *new_pte, pte; spinlock_t *old_ptl, *new_ptl; + unsigned long old_start = old_addr; + emm_notify(mm, emm_invalidate_start, old_start, old_end); if (vma->vm_file) { /* * Subtle point from Rajesh Venkatasubramanian: before @@ -116,6 +119,7 @@ static void move_ptes(struct vm_area_str pte_unmap_unlock(old_pte - 1, old_ptl); if (mapping) spin_unlock(&mapping->i_mmap_lock); + emm_notify(mm, emm_invalidate_end, old_start, old_end); } #define LATENCY_LIMIT (64 * PAGE_SIZE) Index: linux-2.6.25-rc3-mm1/mm/filemap_xip.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/filemap_xip.c 2008-03-04 11:29:00.600332133 -0800 +++ linux-2.6.25-rc3-mm1/mm/filemap_xip.c 2008-03-05 16:10:01.431331796 -0800 @@ -190,6 +190,8 @@ __xip_unmap (struct address_space * mapp address = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT); BUG_ON(address < vma->vm_start || address >= vma->vm_end); + emm_notify(mm, emm_invalidate_start, + address, address + PAGE_SIZE); pte = page_check_address(page, mm, address, &ptl); if (pte) { /* Nuke the page table entry. */ @@ -201,6 +203,8 @@ __xip_unmap (struct address_space * mapp pte_unmap_unlock(pte, ptl); page_cache_release(page); } + emm_notify(mm, emm_invalidate_end, + address, address + PAGE_SIZE); } spin_unlock(&mapping->i_mmap_lock); } Index: linux-2.6.25-rc3-mm1/mm/fremap.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/fremap.c 2008-02-24 13:25:54.000000000 -0800 +++ linux-2.6.25-rc3-mm1/mm/fremap.c 2008-03-05 16:10:13.706837775 -0800 @@ -214,7 +214,9 @@ asmlinkage long sys_remap_file_pages(uns spin_unlock(&mapping->i_mmap_lock); } + emm_notify(mm, emm_invalidate_start, start, end); err = populate_range(mm, vma, start, size, pgoff); + emm_notify(mm, emm_invalidate_end, start, end); if (!err && !(flags & MAP_NONBLOCK)) { if (unlikely(has_write_lock)) { downgrade_write(&mm->mmap_sem); Index: linux-2.6.25-rc3-mm1/mm/hugetlb.c =================================================================== --- linux-2.6.25-rc3-mm1.orig/mm/hugetlb.c 2008-03-04 11:29:00.600332133 -0800 +++ linux-2.6.25-rc3-mm1/mm/hugetlb.c 2008-03-05 16:10:27.750272449 -0800 @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -796,6 +797,7 @@ void __unmap_hugepage_range(struct vm_ar BUG_ON(start & ~HPAGE_MASK); BUG_ON(end & ~HPAGE_MASK); + emm_notify(mm, emm_invalidate_start, start, end); spin_lock(&mm->page_table_lock); for (address = start; address < end; address += HPAGE_SIZE) { ptep = huge_pte_offset(mm, address); @@ -816,6 +818,7 @@ void __unmap_hugepage_range(struct vm_ar } spin_unlock(&mm->page_table_lock); flush_tlb_range(vma, start, end); + emm_notify(mm, emm_invalidate_end, start, end); list_for_each_entry_safe(page, tmp, &page_list, lru) { list_del(&page->lru); put_page(page); From weiny2 at llnl.gov Wed Mar 5 17:46:34 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 5 Mar 2008 17:46:34 -0800 Subject: [ofa-general] [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h Message-ID: <20080305174634.75495742.weiny2@llnl.gov> Whilst writing some test code, I came across this issue again. '[openib-general] Conflicting typedefs for "ib_gid_t"' http://lists.openfabrics.org/pipermail/general/2006-August/024903.html One fix which suggested was to change the ib_gid_t to mad_gid_t. http://lists.openfabrics.org/pipermail/general/2006-August/024928.html I think that is a pretty good choice. I know there are many ways to fix this but this seems to be the least invasive. (I was actually surprised by how little the code needed to change.) Of course I don't know if other software relies on the libibmad interface; so if your package uses mad.h, please comment. And finally I have just be running this afternoon with the fix. So there has been limited testing. Thanks, Ira >From df270f4b955c16f25c65a89be074c78fd2f3521d Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Wed, 5 Mar 2008 17:25:33 -0800 Subject: [PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h Signed-off-by: Ira K. Weiny --- infiniband-diags/src/ibaddr.c | 2 +- libibmad/include/infiniband/mad.h | 8 ++++---- libibmad/src/resolve.c | 2 +- libibmad/src/sa.c | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/infiniband-diags/src/ibaddr.c b/infiniband-diags/src/ibaddr.c index f71edf8..2ac6a4f 100644 --- a/infiniband-diags/src/ibaddr.c +++ b/infiniband-diags/src/ibaddr.c @@ -56,7 +56,7 @@ ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) uint8_t nodeinfo[64]; uint64_t guid, prefix; char buf1[64], buf2[64]; - ib_gid_t gid; + mad_gid_t gid; int lmc; if (!smp_query(nodeinfo, portid, IB_ATTR_NODE_INFO, 0, 0)) diff --git a/libibmad/include/infiniband/mad.h b/libibmad/include/infiniband/mad.h index 15b8246..adce3e8 100644 --- a/libibmad/include/infiniband/mad.h +++ b/libibmad/include/infiniband/mad.h @@ -156,7 +156,7 @@ enum GSI_ATTR_ID { #define IB_VENDOR_OPENIB_SYSSTAT_CLASS (IB_VENDOR_RANGE2_START_CLASS + 3) #define IB_OPENIB_OUI (0x001405) -typedef uint8_t ib_gid_t[16]; +typedef uint8_t mad_gid_t[16]; typedef struct { int cnt; @@ -189,7 +189,7 @@ typedef struct portid { int lid; /* lid or 0 if directed route */ ib_dr_path_t drpath; int grh_present; /* flag */ - ib_gid_t gid; + mad_gid_t gid; uint32_t qp; uint32_t qkey; uint8_t sl; @@ -777,7 +777,7 @@ uint8_t * sa_call(void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, unsigned timeout); uint8_t * sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, unsigned timeout); -int ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, +int ib_path_query(mad_gid_t srcgid, mad_gid_t destgid, ib_portid_t *sm_id, void *buf); /* returns lid */ inline static uint8_t * @@ -799,7 +799,7 @@ int ib_resolve_guid(ib_portid_t *portid, uint64_t *guid, ib_portid_t *sm_id, int timeout); int ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_portid_t *sm_id); -int ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid); +int ib_resolve_self(ib_portid_t *portid, int *portnum, mad_gid_t *gid); /* gs.c */ uint8_t *perf_classportinfo_query(void *rcvbuf, ib_portid_t *dest, int port, diff --git a/libibmad/src/resolve.c b/libibmad/src/resolve.c index d8365b2..7d8eb6e 100644 --- a/libibmad/src/resolve.c +++ b/libibmad/src/resolve.c @@ -138,7 +138,7 @@ ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_por } int -ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid) +ib_resolve_self(ib_portid_t *portid, int *portnum, mad_gid_t *gid) { ib_portid_t self = {0}; uint8_t portinfo[64]; diff --git a/libibmad/src/sa.c b/libibmad/src/sa.c index ddbfce3..fde10bd 100644 --- a/libibmad/src/sa.c +++ b/libibmad/src/sa.c @@ -112,7 +112,7 @@ sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, IB_PR_COMPMASK_NUMBPATH) int -ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, void *buf) +ib_path_query(mad_gid_t srcgid, mad_gid_t destgid, ib_portid_t *sm_id, void *buf) { int npath; ib_sa_call_t sa = {0}; -- 1.5.1 From sashak at voltaire.com Wed Mar 5 18:01:12 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 6 Mar 2008 02:01:12 +0000 Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <1204744812.761.146.camel@cardanus.llnl.gov> References: <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> <20080301161023.GM1485@sashak.voltaire.com> <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> <20080305104353.GA27256@sashak.voltaire.com> <1204737008.761.121.camel@cardanus.llnl.gov> <20080305182212.GR30723@sashak.voltaire.com> <1204744812.761.146.camel@cardanus.llnl.gov> Message-ID: <20080306020112.GD28730@sashak.voltaire.com> On 11:20 Wed 05 Mar , Al Chu wrote: > > Perhaps it does "reach __osm_pi_rcv_process_switch_port", but the > need_update flag is just not set? Is it possible for those remote side > ports to be at ARMED or ACTIVE before the 2nd heavy sweep? If so, then > that remote side port would have their need_update flag cleared, and > thus ignore_existing_lfts wouldn't be set in > __osm_pi_rcv_process_switch_port(). Yes, this makes sense. It is possible that port was in DOWN state during discovery, but becomes INIT later yet in 1st heavy sweep before link manager set this to ARMED and ACTIVE. Actually your patch fixes exactly this case - nice catch! So I'm going to apply both patches then. Sasha From sashak at voltaire.com Wed Mar 5 18:11:17 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 6 Mar 2008 02:11:17 +0000 Subject: [ofa-general] Re: [PATCH] opensm: enforce routing paths rebalancing on switch reconnection In-Reply-To: <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> References: <60144.128.15.244.44.1204258639.squirrel@127.0.0.1> <20080229172135.GD1485@sashak.voltaire.com> <50643.128.15.244.112.1204307903.squirrel@127.0.0.1> <20080301014635.GF1485@sashak.voltaire.com> <20080301155203.GK1485@sashak.voltaire.com> <20080301160825.GL1485@sashak.voltaire.com> <20080301161023.GM1485@sashak.voltaire.com> <49430.128.15.244.2.1204391481.squirrel@127.0.0.1> <46002.128.15.244.18.1204472819.squirrel@127.0.0.1> Message-ID: <20080306021117.GE28730@sashak.voltaire.com> On 07:46 Sun 02 Mar , Albert Chu wrote: > Hey Sasha, > > In order to make things work, I also had to add this patch. Seems like a > corner case that needs to be handled since we never fall into > __osm_pi_rcv_process_switch_port(). (BTW, I am working off a 3.1.10 > branch for the test cluster, so this patch is forward ported and > technically untested.) > > --- a/opensm/opensm/osm_port_info_rcv.c > +++ b/opensm/opensm/osm_port_info_rcv.c > @@ -564,6 +564,7 @@ void osm_pi_rcv_process(IN void *context, IN void *data) > ", Commencing heavy sweep\n", > cl_ntoh64(node_guid), cl_ntoh64(port_guid)); > sm->p_subn->force_heavy_sweep = 1; > + sm->p_subn->ignore_existing_lfts = 1; > goto Exit; > } Applied. Thanks. Sasha From npiggin at suse.de Wed Mar 5 18:59:08 2008 From: npiggin at suse.de (Nick Piggin) Date: Thu, 6 Mar 2008 03:59:08 +0100 Subject: [ofa-general] Re: [PATCH] mmu notifiers #v8 In-Reply-To: References: <20080220103942.GU7128@v2.random> <20080221045430.GC15215@wotan.suse.de> <20080221144023.GC9427@v2.random> <20080221161028.GA14220@sgi.com> <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303032934.GA3301@wotan.suse.de> <20080305003722.GG1510@wotan.suse.de> Message-ID: <20080306025908.GC27150@wotan.suse.de> On Wed, Mar 05, 2008 at 10:48:24AM -0800, Christoph Lameter wrote: > On Wed, 5 Mar 2008, Nick Piggin wrote: > > > Um, it's bound to the *Linux page tables*, yes. And I have no idea why > > you would use the paravirt ops for this. > > paravirt ops allows interception of page table operations? Maybe possible but it's totally the wrong API for it. From -rose at accomline.com Wed Mar 5 17:31:05 2008 From: -rose at accomline.com (forest emrys) Date: Thu, 06 Mar 2008 01:31:05 +0000 Subject: [ofa-general] emilio Message-ID: <000701c87f38$015c7683$43fea7be@cnmfba> We can reduce your debt! www.debt666.info After dashing at top speed through a busy day, trying to pack in every last e-mail, phone call, and laundry load, how can anyone expect to jump into bed and immediately nod off? Even when you're relaxed, it typically takes about 20 minutes to fall asleep, says sleep expert James B. Maas. From Thomas.Talpey at netapp.com Wed Mar 5 20:02:10 2008 From: Thomas.Talpey at netapp.com (Talpey, Thomas) Date: Wed, 05 Mar 2008 23:02:10 -0500 Subject: [ofa-general] detecting "recv buffer not available" events In-Reply-To: <1204669319.5109.579.camel@brick.pathscale.com> References: <47CDBE41.1060600@opengridcomputing.com> <1204669319.5109.579.camel@brick.pathscale.com> Message-ID: At 05:21 PM 3/4/2008, Ralph Campbell wrote: >There is no generic counter you can query to find the number >of RNR retires. The best you can do is set the IBV_QP_RNR_RETRY >parameter to zero and see if you get any IBV_WC_RNR_RETRY_EXC_ERR >error completions. If IBV_QP_RNR_RETRY is N, the HCA >will do up to N=0..6 retries for you (or if it is 7, an infinite >number of times). The NFS/RDMA client sets this to 0 when invoking rdma_connect(), yet no failures are seen on IB sends even when the server fails to promptly post receives. So, does the setting mean anything on the receiving end? I.e., no matter what the NFS/RDMA server sets it to, shouldn't we expect the client's sends to fail if they don't land on the first try in a receive buffer there? That's why the client zeroes rnr_retry - the RPC/RDMA protocol manages this so we don't want the transport "helping". Tom. > >On Tue, 2008-03-04 at 15:25 -0600, Steve Wise wrote: >> All, >> >> This question is regarding SENDs arriving when no RECV buffer is posted. >> For IB, somehow the transport can deal with this and force the sender >> transport to retransmit. >> >> First, is my statement above correct? >> >> If so, How can I detect that this is happening on an IB RC? IE is there >> some stat I can query? >> >> Thanks, >> >> Steve. >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >http://openib.org/mailman/listinfo/openib-general > >_______________________________________________ >general mailing list >general at lists.openfabrics.org >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From Thomas.Talpey at netapp.com Wed Mar 5 20:13:14 2008 From: Thomas.Talpey at netapp.com (Talpey, Thomas) Date: Wed, 05 Mar 2008 23:13:14 -0500 Subject: [nfs-rdma-devel] [ofa-general] Status of NFS-RDMA ? (fwd) In-Reply-To: <20080303132038.GH31715@cefeid.wcss.wroc.pl> References: <20080206154701.GA11384@cefeid.wcss.wroc.pl> <1202320491.14810.29.camel@trinity.ogc.int> <20080206213543.GA21176@cefeid.wcss.wroc.pl> <20080209221711.GB20332@cefeid.wcss.wroc.pl> <47C7C261.2060007@hamburgnet.de> <1204505291.1567.15.camel@trinity.ogc.int> <20080303132038.GH31715@cefeid.wcss.wroc.pl> Message-ID: At 08:20 AM 3/3/2008, Pawel Dziekonski wrote: >both nfs server and client have Mellanox MT25204 HCAs. tests were done >by connecting them port to port (without switch) with DDR cable. >reported link was 20Gbps. What kernel base is your client running also? (uname -a) There are some known issues with cached write throughput over NFS above 1Gb, that we may be able to work around but it's kernel-dependent. >currently I have a Flextronix switch that reports itself as "MT47396 >Infiniscale-III Mellanox Technologies". Looking at your results from earlier this month, it's not at all clear that the nfs/rdma run was actually using nfs/rdma. The speeds and cpu loads were very similar to the ethernet results. Can we take this offline and look into it more? All on the client, I'll be interested in the exact mount command you ran, the output of "cat /proc/mounts", the contents of dmesg/kernel logs after a run, and the output of "nfsstat". Tom. From cima6002 at 012.net.il Wed Mar 5 20:58:14 2008 From: cima6002 at 012.net.il (=?windows-1255?Q?=E3=F0=E9?=) Date: Thu, 6 Mar 2008 06:58:14 +0200 Subject: [ofa-general] =?windows-1255?b?7uf0+SD56fjl+ukg8On36eXvIOzu+fjj?= =?windows-1255?q?=3F?= Message-ID: מחפש שירותי ניקיון למשרד? החיפושים שלך הסתיימו ! לFIX שירותי ניקיון צוות קבוע ואיכותי של עובדי ניקיון מנוסים ומסורים, עם התמחות בניקוי משרדים בשיחת טלפון אחת תקבל: * ניקוי משרדים ושטחי ציבור בכל גודל * ניקוי שטיחים * ניקוי רצפות יסודי (פוליש) * ניקוי חלונות (גם בסנפלינג) * אספקת חומרי ניקוי וניירות * שירותי ניקיון על בסיס קבוע/זמני/חד-פעמי 12 שנים, 120 עובדים ומאות לקוחות מרוצים לפרטים נוספים צור קשר 03-7524030 או השב למייל זה מה אומרים הלקוחות... " לעונג לי לכתוב לכם מכתב הערכה ותודה על שירותי הניקיון הניתנים מזה תקופה למשרדינו. משרדנו משתמש בשירותכם שלוש פעמים בשבוע והניקיון והסדר ניכרים בכל פינה במשרד, ומורגשת תחושת האכפתיות והרצון הטוב של המנקים. ברצוני לציין גם את רצונכם הטוב להיענות תמיד לפניותיי ורצונותיי, ובמיוחד את האחראי על האזור, האדון ניסים שדואג ליצור עמי קשר לעתים קרובות ולוודא שהכל תקין. המשיכו בדרך זו ותבוא עליכם הברכה. בכבוד רב ל.א. מזכירת החברה. (חברת מסחר בינלאומי) " כתובתינו: FIX שירותי ניקיון ואחזקה, רח' המעפיל 2 רמת גן להסרה אנא שלח מייל חוזר עם המילה הסר בנושא From sashak at voltaire.com Wed Mar 5 22:13:53 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 6 Mar 2008 06:13:53 +0000 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <20080305174634.75495742.weiny2@llnl.gov> References: <20080305174634.75495742.weiny2@llnl.gov> Message-ID: <20080306061353.GA2442@sashak.voltaire.com> Hi Ira, On 17:46 Wed 05 Mar , Ira Weiny wrote: > Whilst writing some test code, I came across this issue again. > > '[openib-general] Conflicting typedefs for "ib_gid_t"' > http://lists.openfabrics.org/pipermail/general/2006-August/024903.html > > One fix which suggested was to change the ib_gid_t to mad_gid_t. > > http://lists.openfabrics.org/pipermail/general/2006-August/024928.html > > I think that is a pretty good choice. I know there are many ways to fix this > but this seems to be the least invasive. (I was actually surprised by how > little the code needed to change.) Of course I don't know if other software > relies on the libibmad interface; so if your package uses mad.h, please > comment. I'm using ib_gid_t from mad.h in ibsim. Of course it is trivial to change there. The only potential issue I can see with it is sort of incompatibility with released OFED libraries. But this is temporary. Sasha From avi at qumranet.com Wed Mar 5 22:12:01 2008 From: avi at qumranet.com (Avi Kivity) Date: Thu, 06 Mar 2008 08:12:01 +0200 Subject: [ofa-general] Re: [kvm-devel] Notifier for Externally Mapped Memory (EMM) V1 In-Reply-To: References: Message-ID: <47CF8B31.6040001@qumranet.com> Christoph Lameter wrote: > > /* > + * Notifier for devices establishing their own references to Linux > + * kernel pages in addition to the regular mapping via page > + * table and rmap. The notifier allows the device to drop the mapping > + * when the VM removes references to pages. > + */ > +enum emm_operation { > + emm_release, /* Process existing, */ > + emm_invalidate_start, /* Before the VM unmaps pages */ > + emm_invalidate_end, /* After the VM unmapped pages */ > + emm_referenced /* Check if a range was referenced */ > +}; > Check and clear btw, a similar test and clear dirty would be useful as well, no? > + > +struct emm_notifier { > + int (*callback)(struct emm_notifier *e, struct mm_struct *mm, > + enum emm_operation op, > + unsigned long start, unsigned long end); > + struct emm_notifier *next; > +}; > + > It is cleaner for the user to specify individual callbacks instead of having a switch. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. From wedgeeog at docsnuggle.de Wed Mar 5 23:07:57 2008 From: wedgeeog at docsnuggle.de (Aurelio Cisneros) Date: Thu, 6 Mar 2008 15:07:57 +0800 Subject: [ofa-general] ROLEX at unbelievable prices! Message-ID: <638498337.38139203224462@docsnuggle.de> Ex sy qui jb site R ir ep xkr li xl ca W eda at src ch mpg es!Unbelievable Quality At Unbeatable Prices!Buy R re ep adg lic xvj a W zq at pr ch igm es with 25% Discount!ENTER Aurelio Cisneros -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at dev.mellanox.co.il Thu Mar 6 02:43:29 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 06 Mar 2008 12:43:29 +0200 Subject: [ofa-general] [OpenSM]: renaming of opensm init script to opensmd In-Reply-To: <1204666444.761.101.camel@cardanus.llnl.gov> References: <1204666444.761.101.camel@cardanus.llnl.gov> Message-ID: <47CFCAD1.8040206@mellanox.co.il> Al Chu wrote: > Hey Sasha, > > One of the sys-admins was wondering why the script name was changed from > opensm to opensmd (which requires him to go change some internal scripts > he uses). Our local Redhat guy told me in the past rhel/fedora > convention is the following (which is a comment stolen from rpmlint). > > 'incoherent-init-script-name', > '''The init script name should be the same as the package name in lower > case, > or one with 'd' appended if it invokes a process by that name.''', > We have changed it to be compatible with previous OFED releases Also since opensm executable is with the name opensm then having the daemon in the same name was confusing Tziporet From tziporet at dev.mellanox.co.il Thu Mar 6 02:46:55 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 06 Mar 2008 12:46:55 +0200 Subject: [ofa-general] [PATCH] opensm: release notes update In-Reply-To: <20080305162729.GP30723@sashak.voltaire.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305162729.GP30723@sashak.voltaire.com> Message-ID: <47CFCB9F.6070109@mellanox.co.il> Sasha Khapyorsky wrote: > On 10:22 Tue 04 Mar , Hal Rosenstock wrote: > >> As I recall, the QoS manager (at OFED 1.3) is experimental. If so, in >> the release notes, QoS manager should be indicated as experimental in >> the release notes. Additionally, there should be some statement of what >> HCAs and firmware versions this works with (and any switch firmware >> dependencies if any). >> >> It also brings on new tools requirements for building (versions of yacc >> and lex) which should also be indicated there (and perhaps checked >> somehow in the build process). >> >> Can this get added/updated to the release notes ? >> > > Do you want me to update OFED RN too? This will be good for future OFED fix releases Tziporet From vlad at lists.openfabrics.org Thu Mar 6 03:08:03 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Thu, 6 Mar 2008 03:08:03 -0800 (PST) Subject: [ofa-general] ofa_1_3_kernel 20080306-0200 daily build status Message-ID: <20080306110803.3F92FE60B87@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.24 Passed on ia64 with linux-2.6.23 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From pawel.dziekonski at wcss.pl Thu Mar 6 05:41:31 2008 From: pawel.dziekonski at wcss.pl (Pawel Dziekonski) Date: Thu, 6 Mar 2008 14:41:31 +0100 Subject: [nfs-rdma-devel] [ofa-general] Status of NFS-RDMA ? (fwd) In-Reply-To: References: <20080206154701.GA11384@cefeid.wcss.wroc.pl> <1202320491.14810.29.camel@trinity.ogc.int> <20080206213543.GA21176@cefeid.wcss.wroc.pl> <20080209221711.GB20332@cefeid.wcss.wroc.pl> <47C7C261.2060007@hamburgnet.de> <1204505291.1567.15.camel@trinity.ogc.int> <20080303132038.GH31715@cefeid.wcss.wroc.pl> Message-ID: <20080306134131.GB21377@cefeid.wcss.wroc.pl> On Wed, 05 Mar 2008 at 11:13:14PM -0500, Talpey, Thomas wrote: > At 08:20 AM 3/3/2008, Pawel Dziekonski wrote: > >both nfs server and client have Mellanox MT25204 HCAs. tests were > >done by connecting them port to port (without switch) with DDR > >cable. reported link was 20Gbps. > > What kernel base is your client running also? (uname -a) There are > some known issues with cached write throughput over NFS above 1Gb, > that we may be able to work around but it's kernel-dependent. > > >currently I have a Flextronix switch that reports itself as > >"MT47396 Infiniscale-III Mellanox Technologies". > > Looking at your results from earlier this month, it's not at all > clear that the nfs/rdma run was actually using nfs/rdma. The speeds > and cpu loads were very similar to the ethernet results. > > Can we take this offline and look into it more? All on the client, > I'll be interested in the exact mount command you ran, the output of > "cat /proc/mounts", the contents of dmesg/kernel logs after a run, > and the output of "nfsstat". Hi, thanks for answering. I can not provide any details now. I will repeat whole setup and benchmarks ASAP. Pawel -- Pawel Dziekonski Wroclaw Centre for Networking & Supercomputing, HPC Department Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl From chu11 at llnl.gov Thu Mar 6 08:45:23 2008 From: chu11 at llnl.gov (Al Chu) Date: Thu, 06 Mar 2008 08:45:23 -0800 Subject: [ofa-general] [OpenSM]: renaming of opensm init script to opensmd In-Reply-To: <47CFCAD1.8040206@mellanox.co.il> References: <1204666444.761.101.camel@cardanus.llnl.gov> <47CFCAD1.8040206@mellanox.co.il> Message-ID: <1204821923.17718.24.camel@cardanus.llnl.gov> On Thu, 2008-03-06 at 12:43 +0200, Tziporet Koren wrote: > Al Chu wrote: > > Hey Sasha, > > > > One of the sys-admins was wondering why the script name was changed from > > opensm to opensmd (which requires him to go change some internal scripts > > he uses). Our local Redhat guy told me in the past rhel/fedora > > convention is the following (which is a comment stolen from rpmlint). > > > > 'incoherent-init-script-name', > > '''The init script name should be the same as the package name in lower > > case, > > or one with 'd' appended if it invokes a process by that name.''', > > > We have changed it to be compatible with previous OFED releases > Also since opensm executable is with the name opensm then having the > daemon in the same name was confusing Hey Tziporet, If the distros have no issue with it, then that's fine. I'm concerned they will change it b/c of their conventions, leading to different init scripts in different distros. Al > Tziporet -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From hrosenstock at xsigo.com Thu Mar 6 08:58:11 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 06 Mar 2008 08:58:11 -0800 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <20080305174634.75495742.weiny2@llnl.gov> References: <20080305174634.75495742.weiny2@llnl.gov> Message-ID: <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-05 at 17:46 -0800, Ira Weiny wrote: > Whilst writing some test code, I came across this issue again. > > '[openib-general] Conflicting typedefs for "ib_gid_t"' > http://lists.openfabrics.org/pipermail/general/2006-August/024903.html > > One fix which suggested was to change the ib_gid_t to mad_gid_t. > > http://lists.openfabrics.org/pipermail/general/2006-August/024928.html > > I think that is a pretty good choice. I know there are many ways to fix this > but this seems to be the least invasive. (I was actually surprised by how > little the code needed to change.) Of course I don't know if other software > relies on the libibmad interface; so if your package uses mad.h, please > comment. And finally I have just be running this afternoon with the fix. So > there has been limited testing. > > Thanks, > Ira > > > From df270f4b955c16f25c65a89be074c78fd2f3521d Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Wed, 5 Mar 2008 17:25:33 -0800 > Subject: [PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h > > > Signed-off-by: Ira K. Weiny > --- > infiniband-diags/src/ibaddr.c | 2 +- > libibmad/include/infiniband/mad.h | 8 ++++---- > libibmad/src/resolve.c | 2 +- > libibmad/src/sa.c | 2 +- > 4 files changed, 7 insertions(+), 7 deletions(-) > > diff --git a/infiniband-diags/src/ibaddr.c b/infiniband-diags/src/ibaddr.c > index f71edf8..2ac6a4f 100644 > --- a/infiniband-diags/src/ibaddr.c > +++ b/infiniband-diags/src/ibaddr.c > @@ -56,7 +56,7 @@ ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) > uint8_t nodeinfo[64]; > uint64_t guid, prefix; > char buf1[64], buf2[64]; > - ib_gid_t gid; > + mad_gid_t gid; > int lmc; > > if (!smp_query(nodeinfo, portid, IB_ATTR_NODE_INFO, 0, 0)) > diff --git a/libibmad/include/infiniband/mad.h b/libibmad/include/infiniband/mad.h > index 15b8246..adce3e8 100644 > --- a/libibmad/include/infiniband/mad.h > +++ b/libibmad/include/infiniband/mad.h > @@ -156,7 +156,7 @@ enum GSI_ATTR_ID { > #define IB_VENDOR_OPENIB_SYSSTAT_CLASS (IB_VENDOR_RANGE2_START_CLASS + 3) > #define IB_OPENIB_OUI (0x001405) > > -typedef uint8_t ib_gid_t[16]; Rather than eliminating ib_gid_t from mad.h initially, can this be marked as deprecated and then removed in a subsequent release to ease the migration ? Or does that still result in conflicting typedefs ? > +typedef uint8_t mad_gid_t[16]; Should this be ib_mad_gid_t ? -- Hal > typedef struct { > int cnt; > @@ -189,7 +189,7 @@ typedef struct portid { > int lid; /* lid or 0 if directed route */ > ib_dr_path_t drpath; > int grh_present; /* flag */ > - ib_gid_t gid; > + mad_gid_t gid; > uint32_t qp; > uint32_t qkey; > uint8_t sl; > @@ -777,7 +777,7 @@ uint8_t * sa_call(void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, > unsigned timeout); > uint8_t * sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, > ib_sa_call_t *sa, unsigned timeout); > -int ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, > +int ib_path_query(mad_gid_t srcgid, mad_gid_t destgid, ib_portid_t *sm_id, > void *buf); /* returns lid */ > > inline static uint8_t * > @@ -799,7 +799,7 @@ int ib_resolve_guid(ib_portid_t *portid, uint64_t *guid, > ib_portid_t *sm_id, int timeout); > int ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, > int dest_type, ib_portid_t *sm_id); > -int ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid); > +int ib_resolve_self(ib_portid_t *portid, int *portnum, mad_gid_t *gid); > > /* gs.c */ > uint8_t *perf_classportinfo_query(void *rcvbuf, ib_portid_t *dest, int port, > diff --git a/libibmad/src/resolve.c b/libibmad/src/resolve.c > index d8365b2..7d8eb6e 100644 > --- a/libibmad/src/resolve.c > +++ b/libibmad/src/resolve.c > @@ -138,7 +138,7 @@ ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_por > } > > int > -ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid) > +ib_resolve_self(ib_portid_t *portid, int *portnum, mad_gid_t *gid) > { > ib_portid_t self = {0}; > uint8_t portinfo[64]; > diff --git a/libibmad/src/sa.c b/libibmad/src/sa.c > index ddbfce3..fde10bd 100644 > --- a/libibmad/src/sa.c > +++ b/libibmad/src/sa.c > @@ -112,7 +112,7 @@ sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, > IB_PR_COMPMASK_NUMBPATH) > > int > -ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, void *buf) > +ib_path_query(mad_gid_t srcgid, mad_gid_t destgid, ib_portid_t *sm_id, void *buf) > { > int npath; > ib_sa_call_t sa = {0}; From pw at osc.edu Thu Mar 6 09:20:20 2008 From: pw at osc.edu (Pete Wyckoff) Date: Thu, 6 Mar 2008 12:20:20 -0500 Subject: [ofa-general] Re: valgrind warnings in libibverbs & PVFS In-Reply-To: <47CE0A06.7060509@scl.ameslab.gov> References: <47CE0A06.7060509@scl.ameslab.gov> Message-ID: <20080306172020.GD7754@osc.edu> troy at scl.ameslab.gov wrote on Tue, 04 Mar 2008 20:48 -0600: > I am trying to track down some issues with PVFS using IB with valgrind, and > I'm trying to make a run of 'valgrind pvfs2-ls' come out with no errors. > > The first thing I managed to figure out is this change to libibverbs: > > p4l4:/usr/src/ib/ofed-1_3_git/libibverbs/Bppc32# git diff > diff --git a/src/cmd.c b/src/cmd.c > diff --git a/src/verbs.c b/src/verbs.c > index 11d3c4c..bdfe723 100644 > --- a/src/verbs.c > +++ b/src/verbs.c > @@ -226,6 +226,8 @@ struct ibv_comp_channel *ibv_create_comp_channel(struct > ibv_context *context) > return NULL; > } > > + VALGRIND_MAKE_MEM_DEFINED(&resp.fd, sizeof(resp.fd)); > + > channel->context = context; > channel->fd = resp.fd; > channel->refcnt = 0; > This change makes sense, although I probably would have used &resp, sizeof(resp). There are a bunch of places that use IBV_INIT_CMD_RESP, some of which have VALGRIND annotations too. But there's a bunch that do not. Would make a nice cleanup to do them all. The basic issue of having write(fd, ..) mysteriously cause some other chunk of memory to get written is something that valgrind cannot track. > However, I am still getting two errors, and I can't seem to figure out if > it's a PVFS issue, an ibverbs issue, or a libmthca issue, and I'm wondering > how to track this down. > > troy at p4l4:/usr/src/pvfs2-hg/Bppc32$ valgrind src/apps/admin/pvfs2-ls > ==2541== Memcheck, a memory error detector. > ==2541== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. > ==2541== Using LibVEX rev 1658, a library for dynamic binary translation. > ==2541== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. > ==2541== Using valgrind-3.2.1-Debian, a dynamic binary instrumentation > framework. > ==2541== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. > ==2541== For more details, rerun with: -v > ==2541== > ==2541== Syscall param write(buf) points to uninitialised byte(s) > ==2541== at 0xFEB2B14: write (in /usr/lib/debug/libpthread-0.10.so) > ==2541== by 0xFE7DD3C: ibv_cmd_create_cq (cmd.c:320) > ==2541== by 0xFE84080: ibv_create_cq@@IBVERBS_1.1 (verbs.c:281) > ==2541== by 0x10055C0C: openib_ib_initialize (openib.c:956) > ==2541== by 0x10051458: BMI_ib_initialize (ib.c:2001) > ==2541== by 0x1004CA00: activate_method (bmi.c:2008) > ==2541== by 0x1004CEE4: BMI_addr_lookup (bmi.c:1555) > ==2541== by 0x10032D20: PVFS_isys_fs_add (fs-add.sm:127) > ==2541== by 0x10032F50: PVFS_sys_fs_add (fs-add.sm:194) > ==2541== by 0x1000B698: main (pvfs2-ls.c:766) > ==2541== Address 0xFEB9D550 is on thread 1's stack Either some unused fields in cmd, or padding between fields. Hard to silence these nicely yet avoid bugs. I have some valgrind rules to avoid some of these. It could actually be a bug, but unlikely. > ==2541== Conditional jump or move depends on uninitialised value(s) > ==2541== at 0xFB19A80: mthca_cq_clean (cq.c:576) > ==2541== by 0xFB1D688: mthca_destroy_qp (verbs.c:674) > ==2541== by 0xFE83758: ibv_destroy_qp@@IBVERBS_1.1 (verbs.c:490) > ==2541== by 0x1005693C: openib_close_connection (openib.c:368) > ==2541== by 0x1004F6E4: ib_close_connection (ib.c:1695) > ==2541== by 0x10051800: BMI_ib_finalize (ib.c:2082) > ==2541== by 0x1004DFA8: BMI_finalize (bmi.c:474) > ==2541== by 0x100112B0: PVFS_sys_finalize (finalize.c:57) > ==2541== by 0x1000BDF0: main (pvfs2-ls.c:850) That may actually be a bug, but you'll have to look at the code there in your version of libmthca. I'm not sure what the ofed to git commit id mapping is. -- Pete From wayne.davey at albaad.com Thu Mar 6 10:02:11 2008 From: wayne.davey at albaad.com (wayne.davey at albaad.com) Date: Thu, 6 Mar 2008 19:02:11 +0100 Subject: [ofa-general] We have a ecard greeting for you. Message-ID: <001a01c87fb4$335db840$47ca3f1d@facvz> You have been sent the Funny Ecard http://200.118.232.52/ From subbukl at gmail.com Thu Mar 6 12:09:19 2008 From: subbukl at gmail.com (subbu kl) Date: Thu, 6 Mar 2008 14:09:19 -0600 Subject: [ofa-general] ib_create_fmr_pool do INIT_HLIST_NODE only if cache is enabled Message-ID: In ib_create_fmr_pool() shouldn't we do INIT_HLIST_NODE only if cache is enabled? proposed patch - ------------------------------------------------------------------------------------------------------------------------------- --- drivers/infiniband/core/fmr_pool.c 2008-02-20 09:38:14.000000000 -0600 +++ drivers/infiniband/core/fmr_pool.mod.c 2008-03-06 10:31: 44.000000000 -0600 @@ -321,7 +321,10 @@ fmr->pool = pool; fmr->remap_count = 0; fmr->ref_count = 0; - INIT_HLIST_NODE(&fmr->cache_node); + + if (params->cache) { + INIT_HLIST_NODE(&fmr->cache_node); + } fmr->fmr = ib_alloc_fmr(pd, params->access, &fmr_attr); if (IS_ERR(fmr->fmr)) { -- ~s u b b u -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Thu Mar 6 12:29:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 6 Mar 2008 20:29:08 +0000 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080306202908.GY30723@sashak.voltaire.com> On 08:58 Thu 06 Mar , Hal Rosenstock wrote: > > > +typedef uint8_t mad_gid_t[16]; > > Should this be ib_mad_gid_t ? Or maybe ibmad_gid_t - to match library name exactly? Sasha From hrosenstock at xsigo.com Thu Mar 6 12:29:58 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 06 Mar 2008 12:29:58 -0800 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <20080306202908.GY30723@sashak.voltaire.com> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> Message-ID: <1204835398.6469.725.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-06 at 20:29 +0000, Sasha Khapyorsky wrote: > On 08:58 Thu 06 Mar , Hal Rosenstock wrote: > > > > > +typedef uint8_t mad_gid_t[16]; > > > > Should this be ib_mad_gid_t ? > > Or maybe ibmad_gid_t - to match library name exactly? Yes but then there would be lots of others to make consistent and follow the standard. Would they all be converted over ? That would be the best thing to do but is more disruptive. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sashak at voltaire.com Thu Mar 6 12:43:48 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 6 Mar 2008 20:43:48 +0000 Subject: [ofa-general] [PATCH] opensm: release notes update In-Reply-To: <47CFCB9F.6070109@mellanox.co.il> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305162729.GP30723@sashak.voltaire.com> <47CFCB9F.6070109@mellanox.co.il> Message-ID: <20080306204348.GZ30723@sashak.voltaire.com> On 12:46 Thu 06 Mar , Tziporet Koren wrote: > Do you want me to update OFED RN too? > This will be good for future OFED fix releases Sure, it would be good. I'm sending the patch. Sasha From sashak at voltaire.com Thu Mar 6 12:44:45 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 6 Mar 2008 20:44:45 +0000 Subject: [ofa-general] [PATCH] ofed/docs: update OpenSM release notes. In-Reply-To: <20080306204348.GZ30723@sashak.voltaire.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305162729.GP30723@sashak.voltaire.com> <47CFCB9F.6070109@mellanox.co.il> <20080306204348.GZ30723@sashak.voltaire.com> Message-ID: <20080306204445.GA30723@sashak.voltaire.com> Update OpenSM release notes - state QoS manager experimental status, add its sw dependencies. Signed-off-by: Sasha Khapyorsky --- opensm_release_notes.txt | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/opensm_release_notes.txt b/opensm_release_notes.txt index 39b3c15..825d758 100644 --- a/opensm_release_notes.txt +++ b/opensm_release_notes.txt @@ -24,7 +24,7 @@ This document includes the following sections: 1.1 Major New Features -* QoS manager +* QoS manager (experimental) This QoS manager implementation is in accordance with IBA QoS Annex. Highly configurable QoS Policy is parsed from OpenSM QoS policy file. Valid QoS parameters will be reported in SA PathRecord and @@ -128,6 +128,9 @@ OFED 1.0, OpenIB gen2 (e.g. IBG2 distribution), OpenIB gen1 (e.g. IBGD distribution), or Mellanox VAPI stacks. The qualified driver versions are provided in Table 2, "Qualified IB Stacks". +Also building of QoS manager policy file parser requires flex and bison +installed. + 1.5 Supported Devices Firmware The main task of OpenSM is to initialize InfiniBand devices. The @@ -261,7 +264,7 @@ or visible bugs were also fixed. * osm_mcast_mgr.c: Possible NULL ptr seg fault -* TrapRepress was failing for mkey != 0 +* TrapRepress was failing for mkey != 0 * IB_PR_COMPMASK was used in MPR @@ -474,4 +477,3 @@ iPath | QLE6140 (PathScale InfiniPath PE-880) Note: OpenSM does not run on an IBM Galaxy (eHCA) as it does not expose QP0 and QP1. However, it does support it as a device on the subnet. - -- 1.5.4.rc2.60.gb2e62 From sashak at voltaire.com Thu Mar 6 12:49:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 6 Mar 2008 20:49:05 +0000 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <1204835398.6469.725.camel@hrosenstock-ws.xsigo.com> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> <1204835398.6469.725.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080306204905.GB30723@sashak.voltaire.com> On 12:29 Thu 06 Mar , Hal Rosenstock wrote: > On Thu, 2008-03-06 at 20:29 +0000, Sasha Khapyorsky wrote: > > On 08:58 Thu 06 Mar , Hal Rosenstock wrote: > > > > > > > +typedef uint8_t mad_gid_t[16]; > > > > > > Should this be ib_mad_gid_t ? > > > > Or maybe ibmad_gid_t - to match library name exactly? > > Yes but then there would be lots of others to make consistent and follow > the standard. Would they all be converted over ? That would be the best > thing to do but is more disruptive. Yes, this would be nice. Not sure when we will do it - one-by-one moving is likely better than nothing anyway. Sasha From hrosenstock at xsigo.com Thu Mar 6 12:39:23 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 06 Mar 2008 12:39:23 -0800 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <20080306204905.GB30723@sashak.voltaire.com> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> <1204835398.6469.725.camel@hrosenstock-ws.xsigo.com> <20080306204905.GB30723@sashak.voltaire.com> Message-ID: <1204835963.6469.731.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-06 at 20:49 +0000, Sasha Khapyorsky wrote: > On 12:29 Thu 06 Mar , Hal Rosenstock wrote: > > On Thu, 2008-03-06 at 20:29 +0000, Sasha Khapyorsky wrote: > > > On 08:58 Thu 06 Mar , Hal Rosenstock wrote: > > > > > > > > > +typedef uint8_t mad_gid_t[16]; > > > > > > > > Should this be ib_mad_gid_t ? > > > > > > Or maybe ibmad_gid_t - to match library name exactly? > > > > Yes but then there would be lots of others to make consistent and follow > > the standard. Would they all be converted over ? That would be the best > > thing to do but is more disruptive. > > Yes, this would be nice. Not sure when we will do it - one-by-one moving > is likely better than nothing anyway. For this one where there's a conflict, that's fine but if others are done one by one, it will be more disruptive. IMO the rest should be done in one fell swoop and the fewer times this changes the better. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From Moskvich-V at Mail.ru Thu Mar 6 12:41:04 2008 From: Moskvich-V at Mail.ru () Date: Thu, 6 Mar 2008 23:41:04 +0300 Subject: [ofa-general] =?iso-8859-1?q?=C2=E0=F8_=E4=EE=EF=EE=EB=ED=E8=F2?= =?iso-8859-1?q?=E5=EB=FC=ED=FB=E9_=E4=EE=F5=EE=E4=2C=E0_=EC=EE=E6?= =?iso-8859-1?q?=E5=F2_=E8_=EE=E1=E5=F1=EF=E5=F7=E5=ED=ED=EE=E5_=E1?= =?iso-8859-1?q?=F3=E4=F3=F9=E5=E5?= Message-ID: <200803062341.MOMJOXCTFJ@82.138.52.148>                                                                                Добрый день !      Извините,что,возможно ,отнимаю у Вас время,но если Вы имеете  доступ к Интернету,умеете пользоваться  мышкой,   имеете 2-3 часа свободного времени,и Вас интересуют деньги,то всего один щелчок мышкой может отделять Вас от состояния !!! Еще раз простите за вторжение.Желаю Вам удачного дня . Жду Вашего ответа : Moskvich-V at Mail.ru  или  Vladimir11682 at Yandex.ru Удачи Вам !!! Информация находиться в прикрепленном   файле  .... Не беспокойтесь, это не вирус,но если Вы ,тем не менее,  опасаетесь  его  открыть  ,не  спешите  его  удалять ,вовсе не сложно проверить почту при помощи антивируса. Это не реклама, а действительно выгодное предложение !!! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Golden Stream.doc Type: application/octet-stream Size: 160256 bytes Desc: not available URL: From sashak at voltaire.com Thu Mar 6 12:56:37 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 6 Mar 2008 20:56:37 +0000 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <1204835963.6469.731.camel@hrosenstock-ws.xsigo.com> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> <1204835398.6469.725.camel@hrosenstock-ws.xsigo.com> <20080306204905.GB30723@sashak.voltaire.com> <1204835963.6469.731.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080306205637.GD30723@sashak.voltaire.com> On 12:39 Thu 06 Mar , Hal Rosenstock wrote: > > For this one where there's a conflict, that's fine but if others are > done one by one, it will be more disruptive. IMO the rest should be done > in one fell swoop and the fewer times this changes the better. Yes, I agree with it. Sasha From gershom at praylubbock.com Thu Mar 6 13:19:11 2008 From: gershom at praylubbock.com (Murat Mathews) Date: Thu, 06 Mar 2008 21:19:11 +0000 Subject: [ofa-general] FontFolio 11 by Adobe 187, Pc or mac at 87%_discount Message-ID: <000801c87fcf$122dac00$0100007f@ailrty> honestech vhs to dvd dlx3 - 29 adobe acrobat professional 7 - 69 roxio toast titanium 8 - 39 adobe golive cs2 - 49 sas jmp statistical discovery 7 - 129 actionScript 3 flash cs3 pro essential training - 39 microsoft visio 2007 professional - 39 creative suite premium 2 - 149 _ visit _xp4less .com_ in Internet browser Remove _ before you visit in Internet browser _ Spain is one of the world's top migrant destinations. As the economy slumps however, popular sentiment toward recent arrivals is shifting from open to angry. In January, the U.S. Immigration Service signed a pact with the Vietnamese government, agreeing to deport thousands of illegal Vietnamese immigrants who are currently under deportation orders. Prior to this pact, the Vietnamese government refused to take in deportees. From weiny2 at llnl.gov Thu Mar 6 14:20:58 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 6 Mar 2008 14:20:58 -0800 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <20080306202908.GY30723@sashak.voltaire.com> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> Message-ID: <20080306142058.742a879e.weiny2@llnl.gov> On Thu, 6 Mar 2008 20:29:08 +0000 Sasha Khapyorsky wrote: > On 08:58 Thu 06 Mar , Hal Rosenstock wrote: > > > > > +typedef uint8_t mad_gid_t[16]; > > > > Should this be ib_mad_gid_t ? > > Or maybe ibmad_gid_t - to match library name exactly? I think when we change every thing from ib_* to ibmad_* we should also change the name of the include file from mad.h to ibmad.h. But I think that is a bigger change for another day. Here is a new version of the patch using ibmad_gid_t. Furthermore users of the new header can define USE_DEPRICATED_IB_GID_T to get the old definition. For example: #define USE_DEPRICATED_IB_GID_T #include We can remove that in the future. What do you think? Ira >From d2db4ac01d88bfec354972b3acf0fd927428cb5b Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Wed, 5 Mar 2008 17:25:33 -0800 Subject: [PATCH] Rename ib_gid_t in mad.h to ibmad_gid_t to prevent name collision with ib_types.h Signed-off-by: Ira K. Weiny --- infiniband-diags/src/ibaddr.c | 2 +- libibmad/include/infiniband/mad.h | 11 +++++++---- libibmad/src/resolve.c | 2 +- libibmad/src/sa.c | 2 +- 4 files changed, 10 insertions(+), 7 deletions(-) diff --git a/infiniband-diags/src/ibaddr.c b/infiniband-diags/src/ibaddr.c index f71edf8..6c56a3e 100644 --- a/infiniband-diags/src/ibaddr.c +++ b/infiniband-diags/src/ibaddr.c @@ -56,7 +56,7 @@ ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) uint8_t nodeinfo[64]; uint64_t guid, prefix; char buf1[64], buf2[64]; - ib_gid_t gid; + ibmad_gid_t gid; int lmc; if (!smp_query(nodeinfo, portid, IB_ATTR_NODE_INFO, 0, 0)) diff --git a/libibmad/include/infiniband/mad.h b/libibmad/include/infiniband/mad.h index 15b8246..2a0eeba 100644 --- a/libibmad/include/infiniband/mad.h +++ b/libibmad/include/infiniband/mad.h @@ -156,7 +156,10 @@ enum GSI_ATTR_ID { #define IB_VENDOR_OPENIB_SYSSTAT_CLASS (IB_VENDOR_RANGE2_START_CLASS + 3) #define IB_OPENIB_OUI (0x001405) -typedef uint8_t ib_gid_t[16]; +typedef uint8_t ibmad_gid_t[16]; +#ifdef USE_DEPRICATED_IB_GID_T +typedef ibmad_gid_t ib_gid_t; +#endif typedef struct { int cnt; @@ -189,7 +192,7 @@ typedef struct portid { int lid; /* lid or 0 if directed route */ ib_dr_path_t drpath; int grh_present; /* flag */ - ib_gid_t gid; + ibmad_gid_t gid; uint32_t qp; uint32_t qkey; uint8_t sl; @@ -777,7 +780,7 @@ uint8_t * sa_call(void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, unsigned timeout); uint8_t * sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, unsigned timeout); -int ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, +int ib_path_query(ibmad_gid_t srcgid, ibmad_gid_t destgid, ib_portid_t *sm_id, void *buf); /* returns lid */ inline static uint8_t * @@ -799,7 +802,7 @@ int ib_resolve_guid(ib_portid_t *portid, uint64_t *guid, ib_portid_t *sm_id, int timeout); int ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_portid_t *sm_id); -int ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid); +int ib_resolve_self(ib_portid_t *portid, int *portnum, ibmad_gid_t *gid); /* gs.c */ uint8_t *perf_classportinfo_query(void *rcvbuf, ib_portid_t *dest, int port, diff --git a/libibmad/src/resolve.c b/libibmad/src/resolve.c index d8365b2..b063867 100644 --- a/libibmad/src/resolve.c +++ b/libibmad/src/resolve.c @@ -138,7 +138,7 @@ ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_por } int -ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid) +ib_resolve_self(ib_portid_t *portid, int *portnum, ibmad_gid_t *gid) { ib_portid_t self = {0}; uint8_t portinfo[64]; diff --git a/libibmad/src/sa.c b/libibmad/src/sa.c index ddbfce3..59ef7de 100644 --- a/libibmad/src/sa.c +++ b/libibmad/src/sa.c @@ -112,7 +112,7 @@ sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, IB_PR_COMPMASK_NUMBPATH) int -ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, void *buf) +ib_path_query(ibmad_gid_t srcgid, ibmad_gid_t destgid, ib_portid_t *sm_id, void *buf) { int npath; ib_sa_call_t sa = {0}; -- 1.5.1 From sW_mooneyham at activeware.com Thu Mar 6 12:35:44 2008 From: sW_mooneyham at activeware.com (arny sue-elle) Date: Thu, 06 Mar 2008 20:35:44 +0000 Subject: [ofa-general] Best Opportunity For US Citizens ONLY / LUGANO TOURISM Message-ID: <95382.shun@sue-elle> Opportunity From LUGANO Best Hotels Hello! Are you looking for an interesting job? Do you want to build perfect career with world leading team? Do you have a few available hours a day? Do you want to try part-time position from world leading tour operator? Bingo! We are glad to say that we are hiring. At this time we have various job openings for US citizens only. You do not need to have any tourism education and experiences, you do not need to relocate, you do not need to invest anything and we will cover all possible charges and fees. All you have to do at this time is just to read requirements and send us your CV or RESUME or just a few words about yourself for qualification. Our HR department will inform you with results very soon. Requirements: Strong computer skills (MS Office(MS Word 97/2003/2007) Any POP3 Mail Client) Good communication skills (Verbal and Written) Mobile phone (Just for Urgent Calls) High School, College, University (Any Education) Must Love People Send your resume/cv/profile to this address only: maria.has.dallas at gmail.com Best Places Best Guides SALARY: $3,500.00/month + % JOB TYPE: Part-Time/Employee PLACE OF WORK: Virtual Office ÷¿ Copyright 2004-2008 | Lugano Tourism -------------- next part -------------- An HTML attachment was scrubbed... URL: From sales.pharmacy at unkcn.com Fri Mar 7 15:26:29 2008 From: sales.pharmacy at unkcn.com (Canadian Pharmacy) Date: Fri, 8 Mar 2008 00:26:29 +0100 Subject: [ofa-general] The Best Source of Cheap Drugs Message-ID: <988187023.39660378632080@encountstyle.com> Do you love sex but have ed problems? Forget about them with Viagra or Cialis pills. Save your money, buy high-quality pills at low price.http://geocities.com/tracyhester737/We provide confidential and secure purchase. http://geocities.com/waltersannabelle/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Thu Mar 6 16:24:55 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 06 Mar 2008 16:24:55 -0800 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <20080306142058.742a879e.weiny2@llnl.gov> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> <20080306142058.742a879e.weiny2@llnl.gov> Message-ID: <1204849495.6469.739.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-06 at 14:20 -0800, Ira Weiny wrote: > On Thu, 6 Mar 2008 20:29:08 +0000 > Sasha Khapyorsky wrote: > > > On 08:58 Thu 06 Mar , Hal Rosenstock wrote: > > > > > > > +typedef uint8_t mad_gid_t[16]; > > > > > > Should this be ib_mad_gid_t ? > > > > Or maybe ibmad_gid_t - to match library name exactly? > > I think when we change every thing from ib_* to ibmad_* we should also change > the name of the include file from mad.h to ibmad.h. But I think that is a > bigger change for another day. > > Here is a new version of the patch using ibmad_gid_t. Furthermore users of the > new header can define USE_DEPRICATED_IB_GID_T to get the old definition. For > example: > > #define USE_DEPRICATED_IB_GID_T ^^^^^^^^^^ typo DEPRECATED Anyhow, I meant by using gcc's deprecated attribute. -- Hal > #include > > We can remove that in the future. > > What do you think? > Ira > > > From d2db4ac01d88bfec354972b3acf0fd927428cb5b Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Wed, 5 Mar 2008 17:25:33 -0800 > Subject: [PATCH] Rename ib_gid_t in mad.h to ibmad_gid_t to prevent name collision with ib_types.h > > > Signed-off-by: Ira K. Weiny > --- > infiniband-diags/src/ibaddr.c | 2 +- > libibmad/include/infiniband/mad.h | 11 +++++++---- > libibmad/src/resolve.c | 2 +- > libibmad/src/sa.c | 2 +- > 4 files changed, 10 insertions(+), 7 deletions(-) > > diff --git a/infiniband-diags/src/ibaddr.c b/infiniband-diags/src/ibaddr.c > index f71edf8..6c56a3e 100644 > --- a/infiniband-diags/src/ibaddr.c > +++ b/infiniband-diags/src/ibaddr.c > @@ -56,7 +56,7 @@ ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) > uint8_t nodeinfo[64]; > uint64_t guid, prefix; > char buf1[64], buf2[64]; > - ib_gid_t gid; > + ibmad_gid_t gid; > int lmc; > > if (!smp_query(nodeinfo, portid, IB_ATTR_NODE_INFO, 0, 0)) > diff --git a/libibmad/include/infiniband/mad.h b/libibmad/include/infiniband/mad.h > index 15b8246..2a0eeba 100644 > --- a/libibmad/include/infiniband/mad.h > +++ b/libibmad/include/infiniband/mad.h > @@ -156,7 +156,10 @@ enum GSI_ATTR_ID { > #define IB_VENDOR_OPENIB_SYSSTAT_CLASS (IB_VENDOR_RANGE2_START_CLASS + 3) > #define IB_OPENIB_OUI (0x001405) > > -typedef uint8_t ib_gid_t[16]; > +typedef uint8_t ibmad_gid_t[16]; > +#ifdef USE_DEPRICATED_IB_GID_T > +typedef ibmad_gid_t ib_gid_t; > +#endif > > typedef struct { > int cnt; > @@ -189,7 +192,7 @@ typedef struct portid { > int lid; /* lid or 0 if directed route */ > ib_dr_path_t drpath; > int grh_present; /* flag */ > - ib_gid_t gid; > + ibmad_gid_t gid; > uint32_t qp; > uint32_t qkey; > uint8_t sl; > @@ -777,7 +780,7 @@ uint8_t * sa_call(void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, > unsigned timeout); > uint8_t * sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, > ib_sa_call_t *sa, unsigned timeout); > -int ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, > +int ib_path_query(ibmad_gid_t srcgid, ibmad_gid_t destgid, ib_portid_t *sm_id, > void *buf); /* returns lid */ > > inline static uint8_t * > @@ -799,7 +802,7 @@ int ib_resolve_guid(ib_portid_t *portid, uint64_t *guid, > ib_portid_t *sm_id, int timeout); > int ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, > int dest_type, ib_portid_t *sm_id); > -int ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid); > +int ib_resolve_self(ib_portid_t *portid, int *portnum, ibmad_gid_t *gid); > > /* gs.c */ > uint8_t *perf_classportinfo_query(void *rcvbuf, ib_portid_t *dest, int port, > diff --git a/libibmad/src/resolve.c b/libibmad/src/resolve.c > index d8365b2..b063867 100644 > --- a/libibmad/src/resolve.c > +++ b/libibmad/src/resolve.c > @@ -138,7 +138,7 @@ ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_por > } > > int > -ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid) > +ib_resolve_self(ib_portid_t *portid, int *portnum, ibmad_gid_t *gid) > { > ib_portid_t self = {0}; > uint8_t portinfo[64]; > diff --git a/libibmad/src/sa.c b/libibmad/src/sa.c > index ddbfce3..59ef7de 100644 > --- a/libibmad/src/sa.c > +++ b/libibmad/src/sa.c > @@ -112,7 +112,7 @@ sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, > IB_PR_COMPMASK_NUMBPATH) > > int > -ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, void *buf) > +ib_path_query(ibmad_gid_t srcgid, ibmad_gid_t destgid, ib_portid_t *sm_id, void *buf) > { > int npath; > ib_sa_call_t sa = {0}; From weiny2 at llnl.gov Thu Mar 6 18:30:37 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 6 Mar 2008 18:30:37 -0800 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <1204849495.6469.739.camel@hrosenstock-ws.xsigo.com> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> <20080306142058.742a879e.weiny2@llnl.gov> <1204849495.6469.739.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080306183037.201d6ac0.weiny2@llnl.gov> On Thu, 06 Mar 2008 16:24:55 -0800 Hal Rosenstock wrote: > On Thu, 2008-03-06 at 14:20 -0800, Ira Weiny wrote: > > On Thu, 6 Mar 2008 20:29:08 +0000 > > Sasha Khapyorsky wrote: > > > > > On 08:58 Thu 06 Mar , Hal Rosenstock wrote: > > > > > > > > > +typedef uint8_t mad_gid_t[16]; > > > > > > > > Should this be ib_mad_gid_t ? > > > > > > Or maybe ibmad_gid_t - to match library name exactly? > > > > I think when we change every thing from ib_* to ibmad_* we should also change > > the name of the include file from mad.h to ibmad.h. But I think that is a > > bigger change for another day. > > > > Here is a new version of the patch using ibmad_gid_t. Furthermore users of the > > new header can define USE_DEPRICATED_IB_GID_T to get the old definition. For > > example: > > > > #define USE_DEPRICATED_IB_GID_T > ^^^^^^^^^^ > typo DEPRECATED Oops, fixed. > > Anyhow, I meant by using gcc's deprecated attribute. > Ok, I added that as well, but that only reports a compile error when someone uses it. Without the USE_DEPRECATED_IB_GID_T macro "ib_gid_t" still conflicts with ib_types.h. I really want to fix that problem. What I like about the macro is if someone has a lot of uses in their code this gives them an easy way to compile against the new version. Make sense? Ira >From 44aae47f104441d921f10ded20800e796d0657d2 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Wed, 5 Mar 2008 17:25:33 -0800 Subject: [PATCH] Rename ib_gid_t in mad.h to ibmad_gid_t to prevent name collision with ib_types.h Signed-off-by: Ira K. Weiny --- infiniband-diags/src/ibaddr.c | 2 +- libibmad/include/infiniband/mad.h | 11 +++++++---- libibmad/src/resolve.c | 2 +- libibmad/src/sa.c | 2 +- 4 files changed, 10 insertions(+), 7 deletions(-) diff --git a/infiniband-diags/src/ibaddr.c b/infiniband-diags/src/ibaddr.c index f71edf8..6c56a3e 100644 --- a/infiniband-diags/src/ibaddr.c +++ b/infiniband-diags/src/ibaddr.c @@ -56,7 +56,7 @@ ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) uint8_t nodeinfo[64]; uint64_t guid, prefix; char buf1[64], buf2[64]; - ib_gid_t gid; + ibmad_gid_t gid; int lmc; if (!smp_query(nodeinfo, portid, IB_ATTR_NODE_INFO, 0, 0)) diff --git a/libibmad/include/infiniband/mad.h b/libibmad/include/infiniband/mad.h index 15b8246..4f19a31 100644 --- a/libibmad/include/infiniband/mad.h +++ b/libibmad/include/infiniband/mad.h @@ -156,7 +156,10 @@ enum GSI_ATTR_ID { #define IB_VENDOR_OPENIB_SYSSTAT_CLASS (IB_VENDOR_RANGE2_START_CLASS + 3) #define IB_OPENIB_OUI (0x001405) -typedef uint8_t ib_gid_t[16]; +typedef uint8_t ibmad_gid_t[16]; +#ifdef USE_DEPRECATED_IB_GID_T +typedef ibmad_gid_t ib_gid_t __attribute__((deprecated)); +#endif typedef struct { int cnt; @@ -189,7 +192,7 @@ typedef struct portid { int lid; /* lid or 0 if directed route */ ib_dr_path_t drpath; int grh_present; /* flag */ - ib_gid_t gid; + ibmad_gid_t gid; uint32_t qp; uint32_t qkey; uint8_t sl; @@ -777,7 +780,7 @@ uint8_t * sa_call(void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, unsigned timeout); uint8_t * sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, unsigned timeout); -int ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, +int ib_path_query(ibmad_gid_t srcgid, ibmad_gid_t destgid, ib_portid_t *sm_id, void *buf); /* returns lid */ inline static uint8_t * @@ -799,7 +802,7 @@ int ib_resolve_guid(ib_portid_t *portid, uint64_t *guid, ib_portid_t *sm_id, int timeout); int ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_portid_t *sm_id); -int ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid); +int ib_resolve_self(ib_portid_t *portid, int *portnum, ibmad_gid_t *gid); /* gs.c */ uint8_t *perf_classportinfo_query(void *rcvbuf, ib_portid_t *dest, int port, diff --git a/libibmad/src/resolve.c b/libibmad/src/resolve.c index d8365b2..b063867 100644 --- a/libibmad/src/resolve.c +++ b/libibmad/src/resolve.c @@ -138,7 +138,7 @@ ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_por } int -ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid) +ib_resolve_self(ib_portid_t *portid, int *portnum, ibmad_gid_t *gid) { ib_portid_t self = {0}; uint8_t portinfo[64]; diff --git a/libibmad/src/sa.c b/libibmad/src/sa.c index ddbfce3..59ef7de 100644 --- a/libibmad/src/sa.c +++ b/libibmad/src/sa.c @@ -112,7 +112,7 @@ sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, IB_PR_COMPMASK_NUMBPATH) int -ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, void *buf) +ib_path_query(ibmad_gid_t srcgid, ibmad_gid_t destgid, ib_portid_t *sm_id, void *buf) { int npath; ib_sa_call_t sa = {0}; -- 1.5.1 From a-alanb at acdaonline.com Fri Mar 7 18:37:33 2008 From: a-alanb at acdaonline.com (Trisha Judd) Date: Fri, 8 Mar 2008 10:37:33 +0800 Subject: [ofa-general] I saw your pic Message-ID: <028612684.83614394501172@acdaonline.com> Hello! I am tired this evening. I am nice girl that would like to chat with you. Email me at Sonja at GreatGolovaSite.com only, because I am using my friend's email to write this. I want to show you some pictures. From sales.pharmacy at uno.zonasa.com Fri Mar 7 19:08:56 2008 From: sales.pharmacy at uno.zonasa.com (Canadian Pharmacy) Date: Fri, 8 Mar 2008 10:08:56 +0700 Subject: [ofa-general] Discounted Pills for All Needs Message-ID: <524535620.60179046887791@unitechnous.com> Buy Must Have medications at Canada based pharmacy. No prescription at all. Save your money, buy pills immediately.http://geocities.com/solomonroberts90/We are verified by VISA and Mastercard. Confidential purchase. http://geocities.com/curtsnow158/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sales.pharmacy at uno.zonasa.com Fri Mar 7 19:08:56 2008 From: sales.pharmacy at uno.zonasa.com (Canadian Pharmacy) Date: Fri, 8 Mar 2008 10:08:56 +0700 Subject: [ofa-general] Discounted Pills for All Needs Message-ID: <524535620.60179046887791@unitechnous.com> Buy Must Have medications at Canada based pharmacy. No prescription at all. Save your money, buy pills immediately.http://geocities.com/solomonroberts90/We are verified by VISA and Mastercard. Confidential purchase. http://geocities.com/curtsnow158/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From consultationsjqn at tibet.de Thu Mar 6 21:23:53 2008 From: consultationsjqn at tibet.de (Darius Flint) Date: Fri, 7 Mar 2008 13:23:53 +0800 Subject: [ofa-general] rep!ic@ watches :: rolex:: fake //atches :: Message-ID: <950125025.64767687714660@tibet.de> W grn atc dv hesG ds uc hxa ci Ba zba gsL wm V Ba fiw gs & Wa ku lle dav tsCu nya ff dzv lin jr ksKe ffe ych dui ai lgw nsLi eqt ght ss ersTi xhx ffa he ny & Co. Je fhr we bgv lryP ims en zc sBo ez x Se lw ts & Ac xiv ces ss or gnj ies The best prices!!! Cl bo ick he fti re!!!  to view all pro qf duct -------------- next part -------------- An HTML attachment was scrubbed... URL: From plaiters at telatron.com Thu Mar 6 23:25:02 2008 From: plaiters at telatron.com (Rob Harvey) Date: Fri, 07 Mar 2008 09:25:02 +0200 Subject: [ofa-general] Indesign CS3 by Adobe 71, Pc or mac at 86%~discount Message-ID: <000a01c88024$04eedc80$0100007f@lwoyj> adobe font folio 11 - 189 autodesk autocad electrical 2006 - 99 zend studio - 49 virtualdj 4.3 for mac - 39 autodesk autocad electrical 2006 - 99 virtual pc 7.0 for mac - 49 autodesk architectural studio 3.0 - 39 quarkxpress passport 7.3 - 79 ~ goto ~winteroemsale .com~ in Internet Exp!orer Take off ~ before you goto in Internet Exp!orer ~ Lawyers for Spain and a treasure hunting company based in Tampa, Fla., square off in federal court over what may be the most valuable shipwreck ever found. Spain believes the half-billion dollars in silver and gold coins may be its property and that the company that recovered it is withholding information that may show that to be the case. U.S. Secretary of State Condoleezza Rice is asking NATO to send more troops to Afghanistan. On a visit to Brussels Thursday, she will also urge additions to the alliance, including two former Soviet republics. From ereed at calculatepie.com Thu Mar 6 23:25:26 2008 From: ereed at calculatepie.com (ereed at calculatepie.com) Date: Fri, 7 Mar 2008 12:55:26 +0530 Subject: [ofa-general] You have one new ecard waiting! Message-ID: <001301c88024$6a1d9240$ea44bbc7@lxpc> Come get your personal funny postcard. You'll bust a gut! http://69.85.108.138/ From Alma at checkpoint.com Fri Mar 7 02:29:33 2008 From: Alma at checkpoint.com (Alma Hamm) Date: Fri, 07 Mar 2008 12:29:33 +0200 Subject: [ofa-general] Make your lady happy Message-ID: <39a001c8803e$2237a960$c0a80202@Alma> Cannot be contented with what Lord gave you? Science has found the way to increase it! Learn more about this super-remedy! http://yutrleap.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ChristoperpositiveMooney at standnow.org Fri Mar 7 02:53:50 2008 From: ChristoperpositiveMooney at standnow.org (Napoleon Rosales) Date: Fri, 7 Mar 2008 02:53:50 -0800 (PST) Subject: [ofa-general] Players from around the world! Message-ID: <20080307105351.4D9B6E607F7@openfabrics.org> After thatit's only fun and winning. http://foljij.net.cn/ From vlad at lists.openfabrics.org Fri Mar 7 03:07:25 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Fri, 7 Mar 2008 03:07:25 -0800 (PST) Subject: [ofa-general] ofa_1_3_kernel 20080307-0200 daily build status Message-ID: <20080307110726.182F1E6085B@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From dori at franceloisirs.com Fri Mar 7 02:04:37 2008 From: dori at franceloisirs.com (batholomew jane) Date: Fri, 07 Mar 2008 10:04:37 +0000 Subject: [ofa-general] Work From Home Message-ID: <000701c88049$01718388$57016aa2@autih> Hi there. We're looking for representatives in the US. Average compensation is $70.000/yr. No fees, nothing, only your strong desire to work and make money. Please visit our site to sign up for this program, for free. http://regdflo.freewebpage.org Thank you for your attention. From hrosenstock at xsigo.com Fri Mar 7 04:31:31 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 07 Mar 2008 04:31:31 -0800 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <20080306183037.201d6ac0.weiny2@llnl.gov> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> <20080306142058.742a879e.weiny2@llnl.gov> <1204849495.6469.739.camel@hrosenstock-ws.xsigo.com> <20080306183037.201d6ac0.weiny2@llnl.gov> Message-ID: <1204893091.6469.766.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-06 at 18:30 -0800, Ira Weiny wrote: > On Thu, 06 Mar 2008 16:24:55 -0800 > Hal Rosenstock wrote: > > > On Thu, 2008-03-06 at 14:20 -0800, Ira Weiny wrote: > > > On Thu, 6 Mar 2008 20:29:08 +0000 > > > Sasha Khapyorsky wrote: > > > > > > > On 08:58 Thu 06 Mar , Hal Rosenstock wrote: > > > > > > > > > > > +typedef uint8_t mad_gid_t[16]; > > > > > > > > > > Should this be ib_mad_gid_t ? > > > > > > > > Or maybe ibmad_gid_t - to match library name exactly? > > > > > > I think when we change every thing from ib_* to ibmad_* we should also change > > > the name of the include file from mad.h to ibmad.h. But I think that is a > > > bigger change for another day. > > > > > > Here is a new version of the patch using ibmad_gid_t. Furthermore users of the > > > new header can define USE_DEPRICATED_IB_GID_T to get the old definition. For > > > example: > > > > > > #define USE_DEPRICATED_IB_GID_T > > ^^^^^^^^^^ > > typo DEPRECATED > > Oops, fixed. > > > > > Anyhow, I meant by using gcc's deprecated attribute. > > > > Ok, I added that as well, but that only reports a compile error when someone > uses it. Without the USE_DEPRECATED_IB_GID_T macro "ib_gid_t" still conflicts > with ib_types.h. That's what I thought but wasn't sure :-( So we missed the window to deprecate now since we ideally would be removing this now... > I really want to fix that problem. What I like about the > macro is if someone has a lot of uses in their code this gives them an easy way > to compile against the new version. > Make sense? I understand what you are trying to accomplish but am not sure wjat the best way to move forward now. This may be the best alternative. I don't have any others to offer. -- Hal > Ira > > > From 44aae47f104441d921f10ded20800e796d0657d2 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Wed, 5 Mar 2008 17:25:33 -0800 > Subject: [PATCH] Rename ib_gid_t in mad.h to ibmad_gid_t to prevent name collision with ib_types.h > > > Signed-off-by: Ira K. Weiny > --- > infiniband-diags/src/ibaddr.c | 2 +- > libibmad/include/infiniband/mad.h | 11 +++++++---- > libibmad/src/resolve.c | 2 +- > libibmad/src/sa.c | 2 +- > 4 files changed, 10 insertions(+), 7 deletions(-) > > diff --git a/infiniband-diags/src/ibaddr.c b/infiniband-diags/src/ibaddr.c > index f71edf8..6c56a3e 100644 > --- a/infiniband-diags/src/ibaddr.c > +++ b/infiniband-diags/src/ibaddr.c > @@ -56,7 +56,7 @@ ib_resolve_addr(ib_portid_t *portid, int portnum, int show_lid, int show_gid) > uint8_t nodeinfo[64]; > uint64_t guid, prefix; > char buf1[64], buf2[64]; > - ib_gid_t gid; > + ibmad_gid_t gid; > int lmc; > > if (!smp_query(nodeinfo, portid, IB_ATTR_NODE_INFO, 0, 0)) > diff --git a/libibmad/include/infiniband/mad.h b/libibmad/include/infiniband/mad.h > index 15b8246..4f19a31 100644 > --- a/libibmad/include/infiniband/mad.h > +++ b/libibmad/include/infiniband/mad.h > @@ -156,7 +156,10 @@ enum GSI_ATTR_ID { > #define IB_VENDOR_OPENIB_SYSSTAT_CLASS (IB_VENDOR_RANGE2_START_CLASS + 3) > #define IB_OPENIB_OUI (0x001405) > > -typedef uint8_t ib_gid_t[16]; > +typedef uint8_t ibmad_gid_t[16]; > +#ifdef USE_DEPRECATED_IB_GID_T > +typedef ibmad_gid_t ib_gid_t __attribute__((deprecated)); > +#endif > > typedef struct { > int cnt; > @@ -189,7 +192,7 @@ typedef struct portid { > int lid; /* lid or 0 if directed route */ > ib_dr_path_t drpath; > int grh_present; /* flag */ > - ib_gid_t gid; > + ibmad_gid_t gid; > uint32_t qp; > uint32_t qkey; > uint8_t sl; > @@ -777,7 +780,7 @@ uint8_t * sa_call(void *rcvbuf, ib_portid_t *portid, ib_sa_call_t *sa, > unsigned timeout); > uint8_t * sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, > ib_sa_call_t *sa, unsigned timeout); > -int ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, > +int ib_path_query(ibmad_gid_t srcgid, ibmad_gid_t destgid, ib_portid_t *sm_id, > void *buf); /* returns lid */ > > inline static uint8_t * > @@ -799,7 +802,7 @@ int ib_resolve_guid(ib_portid_t *portid, uint64_t *guid, > ib_portid_t *sm_id, int timeout); > int ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, > int dest_type, ib_portid_t *sm_id); > -int ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid); > +int ib_resolve_self(ib_portid_t *portid, int *portnum, ibmad_gid_t *gid); > > /* gs.c */ > uint8_t *perf_classportinfo_query(void *rcvbuf, ib_portid_t *dest, int port, > diff --git a/libibmad/src/resolve.c b/libibmad/src/resolve.c > index d8365b2..b063867 100644 > --- a/libibmad/src/resolve.c > +++ b/libibmad/src/resolve.c > @@ -138,7 +138,7 @@ ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, int dest_type, ib_por > } > > int > -ib_resolve_self(ib_portid_t *portid, int *portnum, ib_gid_t *gid) > +ib_resolve_self(ib_portid_t *portid, int *portnum, ibmad_gid_t *gid) > { > ib_portid_t self = {0}; > uint8_t portinfo[64]; > diff --git a/libibmad/src/sa.c b/libibmad/src/sa.c > index ddbfce3..59ef7de 100644 > --- a/libibmad/src/sa.c > +++ b/libibmad/src/sa.c > @@ -112,7 +112,7 @@ sa_rpc_call(void *ibmad_port, void *rcvbuf, ib_portid_t *portid, > IB_PR_COMPMASK_NUMBPATH) > > int > -ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, ib_portid_t *sm_id, void *buf) > +ib_path_query(ibmad_gid_t srcgid, ibmad_gid_t destgid, ib_portid_t *sm_id, void *buf) > { > int npath; > ib_sa_call_t sa = {0}; From taylor at hpc.ufl.edu Fri Mar 7 05:38:36 2008 From: taylor at hpc.ufl.edu (Charles Taylor) Date: Fri, 7 Mar 2008 08:38:36 -0500 Subject: [ofa-general] iWARP cm_id's In-Reply-To: <20080227091003.0cd4f0ad.weiny2@llnl.gov> References: <47BE16F2.8000507@llnl.gov> <1203689708.8793.159.camel@hrosenstock-ws.xsigo.com> <47C4425A.6050703@llnl.gov> <1204048748.454.90.camel@hrosenstock-ws.xsigo.com> <20080227091003.0cd4f0ad.weiny2@llnl.gov> Message-ID: <73F404BE-7B72-45FF-B89F-6E9073B67AD0@hpc.ufl.edu> The provider layer listen() call is passed a cm_id and a backlog. The accept() call is passed a cm_id, and a conn_param. I'm wondering if there is (or should be) any relationship between the cm_id associated with the creation of the listen service and the cm_ids passed in for subsequent accept() calls resulting from connection requests to the CM listener. Thanks, Charlie Taylor UF HPC Center From skimpiness at bakermedia.com Fri Mar 7 06:10:15 2008 From: skimpiness at bakermedia.com (Baniaga Cuddihee) Date: Fri, 07 Mar 2008 14:10:15 +0000 Subject: [ofa-general] idiocies Message-ID: <5151653194.20080307140536@bakermedia.com> Heya, Real men! Milllions of people accross the world have already tested THIS and ARE making their girlfriendds feel brand new sexual seensations! YOU are the best in bed, aren't you ? Girls! Devvelop your sexual relaationship and get even MORE plleasure! Make your boyfriendd a gift! http://bt89tvlch34m147.blogspot.com Face was occupied by a larger circle, halfgreen they become sources of evil.' the rishis said, unusually stormy applause, in which the ladies philosopher, with the impudence of the devil, train of observances, service book, ceremonies, i saw my coolies high up on a ledge opposite to unexpected show of strength, olga tcherny had wearing his power with conscious authority, mild race. Thy subjects are being greatly afflicted that thou purposest to hurte them, and thei shal exclaim, 'i saw him a little while ago. How has their own. abu sofian and molleima were despatched virtuous conduct? blessed be thou, i desire to body) the large trees and plants, with their flowers earth became strewn with carwarriors and horsemen. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jlentini at netapp.com Fri Mar 7 06:57:38 2008 From: jlentini at netapp.com (James Lentini) Date: Fri, 7 Mar 2008 09:57:38 -0500 (EST) Subject: [ofa-general] iWARP cm_id's In-Reply-To: <73F404BE-7B72-45FF-B89F-6E9073B67AD0@hpc.ufl.edu> References: <47BE16F2.8000507@llnl.gov> <1203689708.8793.159.camel@hrosenstock-ws.xsigo.com> <47C4425A.6050703@llnl.gov> <1204048748.454.90.camel@hrosenstock-ws.xsigo.com> <20080227091003.0cd4f0ad.weiny2@llnl.gov> <73F404BE-7B72-45FF-B89F-6E9073B67AD0@hpc.ufl.edu> Message-ID: On Fri, 7 Mar 2008, Charles Taylor wrote: > > > The provider layer listen() call is passed a cm_id and a backlog. > The accept() call is passed a cm_id, and a conn_param. I'm > wondering if there is (or should be) any relationship between the > cm_id associated with the creation of the listen service and the > cm_ids passed in for subsequent accept() calls resulting from > connection requests to the CM listener. > > Thanks, > > Charlie Taylor > UF HPC Center I'm not sure if your are using the userspace or kernel API. In either case, you create an rdma_cm_id for listening. When you receive an RDMA_CM_EVENT_CONNECT_REQUEST event, the RDMA CM passes in a new rdma_cm_id for accepting or rejecting the connection. The kernel and userspace APIs differ in how the RDMA_CM_EVENT_CONNECT_REQUEST event is delivered to the application. For an example of how to use the kernel programming API, take a look at the NFS/RDMA server in 2.6.25. There are some examples of how to use the userspace API in the Sean's librdmacm git tree: http://www.openfabrics.org/git/?p=~shefty/librdmacm.git;a=tree From andrea at qumranet.com Fri Mar 7 07:17:22 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Fri, 7 Mar 2008 16:17:22 +0100 Subject: [ofa-general] [PATCH] 2/4 move all invalidate_page outside of PT lock (#v9 was 1/4) In-Reply-To: References: <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> Message-ID: <20080307151722.GD24114@v2.random> On Tue, Mar 04, 2008 at 02:35:21PM -0800, Christoph Lameter wrote: > It is the atomic dead end that we want to avoid. And your patch is exactly > that. Both the invalidate_page and the RCU locks us into this. I preferred to answer with code to avoid any possible misunderstanding (I through already tried to explain with words and I obviously failed miserably if you ended up writing such an erratic weird claim like above ;). This below simple patch invalidates the "invalidate_page" part, the next patch will invalidate the RCU part, and btw in a way that doesn't forbid unregistering the mmu notifiers at runtime (like your brand new EMM does). This is incremental with my #v9. I still ask Andrew/Linus to merge the #v9 patch I posted a few days ago in .25 so KVM/GRU will be 100% covered in a optimal way on all respects and with maximum flexibility for future changes of API (to allow for future methods that may take more than start,end, this was pointed out once by both me and Avi). My #v9 is zero risk for .25 and it sure worth merging now. Then in .26 we'll modify the semantics of the API to be blocking starting with the below patchx. This is a kernel _internal_ API, and we aren't distributions that have to respect kabi here, but even if we were, making methods sleepable is a 100% backwards compatible semantical change, so there's no possible reason to defer the #v9 merging. The changes in .26 will be transparent to any user (even if they don't need to! even if we turn out to be totally wrong about .26 requiring a minor change of API everything will be perfectly fine). Nothing of this is visible to userland so we can change it at any time as we wish. The reason I keep this incremental (unlike your EMM that does everything all at the same time mixed in a single patch) is to decrease the non obviously safe mangling over mm/* during .25. The below patch is simple, but not as obviously safe as s/ptep_clear_flush/ptep_clear_flush_notify/. Signed-off-by: Andrea Arcangeli diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -134,27 +134,6 @@ static inline void mmu_notifier_mm_init( -#define ptep_clear_flush_notify(__vma, __address, __ptep) \ -({ \ - pte_t __pte; \ - struct vm_area_struct *___vma = __vma; \ - unsigned long ___address = __address; \ - __pte = ptep_clear_flush(___vma, ___address, __ptep); \ - mmu_notifier_invalidate_page(___vma->vm_mm, ___address); \ - __pte; \ -}) - -#define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ -({ \ - int __young; \ - struct vm_area_struct *___vma = __vma; \ - unsigned long ___address = __address; \ - __young = ptep_clear_flush_young(___vma, ___address, __ptep); \ - __young |= mmu_notifier_clear_flush_young(___vma->vm_mm, \ - ___address); \ - __young; \ -}) - #else /* CONFIG_MMU_NOTIFIER */ static inline void mmu_notifier_release(struct mm_struct *mm) @@ -186,9 +165,6 @@ static inline void mmu_notifier_mm_init( { } -#define ptep_clear_flush_young_notify ptep_clear_flush_young -#define ptep_clear_flush_notify ptep_clear_flush - #endif /* CONFIG_MMU_NOTIFIER */ #endif /* _LINUX_MMU_NOTIFIER_H */ diff --git a/mm/filemap_xip.c b/mm/filemap_xip.c --- a/mm/filemap_xip.c +++ b/mm/filemap_xip.c @@ -194,11 +194,13 @@ __xip_unmap (struct address_space * mapp if (pte) { /* Nuke the page table entry. */ flush_cache_page(vma, address, pte_pfn(*pte)); - pteval = ptep_clear_flush_notify(vma, address, pte); + pteval = ptep_clear_flush(vma, address, pte); page_remove_rmap(page, vma); dec_mm_counter(mm, file_rss); BUG_ON(pte_dirty(pteval)); pte_unmap_unlock(pte, ptl); + /* must invalidate_page _before_ freeing the page */ + mmu_notifier_invalidate_page(mm, address); page_cache_release(page); } } diff --git a/mm/memory.c b/mm/memory.c --- a/mm/memory.c +++ b/mm/memory.c @@ -1626,9 +1626,10 @@ static int do_wp_page(struct mm_struct * */ page_table = pte_offset_map_lock(mm, pmd, address, &ptl); - page_cache_release(old_page); + new_page = NULL; if (!pte_same(*page_table, orig_pte)) goto unlock; + page_cache_release(old_page); page_mkwrite = 1; } @@ -1644,6 +1645,7 @@ static int do_wp_page(struct mm_struct * if (ptep_set_access_flags(vma, address, page_table, entry,1)) update_mmu_cache(vma, address, entry); ret |= VM_FAULT_WRITE; + old_page = new_page = NULL; goto unlock; } @@ -1688,7 +1690,7 @@ gotten: * seen in the presence of one thread doing SMC and another * thread doing COW. */ - ptep_clear_flush_notify(vma, address, page_table); + ptep_clear_flush(vma, address, page_table); set_pte_at(mm, address, page_table, entry); update_mmu_cache(vma, address, entry); lru_cache_add_active(new_page); @@ -1700,12 +1702,18 @@ gotten: } else mem_cgroup_uncharge_page(new_page); - if (new_page) +unlock: + pte_unmap_unlock(page_table, ptl); + + if (new_page) { + if (new_page == old_page) + /* cow happened, notify before releasing old_page */ + mmu_notifier_invalidate_page(mm, address); page_cache_release(new_page); + } if (old_page) page_cache_release(old_page); -unlock: - pte_unmap_unlock(page_table, ptl); + if (dirty_page) { if (vma->vm_file) file_update_time(vma->vm_file); diff --git a/mm/rmap.c b/mm/rmap.c --- a/mm/rmap.c +++ b/mm/rmap.c @@ -275,7 +275,7 @@ static int page_referenced_one(struct pa unsigned long address; pte_t *pte; spinlock_t *ptl; - int referenced = 0; + int referenced = 0, clear_flush_young = 0; address = vma_address(page, vma); if (address == -EFAULT) @@ -288,8 +288,11 @@ static int page_referenced_one(struct pa if (vma->vm_flags & VM_LOCKED) { referenced++; *mapcount = 1; /* break early from loop */ - } else if (ptep_clear_flush_young_notify(vma, address, pte)) - referenced++; + } else { + clear_flush_young = 1; + if (ptep_clear_flush_young(vma, address, pte)) + referenced++; + } /* Pretend the page is referenced if the task has the swap token and is in the middle of a page fault. */ @@ -299,6 +302,10 @@ static int page_referenced_one(struct pa (*mapcount)--; pte_unmap_unlock(pte, ptl); + + if (clear_flush_young) + referenced += mmu_notifier_clear_flush_young(mm, address); + out: return referenced; } @@ -455,7 +462,7 @@ static int page_mkclean_one(struct page pte_t entry; flush_cache_page(vma, address, pte_pfn(*pte)); - entry = ptep_clear_flush_notify(vma, address, pte); + entry = ptep_clear_flush(vma, address, pte); entry = pte_wrprotect(entry); entry = pte_mkclean(entry); set_pte_at(mm, address, pte, entry); @@ -463,6 +470,10 @@ static int page_mkclean_one(struct page } pte_unmap_unlock(pte, ptl); + + if (ret) + mmu_notifier_invalidate_page(mm, address); + out: return ret; } @@ -712,15 +723,14 @@ static int try_to_unmap_one(struct page * If it's recently referenced (perhaps page_referenced * skipped over this mm) then we should reactivate it. */ - if (!migration && ((vma->vm_flags & VM_LOCKED) || - (ptep_clear_flush_young_notify(vma, address, pte)))) { + if (!migration && (vma->vm_flags & VM_LOCKED)) { ret = SWAP_FAIL; goto out_unmap; } /* Nuke the page table entry. */ flush_cache_page(vma, address, page_to_pfn(page)); - pteval = ptep_clear_flush_notify(vma, address, pte); + pteval = ptep_clear_flush(vma, address, pte); /* Move the dirty bit to the physical page now the pte is gone. */ if (pte_dirty(pteval)) @@ -775,6 +785,8 @@ static int try_to_unmap_one(struct page out_unmap: pte_unmap_unlock(pte, ptl); + if (ret != SWAP_FAIL) + mmu_notifier_invalidate_page(mm, address); out: return ret; } @@ -813,7 +825,7 @@ static void try_to_unmap_cluster(unsigne spinlock_t *ptl; struct page *page; unsigned long address; - unsigned long end; + unsigned long start, end; address = (vma->vm_start + cursor) & CLUSTER_MASK; end = address + CLUSTER_SIZE; @@ -834,6 +846,8 @@ static void try_to_unmap_cluster(unsigne if (!pmd_present(*pmd)) return; + start = address; + mmu_notifier_invalidate_range_begin(mm, start, end); pte = pte_offset_map_lock(mm, pmd, address, &ptl); /* Update high watermark before we lower rss */ @@ -845,12 +859,12 @@ static void try_to_unmap_cluster(unsigne page = vm_normal_page(vma, address, *pte); BUG_ON(!page || PageAnon(page)); - if (ptep_clear_flush_young_notify(vma, address, pte)) + if (ptep_clear_flush_young(vma, address, pte)) continue; /* Nuke the page table entry. */ flush_cache_page(vma, address, pte_pfn(*pte)); - pteval = ptep_clear_flush_notify(vma, address, pte); + pteval = ptep_clear_flush(vma, address, pte); /* If nonlinear, store the file page offset in the pte. */ if (page->index != linear_page_index(vma, address)) @@ -866,6 +880,7 @@ static void try_to_unmap_cluster(unsigne (*mapcount)--; } pte_unmap_unlock(pte - 1, ptl); + mmu_notifier_invalidate_range_end(mm, start, end); } static int try_to_unmap_anon(struct page *page, int migration) From andrea at qumranet.com Fri Mar 7 07:23:28 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Fri, 7 Mar 2008 16:23:28 +0100 Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <20080307151722.GD24114@v2.random> References: <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> Message-ID: <20080307152328.GE24114@v2.random> This combines the non-sleep-capable RCU locking of #v9 with a seqlock so the mmu notifier fast path will require zero cacheline writes/bouncing while still providing mmu_notifier_unregister and allowing to schedule inside the mmu notifier methods. If we drop mmu_notifier_unregister we can as well drop all seqlock and rcu_read_lock()s. But this locking scheme combination is sexy enough and 100% scalable (the mmu_notifier_list cacheline will be preloaded anyway and that will most certainly include the sequence number value in l1 for free even in Christoph's NUMA systems) so IMHO it worth to keep mmu_notifier_unregister. Signed-off-by: Andrea Arcangeli diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -230,6 +231,7 @@ struct mm_struct { #endif #ifdef CONFIG_MMU_NOTIFIER struct hlist_head mmu_notifier_list; + seqlock_t mmu_notifier_lock; #endif }; diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -130,6 +130,7 @@ static inline void mmu_notifier_mm_init( static inline void mmu_notifier_mm_init(struct mm_struct *mm) { INIT_HLIST_HEAD(&mm->mmu_notifier_list); + seqlock_init(&mm->mmu_notifier_lock); } diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -20,7 +20,9 @@ void __mmu_notifier_release(struct mm_st void __mmu_notifier_release(struct mm_struct *mm) { struct mmu_notifier *mn; + unsigned seq; + seq = read_seqbegin(&mm->mmu_notifier_lock); while (unlikely(!hlist_empty(&mm->mmu_notifier_list))) { mn = hlist_entry(mm->mmu_notifier_list.first, struct mmu_notifier, @@ -28,6 +30,7 @@ void __mmu_notifier_release(struct mm_st hlist_del(&mn->hlist); if (mn->ops->release) mn->ops->release(mn, mm); + BUG_ON(read_seqretry(&mm->mmu_notifier_lock, seq)); } } @@ -42,11 +45,19 @@ int __mmu_notifier_clear_flush_young(str struct mmu_notifier *mn; struct hlist_node *n; int young = 0; + unsigned seq; rcu_read_lock(); +restart: + seq = read_seqbegin(&mm->mmu_notifier_lock); hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { - if (mn->ops->clear_flush_young) + if (mn->ops->clear_flush_young) { + rcu_read_unlock(); young |= mn->ops->clear_flush_young(mn, mm, address); + rcu_read_lock(); + } + if (read_seqretry(&mm->mmu_notifier_lock, seq)) + goto restart; } rcu_read_unlock(); @@ -58,11 +69,19 @@ void __mmu_notifier_invalidate_page(stru { struct mmu_notifier *mn; struct hlist_node *n; + unsigned seq; rcu_read_lock(); +restart: + seq = read_seqbegin(&mm->mmu_notifier_lock); hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { - if (mn->ops->invalidate_page) + if (mn->ops->invalidate_page) { + rcu_read_unlock(); mn->ops->invalidate_page(mn, mm, address); + rcu_read_lock(); + } + if (read_seqretry(&mm->mmu_notifier_lock, seq)) + goto restart; } rcu_read_unlock(); } @@ -72,11 +91,19 @@ void __mmu_notifier_invalidate_range_beg { struct mmu_notifier *mn; struct hlist_node *n; + unsigned seq; rcu_read_lock(); +restart: + seq = read_seqbegin(&mm->mmu_notifier_lock); hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { - if (mn->ops->invalidate_range_begin) + if (mn->ops->invalidate_range_begin) { + rcu_read_unlock(); mn->ops->invalidate_range_begin(mn, mm, start, end); + rcu_read_lock(); + } + if (read_seqretry(&mm->mmu_notifier_lock, seq)) + goto restart; } rcu_read_unlock(); } @@ -86,11 +113,19 @@ void __mmu_notifier_invalidate_range_end { struct mmu_notifier *mn; struct hlist_node *n; + unsigned seq; rcu_read_lock(); +restart: + seq = read_seqbegin(&mm->mmu_notifier_lock); hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { - if (mn->ops->invalidate_range_end) + if (mn->ops->invalidate_range_end) { + rcu_read_unlock(); mn->ops->invalidate_range_end(mn, mm, start, end); + rcu_read_lock(); + } + if (read_seqretry(&mm->mmu_notifier_lock, seq)) + goto restart; } rcu_read_unlock(); } @@ -103,12 +138,20 @@ void __mmu_notifier_invalidate_range_end */ void mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm) { + /* no need of seqlock for hlist_add_head_rcu */ hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_list); } EXPORT_SYMBOL_GPL(mmu_notifier_register); void mmu_notifier_unregister(struct mmu_notifier *mn, struct mm_struct *mm) { + /* + * The seqlock tracks if a hlist_del_rcu happens while a + * notifier method is scheduling and in such a case the "mn" + * memory may have been freed by the time the method returns. + */ + write_seqlock(&mm->mmu_notifier_lock); hlist_del_rcu(&mn->hlist); + write_sequnlock(&mm->mmu_notifier_lock); } EXPORT_SYMBOL_GPL(mmu_notifier_unregister); From andrea at qumranet.com Fri Mar 7 07:52:44 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Fri, 7 Mar 2008 16:52:44 +0100 Subject: [ofa-general] [PATCH] 4/4 i_mmap_lock spinlock2rwsem (#v9 was 1/4) In-Reply-To: <20080307152328.GE24114@v2.random> References: <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> Message-ID: <20080307155244.GF24114@v2.random> This is a rediff of Christoph's plain i_mmap_lock2rwsem patch on top of #v9 1/4 + 2/4 + 3/4 (hence this is called 4/4). This is mostly to show that after 3/4, any patch that plugs on the EMM patchset will plug nicely on top of my MMU notifer patchset too. The patch trigger bug checks here in modprobe: BUG_ON(mm->nr_ptes > (FIRST_USER_ADDRESS+PMD_SIZE-1)>>PMD_SHIFT); kjournald starting. Commit interval 5 seconds EXT3-fs: mounted filesystem with ordered data mode. VFS: Mounted root (ext3 filesystem) readonly. Freeing unused kernel memory: 252k freed ------------[ cut here ]------------ kernel BUG at mm/mmap.c:2063! invalid opcode: 0000 [1] SMP CPU 0 Modules linked in: Pid: 1123, comm: modprobe.sh Not tainted 2.6.25-rc3 #22 RIP: 0010:[] [] exit_mmap+0xef/0xfa RSP: 0000:ffff81003c79bed8 EFLAGS: 00010206 RAX: 0000000000000000 RBX: ffff810001004840 RCX: ffff81003c79bee0 RDX: 0000000000000000 RSI: ffff81003c5e8918 RDI: ffff81003d8048c0 RBP: 0000000000000000 R08: 0000000000000008 R09: ffff810002c00040 R10: 0000000000000002 R11: ffff810001009180 R12: ffff81003c57b800 R13: 0000000000000000 R14: 00000000005f0db0 R15: 00007fff3f2af234 FS: 00007f283714b6f0(0000) GS:ffffffff80694000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000458f40 CR3: 0000000000201000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process modprobe.sh (pid: 1123, threadinfo ffff81003c79a000, task ffff81003cf9ca50) Stack: 0000000000000091 ffff810001004840 ffff81003c57b800 ffff81003c57b880 0000000000000000 ffffffff8022f7bf 0000000000000001 0000000000000001 ffff81003cf9ca50 ffffffff802349b6 0000000000000292 ffffffff80354c63 Call Trace: [] mmput+0x30/0x9d [] do_exit+0x223/0x66c [] __up_read+0x13/0x8a [] do_group_exit+0x6f/0x8a [] system_call_after_swapgs+0x7b/0x80 Code: 7b 18 e8 4a 5c 00 00 c7 43 08 00 00 00 00 eb 0b 48 89 ef e8 d1 fe ff ff 48 89 c5 48 85 ed 75 f0 49 modprobe.sh[1114]: segfault at 0 ip 7f998d2e972b sp 7fff959d8ed0 error 4 in libc-2.6.1.so[7f998d27b000+136000] I didn't look into this but it shows how it would be risky to make this change in .25. It's a bit strange that the bugcheck triggers given I've preempt disabled (I mean CONFIG_PREEMPT_VOLUNTARY=y, nobody should turn off that config option) and so even if code depended on the implicit preempt_disable in spin_lock, no race should happen. The down_read sections at first glance didn't seem capable of altering nr_ptes, but I didn't look seriously into the above. I rediffed it just to be 100% on par with EMM sleep-capabilities (but while retaining more features and cleaner code I hope). ------------------ From: Christoph Lameter Subject: Conversion of i_mmap_lock to semaphore Not there but the system boots and is usable. Complains about atomic contexts because the tlb functions use a get_cpu() and thus disable preempt. Not sure yet what to do about the cond_resched_lock stuff etc. Convert i_mmap_lock to i_mmap_sem The conversion to a rwsemaphore allows callbacks during rmap traversal for files in a non atomic context. A rw style lock allows concurrent walking of the reverse map. Signed-off-by: Christoph Lameter --- arch/x86/mm/hugetlbpage.c | 4 ++-- fs/hugetlbfs/inode.c | 4 ++-- fs/inode.c | 2 +- include/linux/fs.h | 2 +- include/linux/mm.h | 2 +- kernel/fork.c | 4 ++-- mm/filemap.c | 8 ++++---- mm/filemap_xip.c | 4 ++-- mm/fremap.c | 4 ++-- mm/hugetlb.c | 11 +++++------ mm/memory.c | 28 ++++++++-------------------- mm/migrate.c | 4 ++-- mm/mmap.c | 16 ++++++++-------- mm/mremap.c | 4 ++-- mm/rmap.c | 20 +++++++++----------- 15 files changed, 51 insertions(+), 66 deletions(-) diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -69,7 +69,7 @@ static void huge_pmd_share(struct mm_str if (!vma_shareable(vma, addr)) return; - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(svma, &iter, &mapping->i_mmap, idx, idx) { if (svma == vma) continue; @@ -94,7 +94,7 @@ static void huge_pmd_share(struct mm_str put_page(virt_to_page(spte)); spin_unlock(&mm->page_table_lock); out: - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); } /* diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -454,10 +454,10 @@ static int hugetlb_vmtruncate(struct ino pgoff = offset >> PAGE_SHIFT; i_size_write(inode, offset); - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); if (!prio_tree_empty(&mapping->i_mmap)) hugetlb_vmtruncate_list(&mapping->i_mmap, pgoff); - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); truncate_hugepages(inode, offset); return 0; } diff --git a/fs/inode.c b/fs/inode.c --- a/fs/inode.c +++ b/fs/inode.c @@ -210,7 +210,7 @@ void inode_init_once(struct inode *inode INIT_LIST_HEAD(&inode->i_devices); INIT_RADIX_TREE(&inode->i_data.page_tree, GFP_ATOMIC); rwlock_init(&inode->i_data.tree_lock); - spin_lock_init(&inode->i_data.i_mmap_lock); + init_rwsem(&inode->i_data.i_mmap_sem); INIT_LIST_HEAD(&inode->i_data.private_list); spin_lock_init(&inode->i_data.private_lock); INIT_RAW_PRIO_TREE_ROOT(&inode->i_data.i_mmap); diff --git a/include/linux/fs.h b/include/linux/fs.h --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -503,7 +503,7 @@ struct address_space { unsigned int i_mmap_writable;/* count VM_SHARED mappings */ struct prio_tree_root i_mmap; /* tree of private and shared mappings */ struct list_head i_mmap_nonlinear;/*list VM_NONLINEAR mappings */ - spinlock_t i_mmap_lock; /* protect tree, count, list */ + struct rw_semaphore i_mmap_sem; /* protect tree, count, list */ unsigned int truncate_count; /* Cover race condition with truncate */ unsigned long nrpages; /* number of total pages */ pgoff_t writeback_index;/* writeback starts here */ diff --git a/include/linux/mm.h b/include/linux/mm.h --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -709,7 +709,7 @@ struct zap_details { struct address_space *check_mapping; /* Check page->mapping if set */ pgoff_t first_index; /* Lowest page->index to unmap */ pgoff_t last_index; /* Highest page->index to unmap */ - spinlock_t *i_mmap_lock; /* For unmap_mapping_range: */ + struct rw_semaphore *i_mmap_sem; /* For unmap_mapping_range: */ unsigned long truncate_count; /* Compare vm_truncate_count */ }; diff --git a/kernel/fork.c b/kernel/fork.c --- a/kernel/fork.c +++ b/kernel/fork.c @@ -274,12 +274,12 @@ static int dup_mmap(struct mm_struct *mm atomic_dec(&inode->i_writecount); /* insert tmp into the share list, just after mpnt */ - spin_lock(&file->f_mapping->i_mmap_lock); + down_write(&file->f_mapping->i_mmap_sem); tmp->vm_truncate_count = mpnt->vm_truncate_count; flush_dcache_mmap_lock(file->f_mapping); vma_prio_tree_add(tmp, mpnt); flush_dcache_mmap_unlock(file->f_mapping); - spin_unlock(&file->f_mapping->i_mmap_lock); + up_write(&file->f_mapping->i_mmap_sem); } /* diff --git a/mm/filemap.c b/mm/filemap.c --- a/mm/filemap.c +++ b/mm/filemap.c @@ -62,16 +62,16 @@ generic_file_direct_IO(int rw, struct ki /* * Lock ordering: * - * ->i_mmap_lock (vmtruncate) + * ->i_mmap_sem (vmtruncate) * ->private_lock (__free_pte->__set_page_dirty_buffers) * ->swap_lock (exclusive_swap_page, others) * ->mapping->tree_lock * * ->i_mutex - * ->i_mmap_lock (truncate->unmap_mapping_range) + * ->i_mmap_sem (truncate->unmap_mapping_range) * * ->mmap_sem - * ->i_mmap_lock + * ->i_mmap_sem * ->page_table_lock or pte_lock (various, mainly in memory.c) * ->mapping->tree_lock (arch-dependent flush_dcache_mmap_lock) * @@ -88,7 +88,7 @@ generic_file_direct_IO(int rw, struct ki * ->sb_lock (fs/fs-writeback.c) * ->mapping->tree_lock (__sync_single_inode) * - * ->i_mmap_lock + * ->i_mmap_sem * ->anon_vma.lock (vma_adjust) * * ->anon_vma.lock diff --git a/mm/filemap_xip.c b/mm/filemap_xip.c --- a/mm/filemap_xip.c +++ b/mm/filemap_xip.c @@ -184,7 +184,7 @@ __xip_unmap (struct address_space * mapp if (!page) return; - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) { mm = vma->vm_mm; address = vma->vm_start + @@ -204,7 +204,7 @@ __xip_unmap (struct address_space * mapp page_cache_release(page); } } - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); } /* diff --git a/mm/fremap.c b/mm/fremap.c --- a/mm/fremap.c +++ b/mm/fremap.c @@ -206,13 +206,13 @@ asmlinkage long sys_remap_file_pages(uns } goto out; } - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); flush_dcache_mmap_lock(mapping); vma->vm_flags |= VM_NONLINEAR; vma_prio_tree_remove(vma, &mapping->i_mmap); vma_nonlinear_insert(vma, &mapping->i_mmap_nonlinear); flush_dcache_mmap_unlock(mapping); - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); } mmu_notifier_invalidate_range_begin(mm, start, start + size); diff --git a/mm/hugetlb.c b/mm/hugetlb.c --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -746,7 +746,7 @@ void __unmap_hugepage_range(struct vm_ar struct page *page; struct page *tmp; /* - * A page gathering list, protected by per file i_mmap_lock. The + * A page gathering list, protected by per file i_mmap_sem. The * lock is used to avoid list corruption from multiple unmapping * of the same page since we are using page->lru. */ @@ -796,9 +796,9 @@ void unmap_hugepage_range(struct vm_area * do nothing in this case. */ if (vma->vm_file) { - spin_lock(&vma->vm_file->f_mapping->i_mmap_lock); + down_write(&vma->vm_file->f_mapping->i_mmap_sem); __unmap_hugepage_range(vma, start, end); - spin_unlock(&vma->vm_file->f_mapping->i_mmap_lock); + up_write(&vma->vm_file->f_mapping->i_mmap_sem); } } @@ -1041,7 +1041,7 @@ void hugetlb_change_protection(struct vm BUG_ON(address >= end); flush_cache_range(vma, address, end); - spin_lock(&vma->vm_file->f_mapping->i_mmap_lock); + down_read(&vma->vm_file->f_mapping->i_mmap_sem); spin_lock(&mm->page_table_lock); for (; address < end; address += HPAGE_SIZE) { ptep = huge_pte_offset(mm, address); @@ -1056,7 +1056,7 @@ void hugetlb_change_protection(struct vm } } spin_unlock(&mm->page_table_lock); - spin_unlock(&vma->vm_file->f_mapping->i_mmap_lock); + up_read(&vma->vm_file->f_mapping->i_mmap_sem); flush_tlb_range(vma, start, end); } diff --git a/mm/memory.c b/mm/memory.c --- a/mm/memory.c +++ b/mm/memory.c @@ -832,7 +832,6 @@ unsigned long unmap_vmas(struct mmu_gath unsigned long tlb_start = 0; /* For tlb_finish_mmu */ int tlb_start_valid = 0; unsigned long start = start_addr; - spinlock_t *i_mmap_lock = details? details->i_mmap_lock: NULL; int fullmm = (*tlbp)->fullmm; for ( ; vma && vma->vm_start < end_addr; vma = vma->vm_next) { @@ -870,21 +869,11 @@ unsigned long unmap_vmas(struct mmu_gath tlb_finish_mmu(*tlbp, tlb_start, start); - if (need_resched() || - (i_mmap_lock && spin_needbreak(i_mmap_lock))) { - if (i_mmap_lock) { - *tlbp = NULL; - goto out; - } - cond_resched(); - } - *tlbp = tlb_gather_mmu(vma->vm_mm, fullmm); tlb_start_valid = 0; zap_work = ZAP_BLOCK_SIZE; } } -out: return start; /* which is now the end (or restart) address */ } @@ -1746,7 +1735,7 @@ unwritable_page: /* * Helper functions for unmap_mapping_range(). * - * __ Notes on dropping i_mmap_lock to reduce latency while unmapping __ + * __ Notes on dropping i_mmap_sem to reduce latency while unmapping __ * * We have to restart searching the prio_tree whenever we drop the lock, * since the iterator is only valid while the lock is held, and anyway @@ -1765,7 +1754,7 @@ unwritable_page: * can't efficiently keep all vmas in step with mapping->truncate_count: * so instead reset them all whenever it wraps back to 0 (then go to 1). * mapping->truncate_count and vma->vm_truncate_count are protected by - * i_mmap_lock. + * i_mmap_sem. * * In order to make forward progress despite repeatedly restarting some * large vma, note the restart_addr from unmap_vmas when it breaks out: @@ -1815,7 +1804,7 @@ again: restart_addr = zap_page_range(vma, start_addr, end_addr - start_addr, details); - need_break = need_resched() || spin_needbreak(details->i_mmap_lock); + need_break = need_resched(); if (restart_addr >= end_addr) { /* We have now completed this vma: mark it so */ @@ -1829,9 +1818,9 @@ again: goto again; } - spin_unlock(details->i_mmap_lock); + up_write(details->i_mmap_sem); cond_resched(); - spin_lock(details->i_mmap_lock); + down_write(details->i_mmap_sem); return -EINTR; } @@ -1925,9 +1914,9 @@ void unmap_mapping_range(struct address_ details.last_index = hba + hlen - 1; if (details.last_index < details.first_index) details.last_index = ULONG_MAX; - details.i_mmap_lock = &mapping->i_mmap_lock; + details.i_mmap_sem = &mapping->i_mmap_sem; - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); /* Protect against endless unmapping loops */ mapping->truncate_count++; @@ -1942,7 +1931,7 @@ void unmap_mapping_range(struct address_ unmap_mapping_range_tree(&mapping->i_mmap, &details); if (unlikely(!list_empty(&mapping->i_mmap_nonlinear))) unmap_mapping_range_list(&mapping->i_mmap_nonlinear, &details); - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); } EXPORT_SYMBOL(unmap_mapping_range); diff --git a/mm/migrate.c b/mm/migrate.c --- a/mm/migrate.c +++ b/mm/migrate.c @@ -202,12 +202,12 @@ static void remove_file_migration_ptes(s if (!mapping) return; - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) remove_migration_pte(vma, old, new); - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); } /* diff --git a/mm/mmap.c b/mm/mmap.c --- a/mm/mmap.c +++ b/mm/mmap.c @@ -187,7 +187,7 @@ error: } /* - * Requires inode->i_mapping->i_mmap_lock + * Requires inode->i_mapping->i_mmap_sem */ static void __remove_shared_vm_struct(struct vm_area_struct *vma, struct file *file, struct address_space *mapping) @@ -215,9 +215,9 @@ void unlink_file_vma(struct vm_area_stru if (file) { struct address_space *mapping = file->f_mapping; - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); __remove_shared_vm_struct(vma, file, mapping); - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); } } @@ -440,7 +440,7 @@ static void vma_link(struct mm_struct *m mapping = vma->vm_file->f_mapping; if (mapping) { - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); vma->vm_truncate_count = mapping->truncate_count; } anon_vma_lock(vma); @@ -450,7 +450,7 @@ static void vma_link(struct mm_struct *m anon_vma_unlock(vma); if (mapping) - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); mm->map_count++; validate_mm(mm); @@ -537,7 +537,7 @@ again: remove_next = 1 + (end > next-> mapping = file->f_mapping; if (!(vma->vm_flags & VM_NONLINEAR)) root = &mapping->i_mmap; - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); if (importer && vma->vm_truncate_count != next->vm_truncate_count) { /* @@ -621,7 +621,7 @@ again: remove_next = 1 + (end > next-> if (anon_vma) spin_unlock(&anon_vma->lock); if (mapping) - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); if (remove_next) { if (file) @@ -2065,7 +2065,7 @@ void exit_mmap(struct mm_struct *mm) /* Insert vm structure into process list sorted by address * and into the inode's i_mmap tree. If vm_file is non-NULL - * then i_mmap_lock is taken here. + * then i_mmap_sem is taken here. */ int insert_vm_struct(struct mm_struct * mm, struct vm_area_struct * vma) { diff --git a/mm/mremap.c b/mm/mremap.c --- a/mm/mremap.c +++ b/mm/mremap.c @@ -85,7 +85,7 @@ static void move_ptes(struct vm_area_str * and we propagate stale pages into the dst afterward. */ mapping = vma->vm_file->f_mapping; - spin_lock(&mapping->i_mmap_lock); + down_write(&mapping->i_mmap_sem); if (new_vma->vm_truncate_count && new_vma->vm_truncate_count != vma->vm_truncate_count) new_vma->vm_truncate_count = 0; @@ -121,7 +121,7 @@ static void move_ptes(struct vm_area_str pte_unmap_nested(new_pte - 1); pte_unmap_unlock(old_pte - 1, old_ptl); if (mapping) - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); } #define LATENCY_LIMIT (64 * PAGE_SIZE) diff --git a/mm/rmap.c b/mm/rmap.c --- a/mm/rmap.c +++ b/mm/rmap.c @@ -24,7 +24,7 @@ * inode->i_alloc_sem (vmtruncate_range) * mm->mmap_sem * page->flags PG_locked (lock_page) - * mapping->i_mmap_lock + * mapping->i_mmap_sem * anon_vma->lock * mm->page_table_lock or pte_lock * zone->lru_lock (in mark_page_accessed, isolate_lru_page) @@ -372,14 +372,14 @@ static int page_referenced_file(struct p * The page lock not only makes sure that page->mapping cannot * suddenly be NULLified by truncation, it makes sure that the * structure at mapping cannot be freed and reused yet, - * so we can safely take mapping->i_mmap_lock. + * so we can safely take mapping->i_mmap_sem. */ BUG_ON(!PageLocked(page)); - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); /* - * i_mmap_lock does not stabilize mapcount at all, but mapcount + * i_mmap_sem does not stabilize mapcount at all, but mapcount * is more likely to be accurate if we note it after spinning. */ mapcount = page_mapcount(page); @@ -402,7 +402,7 @@ static int page_referenced_file(struct p break; } - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); return referenced; } @@ -487,12 +487,12 @@ static int page_mkclean_file(struct addr BUG_ON(PageAnon(page)); - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) { if (vma->vm_flags & VM_SHARED) ret += page_mkclean_one(page, vma); } - spin_unlock(&mapping->i_mmap_lock); + up_read(&mapping->i_mmap_sem); return ret; } @@ -924,7 +924,7 @@ static int try_to_unmap_file(struct page unsigned long max_nl_size = 0; unsigned int mapcount; - spin_lock(&mapping->i_mmap_lock); + down_read(&mapping->i_mmap_sem); vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) { ret = try_to_unmap_one(page, vma, migration); if (ret == SWAP_FAIL || !page_mapped(page)) @@ -961,7 +961,6 @@ static int try_to_unmap_file(struct page mapcount = page_mapcount(page); if (!mapcount) goto out; - cond_resched_lock(&mapping->i_mmap_lock); max_nl_size = (max_nl_size + CLUSTER_SIZE - 1) & CLUSTER_MASK; if (max_nl_cursor == 0) @@ -983,7 +982,6 @@ static int try_to_unmap_file(struct page } vma->vm_private_data = (void *) max_nl_cursor; } - cond_resched_lock(&mapping->i_mmap_lock); max_nl_cursor += CLUSTER_SIZE; } while (max_nl_cursor <= max_nl_size); @@ -995,7 +993,7 @@ static int try_to_unmap_file(struct page list_for_each_entry(vma, &mapping->i_mmap_nonlinear, shared.vm_set.list) vma->vm_private_data = NULL; out: - spin_unlock(&mapping->i_mmap_lock); + up_write(&mapping->i_mmap_sem); return ret; } From LeeroyRobbins at todddavid.com Sat Mar 8 08:43:27 2008 From: LeeroyRobbins at todddavid.com (Leeroy Robbins) Date: Fri, 8 Mar 2008 12:43:27 -0400 Subject: [ofa-general] You Don't Need Miracles to Make it Bigger Message-ID: <01c8811a$0160e800$fc6158c8@balexander> Our VPXL has helped many men to make their cock bigger in a safe and effective way. There is no reason why you can't be one of them. Order VPXL and have a pleasure of being confident. http://raceias.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.p.zijlstra at chello.nl Fri Mar 7 08:52:42 2008 From: a.p.zijlstra at chello.nl (Peter Zijlstra) Date: Fri, 07 Mar 2008 17:52:42 +0100 Subject: [ofa-general] ***SPAM*** Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <20080307152328.GE24114@v2.random> References: <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> Message-ID: <1204908762.8514.114.camel@twins> On Fri, 2008-03-07 at 16:23 +0100, Andrea Arcangeli wrote: > @@ -42,11 +45,19 @@ int __mmu_notifier_clear_flush_young(str > struct mmu_notifier *mn; > struct hlist_node *n; > int young = 0; > + unsigned seq; > > rcu_read_lock(); > +restart: > + seq = read_seqbegin(&mm->mmu_notifier_lock); > hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { > - if (mn->ops->clear_flush_young) > + if (mn->ops->clear_flush_young) { hlist_del_rcu(&mn->hlist) > + rcu_read_unlock(); kfree(mn); > young |= mn->ops->clear_flush_young(mn, mm, address); *BANG* > + rcu_read_lock(); > + } > + if (read_seqretry(&mm->mmu_notifier_lock, seq)) > + goto restart; > } > rcu_read_unlock(); From a-angelo at 623.ams.or.at Sat Mar 8 09:28:58 2008 From: a-angelo at 623.ams.or.at (Emma Reilly) Date: Sat, 9 Mar 2008 01:28:58 +0800 Subject: [ofa-general] What are you up to? Message-ID: <01c88184$f23fe900$ceda797d@a-angelo> Hello! I am tired this evening. I am nice girl that would like to chat with you. Email me at Agneta at GreatGolovaSite.com only, because I am using my friend's email to write this. Don't miss some of my naughty pictures. From patrick.latifi at qlogic.com Fri Mar 7 09:37:41 2008 From: patrick.latifi at qlogic.com (Patrick Marchand Latifi) Date: Fri, 07 Mar 2008 09:37:41 -0800 Subject: [ofa-general] [PATCH] [DAPL v1] uDAT: fix reuse of va_list in debugging mode Message-ID: <20080307173741.2135.35188.stgit@b64-10.internal.keyresearch.com> Make sure we reinitialize the va_list since va_list is undefined if a function traverses the va_list with va_arg. This patch fixes the uDAT debugging case when both stdout and syslog output is wanted. Signed-off-by: Patrick Marchand Latifi --- dat/udat/linux/dat_osd.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/dat/udat/linux/dat_osd.c b/dat/udat/linux/dat_osd.c index e49c489..bc66828 100644 --- a/dat/udat/linux/dat_osd.c +++ b/dat/udat/linux/dat_osd.c @@ -116,20 +116,20 @@ dat_os_dbg_print ( { va_list args; - va_start (args, fmt); - if ( DAT_OS_DBG_DEST_STDOUT & g_dbg_dest ) { + va_start (args, fmt); vfprintf (stdout, fmt, args); fflush (stdout); + va_end (args); } if ( DAT_OS_DBG_DEST_SYSLOG & g_dbg_dest ) { + va_start (args, fmt); vsyslog (LOG_USER | LOG_DEBUG, fmt, args); + va_end (args); } - - va_end (args); } } From patrick.latifi at qlogic.com Fri Mar 7 09:39:22 2008 From: patrick.latifi at qlogic.com (Patrick Marchand Latifi) Date: Fri, 07 Mar 2008 09:39:22 -0800 Subject: [ofa-general] [PATCH] [DAPL v2] uDAT: fix reuse of va_list in debugging mode Message-ID: <20080307173922.2266.39882.stgit@b64-10.internal.keyresearch.com> Make sure we reinitialize the va_list since va_list is undefined if a function traverses the va_list with va_arg. This patch fixes the uDAT debugging case when both stdout and syslog output is wanted. Signed-off-by: Patrick Marchand Latifi --- dat/udat/linux/dat_osd.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/dat/udat/linux/dat_osd.c b/dat/udat/linux/dat_osd.c index e49c489..bc66828 100644 --- a/dat/udat/linux/dat_osd.c +++ b/dat/udat/linux/dat_osd.c @@ -116,20 +116,20 @@ dat_os_dbg_print ( { va_list args; - va_start (args, fmt); - if ( DAT_OS_DBG_DEST_STDOUT & g_dbg_dest ) { + va_start (args, fmt); vfprintf (stdout, fmt, args); fflush (stdout); + va_end (args); } if ( DAT_OS_DBG_DEST_SYSLOG & g_dbg_dest ) { + va_start (args, fmt); vsyslog (LOG_USER | LOG_DEBUG, fmt, args); + va_end (args); } - - va_end (args); } } From Margaret at kaplancollege.com Fri Mar 7 09:44:44 2008 From: Margaret at kaplancollege.com (Margaret Burt) Date: Fri, 07 Mar 2008 19:44:44 +0200 Subject: [ofa-general] =?iso-8859-1?q?Encouraging_news_for_man=C2=92s_heal?= =?iso-8859-1?q?th!?= Message-ID: <261901c8807a$eddda330$c0a80202@Margaret> Your lassie would be thankful beyond any words, if you increased your package! Act now to become more competitive among other men! http://burituap.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrea at qumranet.com Fri Mar 7 09:50:19 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Fri, 7 Mar 2008 18:50:19 +0100 Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <1204908762.8514.114.camel@twins> References: <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <1204908762.8514.114.camel@twins> Message-ID: <20080307175019.GK24114@v2.random> On Fri, Mar 07, 2008 at 05:52:42PM +0100, Peter Zijlstra wrote: > hlist_del_rcu(&mn->hlist) > > > + rcu_read_unlock(); > > kfree(mn); > > > young |= mn->ops->clear_flush_young(mn, mm, address); > > *BANG* My objective was to allow mmu_notifier_register/unregister to be called with the same mmu notifier object, I didn't mean the object could have been freed until ->release is called. However you reminded me that after unregistering ->release won't be called so unregister isn't very useful and I doubt we can keep it ;). In the meantime I've also been thinking that we could need the write_seqlock in mmu_notifier_register, to know when to restart the loop if somebody does a mmu_notifier_register; synchronize_rcu(). Otherwise there's no way to be sure the mmu notifier will start firing immediately after synchronize_rcu. I'm unsure if it's acceptable that in-progress mmu notifier invocations, don't need to notice the fact that somebody did mmu_notifier_register; synchronize_rcu. If they don't need to notice, then we can just drop unregister and all rcu_read_lock()s instead of adding write_seqlock to the register operation. Overall my effort is to try to avoid expand the list walk with explicit memory barriers like in EMM while trying to be equally efficient. Another issue is that the _begin/_end logic doesn't provide any guarantee that the _begin will start firing before _end, if a kernel module is loaded while another cpu is already running inside some munmap operation etc.. The KVM usage of mmu notifier has no problem with that detail, but KVM doesn't use _begin at all, I wonder if others would have problems. This is a kind of a separate problem, but quite related to the question if the notifiers must be guaranteed to start firing immediately after mmu_notifier_unregister;synchronize_rcu or not, that's why I mentioned it here. Once I get comments on the suggested direction for these details, I'll quickly repost a replacement patch for 3/4. Thanks Peter! From a.p.zijlstra at chello.nl Fri Mar 7 10:01:35 2008 From: a.p.zijlstra at chello.nl (Peter Zijlstra) Date: Fri, 07 Mar 2008 19:01:35 +0100 Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <20080307175019.GK24114@v2.random> References: <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <1204908762.8514.114.camel@twins> <20080307175019.GK24114@v2.random> Message-ID: <1204912895.8514.120.camel@twins> On Fri, 2008-03-07 at 18:50 +0100, Andrea Arcangeli wrote: > Overall my effort is to try to avoid expand the list walk with > explicit memory barriers like in EMM while trying to be equally > efficient. I think we can do with a smb_wmb(); like Christoph (and like hlist_add_rcu()), but replace the smb_rmb() Christoph has with a smp_read_barrier_depends(). That should give much the same results. The reason Christoph can do without RCU is because he doesn't allow unregister, and as soon as you drop that you'll end up with something similar. > Another issue is that the _begin/_end logic doesn't provide any > guarantee that the _begin will start firing before _end, if a kernel > module is loaded while another cpu is already running inside some > munmap operation etc.. The KVM usage of mmu notifier has no problem > with that detail, but KVM doesn't use _begin at all, I wonder if > others would have problems. This is a kind of a separate problem, but > quite related to the question if the notifiers must be guaranteed to > start firing immediately after mmu_notifier_unregister;synchronize_rcu > or not, that's why I mentioned it here. Curious problem indeed. Would it make sense to require registering these MMU notifiers when the process is still single threaded along with the requirement that they can never be removed again from a running process? For KVM this should be quite doable, but I must admit I haven't been paying enough attention to know if its possible for these other users. From ardavis at ichips.intel.com Fri Mar 7 10:14:16 2008 From: ardavis at ichips.intel.com (Arlin Davis) Date: Fri, 07 Mar 2008 10:14:16 -0800 Subject: [ofa-general] Re: [PATCH] [DAPL v1] uDAT: fix reuse of va_list in debugging mode In-Reply-To: <20080307173741.2135.35188.stgit@b64-10.internal.keyresearch.com> References: <20080307173741.2135.35188.stgit@b64-10.internal.keyresearch.com> Message-ID: <47D185F8.7030307@ichips.intel.com> Patrick Marchand Latifi wrote: > Make sure we reinitialize the va_list since va_list is undefined > if a function traverses the va_list with va_arg. > > This patch fixes the uDAT debugging case when both stdout and > syslog output is wanted. > already committed for both v1 and v2. -arlin From patrick.latifi at qlogic.com Fri Mar 7 10:34:37 2008 From: patrick.latifi at qlogic.com (Patrick Marchand Latifi) Date: Fri, 7 Mar 2008 10:34:37 -0800 Subject: [ofa-general] Re: [PATCH] [DAPL v1] uDAT: fix reuse of va_list in debugging mode In-Reply-To: <47D185F8.7030307@ichips.intel.com> References: <20080307173741.2135.35188.stgit@b64-10.internal.keyresearch.com> <47D185F8.7030307@ichips.intel.com> Message-ID: <20080307183437.GJ27519@jet.pathscale.com> Hi Arlin, On Fri, Mar 07, 2008 at 10:14:16AM -0800, Arlin Davis wrote: > Patrick Marchand Latifi wrote: > >Make sure we reinitialize the va_list since va_list is undefined > >if a function traverses the va_list with va_arg. > > > >This patch fixes the uDAT debugging case when both stdout and > >syslog output is wanted. > > > > already committed for both v1 and v2. > > -arlin Perhaps I should have been a bit more explicit since this patch fixes another instance of the same bug in the file (dat/udat/linux/dat_osd.c) The previous patch that got committed was for a different file (dapl/common/dapl_debug.c). -pat From andrea at qumranet.com Fri Mar 7 10:45:52 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Fri, 7 Mar 2008 19:45:52 +0100 Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <1204912895.8514.120.camel@twins> References: <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <1204908762.8514.114.camel@twins> <20080307175019.GK24114@v2.random> <1204912895.8514.120.camel@twins> Message-ID: <20080307184552.GL24114@v2.random> On Fri, Mar 07, 2008 at 07:01:35PM +0100, Peter Zijlstra wrote: > The reason Christoph can do without RCU is because he doesn't allow > unregister, and as soon as you drop that you'll end up with something Not sure to follow, what do you mean "he doesn't allow"? We'll also have to rip unregister regardless after you pointed out the ->release won't be called after calling my mmu_notifier_unregister in 3/4. If you figured out how to retain mmu_notifier_unregister I'm not seeing it anymore. > Curious problem indeed. Would it make sense to require registering these > MMU notifiers when the process is still single threaded along with the > requirement that they can never be removed again from a running process? I'm afraid that won't help much (even if the mmu notifiers users could cope with that restriction like KVM can) because the VM will run concurrently in another CPU despite the task is single threaded. See 2/4 in try_to_unmap_cluster: _start/end are not only invoked in the context of the current task. PS. this problem I pointed out of _end possibly called before _begin is the same for #v9 and EMM V1 as far as I can tell. From andrea at qumranet.com Fri Mar 7 11:12:57 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Fri, 7 Mar 2008 20:12:57 +0100 Subject: [ofa-general] Re: Notifier for Externally Mapped Memory (EMM) V1 In-Reply-To: References: Message-ID: <20080307191257.GN24114@v2.random> On Wed, Mar 05, 2008 at 04:22:11PM -0800, Christoph Lameter wrote: > + if (e->callback) { > + x = e->callback(e, mm, op, start, end); > + if (x) > + return x; [..] > + > + if (emm_notify(mm, emm_referenced, address, address + PAGE_SIZE)) > + referenced++; This has still the same aging bug as in the RFC version. From andrea at qumranet.com Fri Mar 7 11:47:28 2008 From: andrea at qumranet.com (Andrea Arcangeli) Date: Fri, 7 Mar 2008 20:47:28 +0100 Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <20080307184552.GL24114@v2.random> References: <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <1204908762.8514.114.camel@twins> <20080307175019.GK24114@v2.random> <1204912895.8514.120.camel@twins> <20080307184552.GL24114@v2.random> Message-ID: <20080307194728.GP24114@v2.random> On Fri, Mar 07, 2008 at 07:45:52PM +0100, Andrea Arcangeli wrote: > On Fri, Mar 07, 2008 at 07:01:35PM +0100, Peter Zijlstra wrote: > > The reason Christoph can do without RCU is because he doesn't allow > > unregister, and as soon as you drop that you'll end up with something > > Not sure to follow, what do you mean "he doesn't allow"? We'll also > have to rip unregister regardless after you pointed out the ->release > won't be called after calling my mmu_notifier_unregister in 3/4. If > you figured out how to retain mmu_notifier_unregister I'm not seeing > it anymore. Given I don't see other (buggy ;) ways anymore to retain mmu_notifier_unregister, I did like in EMM and I dropped the unregister function. To me it looks like this will be enough and equally efficient as the expanded version in EMM that is not using the highlevel hlist_rcu macros. If you can see any pitfall let me know! Thanks a lot for the help. ------ This is a replacement for the previously posted 3/4, one of the pieces to allow the mmu notifier methods to sleep. Signed-off-by: Andrea Arcangeli diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -70,17 +70,6 @@ static inline int mm_has_notifiers(struc */ extern void mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm); -/* - * Must hold the mmap_sem for write. - * - * RCU is used to traverse the list. A quiescent period needs to pass - * before the "struct mmu_notifier" can be freed. Alternatively it - * can be synchronously freed inside ->release when the list can't - * change anymore and nobody could possibly walk it. - */ -extern void mmu_notifier_unregister(struct mmu_notifier *mn, - struct mm_struct *mm); - extern void __mmu_notifier_release(struct mm_struct *mm); extern int __mmu_notifier_clear_flush_young(struct mm_struct *mm, unsigned long address); diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -43,12 +43,10 @@ int __mmu_notifier_clear_flush_young(str struct hlist_node *n; int young = 0; - rcu_read_lock(); hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { if (mn->ops->clear_flush_young) young |= mn->ops->clear_flush_young(mn, mm, address); } - rcu_read_unlock(); return young; } @@ -59,12 +57,10 @@ void __mmu_notifier_invalidate_page(stru struct mmu_notifier *mn; struct hlist_node *n; - rcu_read_lock(); hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { if (mn->ops->invalidate_page) mn->ops->invalidate_page(mn, mm, address); } - rcu_read_unlock(); } void __mmu_notifier_invalidate_range_begin(struct mm_struct *mm, @@ -73,12 +69,10 @@ void __mmu_notifier_invalidate_range_beg struct mmu_notifier *mn; struct hlist_node *n; - rcu_read_lock(); hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { if (mn->ops->invalidate_range_begin) mn->ops->invalidate_range_begin(mn, mm, start, end); } - rcu_read_unlock(); } void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, @@ -87,12 +81,10 @@ void __mmu_notifier_invalidate_range_end struct mmu_notifier *mn; struct hlist_node *n; - rcu_read_lock(); hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { if (mn->ops->invalidate_range_end) mn->ops->invalidate_range_end(mn, mm, start, end); } - rcu_read_unlock(); } /* @@ -106,9 +98,3 @@ void mmu_notifier_register(struct mmu_no hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_list); } EXPORT_SYMBOL_GPL(mmu_notifier_register); - -void mmu_notifier_unregister(struct mmu_notifier *mn, struct mm_struct *mm) -{ - hlist_del_rcu(&mn->hlist); -} -EXPORT_SYMBOL_GPL(mmu_notifier_unregister); From clameter at sgi.com Fri Mar 7 11:54:32 2008 From: clameter at sgi.com (Christoph Lameter) Date: Fri, 7 Mar 2008 11:54:32 -0800 (PST) Subject: [ofa-general] Re: [PATCH] 2/4 move all invalidate_page outside of PT lock (#v9 was 1/4) In-Reply-To: <20080307151722.GD24114@v2.random> References: <20080227192610.GF28483@v2.random> <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> Message-ID: On Fri, 7 Mar 2008, Andrea Arcangeli wrote: > This below simple patch invalidates the "invalidate_page" part, the > next patch will invalidate the RCU part, and btw in a way that doesn't > forbid unregistering the mmu notifiers at runtime (like your brand new > EMM does). Sounds good. > The reason I keep this incremental (unlike your EMM that does > everything all at the same time mixed in a single patch) is to > decrease the non obviously safe mangling over mm/* during .25. The > below patch is simple, but not as obviously safe as > s/ptep_clear_flush/ptep_clear_flush_notify/. There was never a chance to merge for .25. Lets drop that and focus on a solution that is good for all. > #endif /* _LINUX_MMU_NOTIFIER_H */ > diff --git a/mm/filemap_xip.c b/mm/filemap_xip.c > --- a/mm/filemap_xip.c > +++ b/mm/filemap_xip.c > @@ -194,11 +194,13 @@ __xip_unmap (struct address_space * mapp > if (pte) { > /* Nuke the page table entry. */ > flush_cache_page(vma, address, pte_pfn(*pte)); > - pteval = ptep_clear_flush_notify(vma, address, pte); > + pteval = ptep_clear_flush(vma, address, pte); > page_remove_rmap(page, vma); > dec_mm_counter(mm, file_rss); > BUG_ON(pte_dirty(pteval)); > pte_unmap_unlock(pte, ptl); > + /* must invalidate_page _before_ freeing the page */ > + mmu_notifier_invalidate_page(mm, address); > page_cache_release(page); > } > } Ok but we still hold the i_mmap_lock here. > @@ -834,6 +846,8 @@ static void try_to_unmap_cluster(unsigne > if (!pmd_present(*pmd)) > return; > > + start = address; > + mmu_notifier_invalidate_range_begin(mm, start, end); Hmmmm.. Okay you going for range invalidate here like EMM but there are still some invalidate_pages() left. From clameter at sgi.com Fri Mar 7 12:00:26 2008 From: clameter at sgi.com (Christoph Lameter) Date: Fri, 7 Mar 2008 12:00:26 -0800 (PST) Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <20080307152328.GE24114@v2.random> References: <20080302155457.GK8091@v2.random> <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> Message-ID: On Fri, 7 Mar 2008, Andrea Arcangeli wrote: > This combines the non-sleep-capable RCU locking of #v9 with a seqlock > so the mmu notifier fast path will require zero cacheline > writes/bouncing while still providing mmu_notifier_unregister and > allowing to schedule inside the mmu notifier methods. If we drop > mmu_notifier_unregister we can as well drop all seqlock and > rcu_read_lock()s. But this locking scheme combination is sexy enough > and 100% scalable (the mmu_notifier_list cacheline will be preloaded > anyway and that will most certainly include the sequence number value > in l1 for free even in Christoph's NUMA systems) so IMHO it worth to > keep mmu_notifier_unregister. Well its adds lots of processing. Not sure if its really worth it. Seems that this scheme cannot work since the existence of the structure passed to the callbacks is not guaranteed since the RCU locks are not held. You need some kind of a refcount to give the existence guarantee. > diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c > --- a/mm/mmu_notifier.c > +++ b/mm/mmu_notifier.c > @@ -20,7 +20,9 @@ void __mmu_notifier_release(struct mm_st > void __mmu_notifier_release(struct mm_struct *mm) > { > struct mmu_notifier *mn; > + unsigned seq; > > + seq = read_seqbegin(&mm->mmu_notifier_lock); > while (unlikely(!hlist_empty(&mm->mmu_notifier_list))) { > mn = hlist_entry(mm->mmu_notifier_list.first, > struct mmu_notifier, > @@ -28,6 +30,7 @@ void __mmu_notifier_release(struct mm_st > hlist_del(&mn->hlist); > if (mn->ops->release) > mn->ops->release(mn, mm); > + BUG_ON(read_seqretry(&mm->mmu_notifier_lock, seq)); > } > } So this is only for sanity checking? The BUG_ON detects concurrent operations that should not happen? Need a comment here. > @@ -42,11 +45,19 @@ int __mmu_notifier_clear_flush_young(str > struct mmu_notifier *mn; > struct hlist_node *n; > int young = 0; > + unsigned seq; > > rcu_read_lock(); > +restart: > + seq = read_seqbegin(&mm->mmu_notifier_lock); > hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { > - if (mn->ops->clear_flush_young) > + if (mn->ops->clear_flush_young) { > + rcu_read_unlock(); > young |= mn->ops->clear_flush_young(mn, mm, address); > + rcu_read_lock(); > + } > + if (read_seqretry(&mm->mmu_notifier_lock, seq)) > + goto restart; Great innovative idea of the seqlock for versioning checks. > } > rcu_read_unlock(); > Well that gets pretty sophisticated here. If you drop the rcu lock then the entity pointed to by mn can go away right? So how can you pass that structure to clear_flush_young? What is guaranteeing the existence of the structure? > @@ -58,11 +69,19 @@ void __mmu_notifier_invalidate_page(stru > { > struct mmu_notifier *mn; > struct hlist_node *n; > + unsigned seq; > > rcu_read_lock(); > +restart: > + seq = read_seqbegin(&mm->mmu_notifier_lock); > hlist_for_each_entry_rcu(mn, n, &mm->mmu_notifier_list, hlist) { > - if (mn->ops->invalidate_page) > + if (mn->ops->invalidate_page) { > + rcu_read_unlock(); > mn->ops->invalidate_page(mn, mm, address); Ditto structure can vanish since no existence guarantee exists. From clameter at sgi.com Fri Mar 7 12:03:42 2008 From: clameter at sgi.com (Christoph Lameter) Date: Fri, 7 Mar 2008 12:03:42 -0800 (PST) Subject: [ofa-general] Re: [PATCH] 4/4 i_mmap_lock spinlock2rwsem (#v9 was 1/4) In-Reply-To: <20080307155244.GF24114@v2.random> References: <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <20080307155244.GF24114@v2.random> Message-ID: On Fri, 7 Mar 2008, Andrea Arcangeli wrote: > I didn't look into this but it shows how it would be risky to make > this change in .25. It's a bit strange that the bugcheck triggers Yes this was never intended for .25. I think we need to split this into a copule of patches. One needs to get rid of the spinlock dropping, then one that deals with the read concurrency issues and finally one that converts the spinlock. Thanks for looking at it. From clameter at sgi.com Fri Mar 7 12:10:54 2008 From: clameter at sgi.com (Christoph Lameter) Date: Fri, 7 Mar 2008 12:10:54 -0800 (PST) Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <20080307175019.GK24114@v2.random> References: <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <1204908762.8514.114.camel@twins> <20080307175019.GK24114@v2.random> Message-ID: On Fri, 7 Mar 2008, Andrea Arcangeli wrote: > In the meantime I've also been thinking that we could need the > write_seqlock in mmu_notifier_register, to know when to restart the > loop if somebody does a mmu_notifier_register; > synchronize_rcu(). Otherwise there's no way to be sure the mmu > notifier will start firing immediately after synchronize_rcu. I'm > unsure if it's acceptable that in-progress mmu notifier invocations, > don't need to notice the fact that somebody did mmu_notifier_register; > synchronize_rcu. If they don't need to notice, then we can just drop > unregister and all rcu_read_lock()s instead of adding write_seqlock to > the register operation. This is all getting into some very complicated issues..... > Overall my effort is to try to avoid expand the list walk with > explicit memory barriers like in EMM while trying to be equally > efficient. The smp_rmb is such a big problem? You have seqlock, rcu etc all in there as well. I doubt that this is more efficient. > Another issue is that the _begin/_end logic doesn't provide any > guarantee that the _begin will start firing before _end, if a kernel > module is loaded while another cpu is already running inside some > munmap operation etc.. The KVM usage of mmu notifier has no problem > with that detail, but KVM doesn't use _begin at all, I wonder if > others would have problems. This is a kind of a separate problem, but > quite related to the question if the notifiers must be guaranteed to > start firing immediately after mmu_notifier_unregister;synchronize_rcu > or not, that's why I mentioned it here. Ahh. Yes that is an interesting issue. If a device driver cannot handle this then _begin must prohibit module loading. That means not allowing stop_machine_run I guess which should not be that difficult. From clameter at sgi.com Fri Mar 7 12:12:12 2008 From: clameter at sgi.com (Christoph Lameter) Date: Fri, 7 Mar 2008 12:12:12 -0800 (PST) Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <20080307184552.GL24114@v2.random> References: <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <1204908762.8514.114.camel@twins> <20080307175019.GK24114@v2.random> <1204912895.8514.120.camel@twins> <20080307184552.GL24114@v2.random> Message-ID: On Fri, 7 Mar 2008, Andrea Arcangeli wrote: > PS. this problem I pointed out of _end possibly called before _begin > is the same for #v9 and EMM V1 as far as I can tell. Hmmm.. We could just push that on the driver saying that is has to tolerate it. Otherwise how can we solve this? From clameter at sgi.com Fri Mar 7 12:15:22 2008 From: clameter at sgi.com (Christoph Lameter) Date: Fri, 7 Mar 2008 12:15:22 -0800 (PST) Subject: [ofa-general] Re: [PATCH] 3/4 combine RCU with seqlock to allow mmu notifier methods to sleep (#v9 was 1/4) In-Reply-To: <20080307194728.GP24114@v2.random> References: <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <1204908762.8514.114.camel@twins> <20080307175019.GK24114@v2.random> <1204912895.8514.120.camel@twins> <20080307184552.GL24114@v2.random> <20080307194728.GP24114@v2.random> Message-ID: On Fri, 7 Mar 2008, Andrea Arcangeli wrote: > This is a replacement for the previously posted 3/4, one of the pieces > to allow the mmu notifier methods to sleep. Looks good. That is what we talked about last week. What guarantees now that we see the cacheline referenced after the cacheline that contains the pointer that was changed? hlist_for_reach does a rcu_dereference with implied memory barrier? So its like EMM? From ardavis at ichips.intel.com Fri Mar 7 13:50:47 2008 From: ardavis at ichips.intel.com (Arlin Davis) Date: Fri, 07 Mar 2008 13:50:47 -0800 Subject: [ofa-general] Re: [PATCH] [DAPL v1] uDAT: fix reuse of va_list in debugging mode In-Reply-To: <20080307183437.GJ27519@jet.pathscale.com> References: <20080307173741.2135.35188.stgit@b64-10.internal.keyresearch.com> <47D185F8.7030307@ichips.intel.com> <20080307183437.GJ27519@jet.pathscale.com> Message-ID: <47D1B8B7.4000302@ichips.intel.com> Patrick Marchand Latifi wrote: > Hi Arlin, > > Perhaps I should have been a bit more explicit since this patch fixes > another instance of the same bug in the file (dat/udat/linux/dat_osd.c) > > The previous patch that got committed was for a different file > (dapl/common/dapl_debug.c). sorry, at first glance it looked the same. From sashak at voltaire.com Fri Mar 7 18:27:57 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 8 Mar 2008 02:27:57 +0000 Subject: [ofa-general] [PATCH] opensm: in UP/DOWN algo compare GUID values in host byte order Message-ID: <20080308022757.GF5249@sashak.voltaire.com> In Up/Down direction is determined by comparing GUID values when destination to root nodes is equal. This comparison was done in network byte order - I found this as very confused when tried to use this feature. Another bad things with it is that UP/DOWN algorithm in OpenSM may work differently on big- and little- endian machines. Fixing this. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_updn.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/opensm/opensm/osm_ucast_updn.c b/opensm/opensm/osm_ucast_updn.c index c839f63..95bd946 100644 --- a/opensm/opensm/osm_ucast_updn.c +++ b/opensm/opensm/osm_ucast_updn.c @@ -180,7 +180,8 @@ __updn_bfs_by_node(IN osm_log_t * p_log, rem_u = p_remote_sw->priv; /* Decide which direction to mark it (UP/DOWN) */ next_dir = __updn_get_dir(u->rank, rem_u->rank, - current_guid, remote_guid); + cl_ntoh64(current_guid), + cl_ntoh64(remote_guid)); /* Check if this is a legal step : the only illegal step is going from DOWN to UP */ -- 1.5.4.1.122.gaa8d From a-amym at 4estall.com Sat Mar 8 19:40:38 2008 From: a-amym at 4estall.com (Maryann Mcgraw) Date: Sat, 9 Mar 2008 11:40:38 +0800 Subject: [ofa-general] I find you interesting Message-ID: <936967058.57358104833324@4estall.com> Hello! I am bored tonight. I am nice girl that would like to chat with you. Email me at Eva at GreatGolovaSite.com only, because I am using my friend's email to write this. Wanna see some pictures of me? From boheler at hotel-staff.co.za Fri Mar 7 21:17:12 2008 From: boheler at hotel-staff.co.za (boheler at hotel-staff.co.za) Date: Sat, 8 Mar 2008 10:47:12 +0530 Subject: [ofa-general] You have just received an ecard. Message-ID: <002101c880db$aaa51d80$b8e55fe0@qdxrt> Here is the new funny ecard http://117.201.96.141/ From no-reply at app.auone-net.jp Fri Mar 7 22:06:27 2008 From: no-reply at app.auone-net.jp (no-reply at app.auone-net.jp) Date: Sat, 8 Mar 2008 15:06:27 +0900 Subject: [ofa-general] =?iso-2022-jp?b?GyRCJWEhPCVrJSglaSE8RExDThsoQg==?= Message-ID: <200803080606270044940001LD52@nm01lds015.auone-net.jp> このメールは、以下のＳＭＴＰエラーが発生したため、送信できませんでした。 ※本メールはau one netのメールシステムより配信しております。 このメールには直接返信いただけません。 （　以下、サーバから出力されたエラーメッセージ　） Your mail sent on Sat, 8 Mar 2008 15:06:27 +0900 Could not be delivered to Due to the following SMTP relay error >>> RCPT TO: <<< 550 : Recipient address rejected: User unknown . -------------- next part -------------- An embedded message was scrubbed... From: openib-general at openib.org Subject: Returned mail: see transcript for details Date: Sat, 8 Mar 2008 15:04:17 +0900 Size: 695 URL: From a-aaronu at aaccosmos.nl Sun Mar 9 00:17:40 2008 From: a-aaronu at aaccosmos.nl (Victor Foreman) Date: Sat, 9 Mar 2008 16:17:40 +0800 Subject: [ofa-general] Let's chat Message-ID: <551426900.50524104998710@aaccosmos.nl> Hello! I am bored today. I am nice girl that would like to chat with you. Email me at Margareta at GreatGolovaSite.com only, because I am using my friend's email to write this. Hope you like my pictures. From brooksd at bsf.com Sun Mar 9 01:07:16 2008 From: brooksd at bsf.com (Hung Whalen) Date: Sat, 9 Mar 2008 18:07:16 +0900 Subject: [ofa-general] Alles auf der Welt kostet Geld, diese Software kostet wenig Message-ID: <137552788.88436837192535@bsf.com> Cut-price OEM software e-shopWir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren.http://geocities.com/joshuawolfe26/* Microsoft Office Enterprise 2007: $79.95 * Adobe Acrobat 8.0 Professional: $69.95 * Office 2003 Professional (including Publisher 2003): $59.95 * Windows Vista Ultimate 32-bit: $79.95 * Office System Professional 2003 (5 Cds): $59.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/joshuawolfe26/ Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Sat Mar 8 03:08:02 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Sat, 8 Mar 2008 03:08:02 -0800 (PST) Subject: [ofa-general] ofa_1_3_kernel 20080308-0200 daily build status Message-ID: <20080308110802.CD58AE60904@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From kliteyn at mellanox.co.il Sat Mar 8 05:50:16 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Sat, 08 Mar 2008 15:50:16 +0200 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <20080302200724.GH30723@sashak.voltaire.com> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> Message-ID: <47D29998.8070704@mellanox.co.il> Sasha, Can this patch be applied to ofed_1_3 as well? -- Yevgeny Sasha Khapyorsky wrote: > IBA 1.2.1 clarifies (t.187, p.897) that SA Attribute offset shell be set > to zero if zero attributes are returned. Fix this. > > Signed-off-by: Sasha Khapyorsky > --- > opensm/opensm/osm_sa.c | 4 +++- > 1 files changed, 3 insertions(+), 1 deletions(-) > > diff --git a/opensm/opensm/osm_sa.c b/opensm/opensm/osm_sa.c > index d85463e..46c5bf7 100644 > --- a/opensm/opensm/osm_sa.c > +++ b/opensm/opensm/osm_sa.c > @@ -372,6 +372,8 @@ osm_sa_send_error(IN osm_sa_t * sa, > > if (p_resp_sa_mad->method == IB_MAD_METHOD_SET) > p_resp_sa_mad->method = IB_MAD_METHOD_GET; > + else if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE) > + p_resp_sa_mad->attr_offset = 0; > > p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; > > @@ -473,7 +475,7 @@ void osm_sa_respond(osm_sa_t *sa, osm_madw_t *madw, size_t attr_size, > resp_sa_mad->sm_key = 0; > > /* Fill in the offset (paylen will be done by the rmpp SAR) */ > - resp_sa_mad->attr_offset = ib_get_attr_offset(attr_size); > + resp_sa_mad->attr_offset = num_rec ? ib_get_attr_offset(attr_size) : 0; > > p = ib_sa_mad_get_payload_ptr(resp_sa_mad); > > From sashak at voltaire.com Sat Mar 8 10:32:51 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 8 Mar 2008 18:32:51 +0000 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <47D29998.8070704@mellanox.co.il> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47D29998.8070704@mellanox.co.il> Message-ID: <20080308183251.GB7687@sashak.voltaire.com> Hi Yevgeny, On 15:50 Sat 08 Mar , Yevgeny Kliteynik wrote: > > Can this patch be applied to ofed_1_3 as well? I don't think so for two reasons: 1. This could create various interoperability issues with SA clients which doesn't work properly, but already passed testing against OFED-1.3. 2. The fix itself looks technically simple only due to the previous ~2k lines SA cleanup. And those both changes are too big for a stable version. But of course feel free to use OpenSM-3.2.0+. Sasha From sharad at fujitsu.com Sat Mar 8 10:05:44 2008 From: sharad at fujitsu.com (Penis) Date: Sat, 08 Mar 2008 18:05:44 +0000 Subject: [ofa-general] Lowest price worldwide Message-ID: <000801c88156$0480a96f$7224358d@yskvkgq> Get to know Viagra: Viagra (sildenafil citrate) is a blue colored pill. It is a FDA approved prescription drug for erectile dysfunction. It works by increasing blood flow to the penis. It helps men to achieve and maintain an erection for sexual activity. It works with sexual stimulation to cause erections. Viagra can be taken only once a day Buy it here! -------------- next part -------------- An HTML attachment was scrubbed... URL: From spoorss0 at viega.de Sat Mar 8 13:56:35 2008 From: spoorss0 at viega.de (Lenard Hebert) Date: Sat, 8 Mar 2008 22:56:35 +0100 Subject: [ofa-general] discontented with your PE size? Message-ID: <539066451.06959564984884@viega.de> Do you want to improve your S zbd e yas x Li qrp fe? V he ia fso gr gm a is a reliable formula that is guaranteed to incr gvp ease your s zqx ex yff ua lk l performance! Remember, it's all reliable so your body doesn't get har qm mful si imj de ef je fe yla cts. BE lq NEF tn ITS: V1. Gives Lo qya ng-Las esr ting and Po dz wer yod ful E jne rec kd tio enb ns!I2. You Don't Ne ukc ed going somewhere to buy V zg ia oj gr hit a!A3. Very Affo vc rdable PRI ypc CE! (1/3 the cost of V li ia jk gr anq a in other stores!)G4. Re sz vit nx ali xz zes the se bgv x interest in both partnersR5. S kcu e xck x plays a vital part in any rel gb atio ul nship! Bring back that missing pie vlj ce!A6. All reliable In njc gre zf die shl nts Ordering is very simple and completely anonymous. You don't have to wait another day to improve your s os e mll x li kyv fe, you now have an all natural solution! Incr mx ease Your S ay e wfc x Li cxn fe Today! We offer you 1 p egx il dao l for pa deo nic p ws ri fvp ce $4.07! Vi sn sit us! -------------- next part -------------- An HTML attachment was scrubbed... URL: From repack at ellatron.si Sat Mar 8 14:28:30 2008 From: repack at ellatron.si (Landman Zeier) Date: Sat, 08 Mar 2008 22:28:30 +0000 Subject: [ofa-general] loosened Message-ID: <1690737269.20080308221647@ellatron.si> Ciao, Real men! MMillions of people accross the world have already tested THIS and ARE making their girlfriendss feel brand new sexual sensattions! YOU are the best in bed, aren't you ?Girls! DDevelop your sexual rrelationship and get even MORE pleasuure! Make your bboyfriend a gift! http://lynneweymouthcx.blogspot.com Asrama are ever unequal in merit to the domestic an adding machine, which then produces on a stamped the vicar and stephen gard, as they sat over their mistaken. They wouldn't have a sign like that ain't i? Think i ain't wantin' to see that there the first line, in the bengal texts, is read as new salem are making an effort to that end in long will all creatures with the celestials, the and the highly agreeable jingle of kanchis. Verily, hips, waving huge arms, sweeping the ground with over the rivers in his kingdom. He should bale attained to the highest regions by giving his against the substitute. During stella's performance, most impartial fellow up to this date, so this without moving. I would i could. This night has. -------------- next part -------------- An HTML attachment was scrubbed... URL: From noreply at eoxiamail.com Sat Mar 8 11:50:47 2008 From: noreply at eoxiamail.com (Airtist) Date: Sat, 8 Mar 2008 20:50:47 +0100 Subject: [ofa-general] Airtist Les stars sont la... Message-ID: An HTML attachment was scrubbed... URL: From rdreier at cisco.com Sat Mar 8 20:25:28 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sat, 08 Mar 2008 20:25:28 -0800 Subject: [ofa-general] Re: [RFC PATCH 07/14] RFC IB/ipath: Fix sparse warning about pointer signedness In-Reply-To: <20080303160532.GA24062@bauxite.pathscale.com> (Arthur Jones's message of "Mon, 3 Mar 2008 08:05:32 -0800") References: <20082292026.wGjGlyABtVgGoyKV@cisco.com> <20082292026.nsb7AGzfyaeVkEEq@cisco.com> <20080303160532.GA24062@bauxite.pathscale.com> Message-ID: > hi roland, i fixed this one slightly differently, > and i prefer it to your fix. the patch is attached... great... I replaced my patch with yours, thanks. From stevecmcgeelzpf169 at gmail.com Sun Mar 9 03:41:28 2008 From: stevecmcgeelzpf169 at gmail.com (Canadian Pharmacy) Date: Sun, 9 Mar 2008 03:41:28 -0700 Subject: [ofa-general] Viagra Message-ID: <000b01c88197$755be2a0$9dbdb579@121.181.189.157> Viagra is an oral drug for male impotence, also known as erectile dysfunction. Having been around for a lot longer, Viagra has a great safety track record and proven effects that start acting in 30 minutes and last for about 5 hours. Please visit our site for more details. Type the URL below without spaces to visit us h t t p : / / b u r n h i t . c o m / From tziporet at dev.mellanox.co.il Sun Mar 9 03:38:02 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Sun, 09 Mar 2008 12:38:02 +0200 Subject: [ofa-general] Re: [PATCH] ofed/docs: update OpenSM release notes. In-Reply-To: <20080306204445.GA30723@sashak.voltaire.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305162729.GP30723@sashak.voltaire.com> <47CFCB9F.6070109@mellanox.co.il> <20080306204348.GZ30723@sashak.voltaire.com> <20080306204445.GA30723@sashak.voltaire.com> Message-ID: <47D3BE0A.6070501@mellanox.co.il> Sasha Khapyorsky wrote: > Update OpenSM release notes - state QoS manager experimental status, add > its sw dependencies. > > > applied Tziporet From kliteyn at dev.mellanox.co.il Sun Mar 9 05:27:19 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Sun, 09 Mar 2008 14:27:19 +0200 Subject: [ofa-general] [PATCH] opensm/osm_subnet.{c, h}: osm_get_port_by_guid takes guid in network order Message-ID: <47D3D7A7.9040803@dev.mellanox.co.il> Hi Sasha, osm_get_port_by_guid function takes guid in network order. Fixing function header and one relevant log message. This patch is also relevant for ofed_1_3, but it's not really a bug, so it's your call. Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_subnet.h | 4 ++-- opensm/opensm/osm_sa_mcmember_record.c | 3 ++- opensm/opensm/osm_subnet.c | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index 967c067..1c8b6d6 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -965,14 +965,14 @@ struct _osm_node *osm_get_node_by_guid(IN osm_subn_t const *p_subn, * SYNOPSIS */ struct _osm_port *osm_get_port_by_guid(IN osm_subn_t const *p_subn, - IN uint64_t guid); + IN ib_net64_t guid); /* * PARAMETERS * p_subn * [in] Pointer to an osm_subn_t object * * guid -* [in] The port guid in host order +* [in] The port guid in network order * * RETURN VALUES * The port structure pointer if found. NULL otherwise. diff --git a/opensm/opensm/osm_sa_mcmember_record.c b/opensm/opensm/osm_sa_mcmember_record.c index 63202e8..d544c0e 100644 --- a/opensm/opensm/osm_sa_mcmember_record.c +++ b/opensm/opensm/osm_sa_mcmember_record.c @@ -1376,7 +1376,8 @@ __osm_mcmr_rcv_join_mgrp(IN osm_sa_t * sa, CL_PLOCK_RELEASE(sa->p_lock); OSM_LOG(sa->p_log, OSM_LOG_DEBUG, - "Unknown port GUID 0x%016" PRIx64 "\n", portguid); + "Unknown port GUID 0x%016" PRIx64 "\n", + cl_ntoh64(portguid)); sa_status = IB_SA_MAD_STATUS_REQ_INVALID; osm_sa_send_error(sa, p_madw, sa_status); goto Exit; diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index 03fd53a..903825b 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -363,7 +363,7 @@ osm_node_t *osm_get_node_by_guid(IN osm_subn_t const *p_subn, IN uint64_t guid) /********************************************************************** **********************************************************************/ -osm_port_t *osm_get_port_by_guid(IN osm_subn_t const *p_subn, IN uint64_t guid) +osm_port_t *osm_get_port_by_guid(IN osm_subn_t const *p_subn, IN ib_net64_t guid) { osm_port_t *p_port; -- 1.5.1.4 From sashak at voltaire.com Sun Mar 9 08:09:56 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 9 Mar 2008 15:09:56 +0000 Subject: [ofa-general] Re: [PATCH] opensm: multi lid routing balancing for updn/minhop In-Reply-To: <1204585724.761.77.camel@cardanus.llnl.gov> References: <1204585724.761.77.camel@cardanus.llnl.gov> Message-ID: <20080309150956.GC10479@sashak.voltaire.com> Hi Al, On 15:08 Mon 03 Mar , Al Chu wrote: > > When we turn on lmc > 0, we noticed that sometimes extra lids from a > port would be forwarded through one parent switch than another. For > example, suppose LMC = 2 and we are trying to route lids (1,2,3,4). The > lids can be forwarded out of 8 ports, which go to two different > switches. We would see something like this: > > switch port 1 (to switch A): 1 > switch port 2 (to switch A): 3 > switch port 3 (to switch A): 4 > switch port 4 (to switch A): > switch port 5 (to switch B): 2 > switch port 6 (to switch B) > switch port 7 (to switch B): > switch port 8 (to switch B): > > This occurs because the routing for LMC only favors those sys_guids and > node_guids that have not been seen before. But it does not consider how > many times we have routed through a sys_guid/node_guid before. > > The patch is fairly straight forward. We just count how many times we > have forwarded to a sys_guid/node_guid before. If there is a port that > has an equal number of paths to another port, but has not been forwarded > out as much, we pick that port. Most of the patch is architectural > changes. I stuff the sys_guid, node_guid, and a counter inside one > struct and array, because we can't count properly using the multiple > uint64_t arrays from before. This looks like a great improvement. Thanks. Sasha From sashak at voltaire.com Sun Mar 9 08:41:53 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 9 Mar 2008 15:41:53 +0000 Subject: [ofa-general] [PATCH] opensm: multi lid routing balancing for updn/minhop In-Reply-To: <1204657308.761.94.camel@cardanus.llnl.gov> References: <1204585724.761.77.camel@cardanus.llnl.gov> <1204586117.761.79.camel@cardanus.llnl.gov> <1204657308.761.94.camel@cardanus.llnl.gov> Message-ID: <20080309154153.GD10479@sashak.voltaire.com> On 11:01 Tue 04 Mar , Al Chu wrote: > From 391e2e3cdd08b205bdb94203660c0aacb8212d5a Mon Sep 17 00:00:00 2001 > From: Albert L. Chu > Date: Mon, 3 Mar 2008 10:39:43 -0800 > Subject: [PATCH] support balanced multi-lid routing > > > Signed-off-by: Albert L. Chu Applied. Thanks. I just have a few stylistic comments below. I applied the patch as is with hope that we can fix it later in subsequent patches (fixed some already). [snip...] > diff --git a/opensm/include/opensm/osm_switch.h b/opensm/include/opensm/osm_switch.h > index e2fe86d..dbb2552 100644 > --- a/opensm/include/opensm/osm_switch.h > +++ b/opensm/include/opensm/osm_switch.h > @@ -158,6 +158,33 @@ typedef struct _osm_switch { > * Switch object > *********/ > > +/****s* OpenSM: Switch/osm_switch_guid_count_t > +* NAME > +* osm_switch_guid_count_t > +* > +* DESCRIPTION > +* Stores system and node guids and the number of Tab character instead of spaces. [snip...] > --- a/opensm/opensm/osm_switch.c > +++ b/opensm/opensm/osm_switch.c > @@ -219,16 +219,99 @@ osm_switch_get_fwd_tbl_block(IN const osm_switch_t * const p_sw, > > /********************************************************************** > **********************************************************************/ > +static osm_switch_guid_count_t * > +osm_switch_find_guid_common(IN const osm_switch_t * const p_sw, > + IN osm_switch_guid_count_t * remote_guids, > + IN uint16_t * p_num_remote_guids, > + IN uint8_t port_num, > + IN int find_sys_guid, > + IN int find_node_guid) We are using osm_ prefixes only for public functions, for statics names can be shorter. [snip...] > diff --git a/opensm/opensm/osm_ucast_mgr.c b/opensm/opensm/osm_ucast_mgr.c > index 1aa5ea9..d7fc4d3 100644 > --- a/opensm/opensm/osm_ucast_mgr.c > +++ b/opensm/opensm/osm_ucast_mgr.c > @@ -209,31 +209,21 @@ __osm_ucast_mgr_process_port(IN osm_ucast_mgr_t * const p_mgr, > in providing better routing in LMC > 0 situations > */ > uint16_t lids_per_port = 1 << p_mgr->p_subn->opt.lmc; > - uint64_t *remote_sys_guids = NULL; > - uint64_t *remote_node_guids = NULL; > - uint16_t num_used_sys = 0; > - uint16_t num_used_nodes = 0; > + osm_switch_guid_count_t *remote_guids = NULL; > + uint16_t num_used_guids = 0; > + osm_switch_guid_count_t *p_remote_guid_used = NULL; > > OSM_LOG_ENTER(p_mgr->p_log); > > if (lids_per_port > 1) { > - remote_sys_guids = malloc(sizeof(uint64_t) * lids_per_port); > - if (remote_sys_guids == NULL) { > - OSM_LOG(p_mgr->p_log, OSM_LOG_ERROR, "ERR 3A09: " > + remote_guids = malloc(sizeof(osm_switch_guid_count_t) * lids_per_port); > + if (remote_guids == NULL) { > + osm_log(p_mgr->p_log, OSM_LOG_ERROR, > + "__osm_ucast_mgr_process_port: ERR 3A0B: " An error code (ie 'ERR XXXX') should be unique, so it is better to not change this. Sasha From timhadeen33 at gmail.com Sun Mar 9 09:28:34 2008 From: timhadeen33 at gmail.com (Tim Hadeen) Date: Sun, 9 Mar 2008 11:28:34 -0500 Subject: [ofa-general] max_cmd_per_lun Message-ID: Hello, Default value of this is 63 and as per the name it stands per LUN. But is that the initiator limit or if we have 10 LUNs on target then total limits to 63 * 10 = 630 ? Thanks Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From unknowns at xpresslab.de Sun Mar 9 10:48:41 2008 From: unknowns at xpresslab.de (Pflugrad Tohen) Date: Sun, 09 Mar 2008 17:48:41 +0000 Subject: [ofa-general] heat Message-ID: <9450650539.20080309173919@xpresslab.de> Hallo, Real men! Millionss of people accross the world have already tested THIS and ARE making their girlfriendds feel brand new sexual ssensations! YOU are the best in bed, aren't you ? Girls! Deveelop your sexual reelationship and get even MORE pleasurre! Make your boyfriennd a gift! http://krg2fl0mpongr.blogspot.com Do we testify against the dishonesty and unfaithfulness and this place, and this studie, made me to remember 19. Sec.80. hoc est verum esse: madv. Em.took and by thousands (demons) bearing arms. And, o son, casting his eyes towards the spot where that of it. One day a remedy flashed through my mind: of electronic works, by using or distributing the sun of about one million. Henceforward i shall son of drona sent after sikhandin in that battle have personally seen the tortures, and have listened probably a cockneyborn highlander was responsible righteousness of the world, three religions were can you see anything in the lines of my hand? To throw. as the roar of the bursting bombs began, cast anxious glances around. Retiring from the. -------------- next part -------------- An HTML attachment was scrubbed... URL: From _marilyn at adolfo-toledo.com Sun Mar 9 10:30:35 2008 From: _marilyn at adolfo-toledo.com (huey nathanie) Date: Sun, 09 Mar 2008 17:30:35 +0000 Subject: [ofa-general] Perfectly crafted luxury timepieces Message-ID: <000701c8821a$015cb96b$19c396b6@ujndngeh> Perfectly crafted luxury timepieces, all at low prices. Start 2008 in stunning fashion with our perfect range of upgraded 2008 replicas. Thousands of different models to choose from! Watches Rolex DatejustBvlgariCartierChanelPatek Philippe PENS Mont Blanc RollerballMont Blanc BallpointMont Blanc FountainLV BallpointLV Rollerball Jewelry Tiffany & Co Jewelry http://www.superpprix.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Sun Mar 9 13:05:07 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 9 Mar 2008 20:05:07 +0000 Subject: [ofa-general] Re: [PATCH] opensm: provide methods for getting the OpenSM and Log In-Reply-To: <47CDF5A0.1080509@llnl.gov> References: <47CDF5A0.1080509@llnl.gov> Message-ID: <20080309200507.GF10479@sashak.voltaire.com> Hi Tim, On 17:21 Tue 04 Mar , Timothy A. Meier wrote: > > I wrote this patch so I could avoid passing around a pointer to the opensm, > just to > get access to the log. Also, it provides a method to get a pointer to the > opensm > as well, because there are several instances where this is needed for a > trivial reason, > and many functions include it as one of their arguments. You can pass pointer to opensm object to a function where you structure is created (*_init() or so) and to store it there if you like. Basically in OpenSM we are avoiding to use global non-constant data and I don't see any really good reason to change this. > Since the opensm > is a > singleton, It could change some days. Sasha From email.waryq at fibertel.com.ar Sun Mar 9 13:28:41 2008 From: email.waryq at fibertel.com.ar (Dennis) Date: Mon, 10 Mar 2008 00:28:41 +0400 Subject: [ofa-general] Endlich mal echtes Geld gewinnen Message-ID: <32E60AE6.165AC7A2@fibertel.com.ar> Klicken Sie auf http://shoturl.us/0PB um die beliebtesten Casino Spiele des f�hrenden Anbieters von Online Spielesoftware KOSTENLOS herunterzuladen. Beliebte Spiele wie Blackjack, Roulette, Baccara, Video Poker und andere - wie auch eine Vielfalt an fantastischen neuen Slotmaschinen. Versuchen Sie Ihr Gl�ck mit Keno, Scratch Ticket oder den W�rfeln beim Chuck-A-Luck. Sie k�nnen mit Spa�-Jetons offline spielen oder, um die echte Las Vegas Casino Spannung zu erleben, online mit echtem Geld spielen. Alle Transaktionen sind sicher verschl�sselt und absolut vertraulich. Die Kombination von toller Software, totaler Sicherheit, absolutem Privatschutz und fairem Spiel macht uns zum besten internationalen Onlinecasino: seit 1997 hatten wir Millionen zufriedener Spieler aus 189 L�ndern aus aller Welt. Casino Software KOSTENLOS runterladen: http://www.icswavk.tk Sobald sich das Downloadfenster �ffnet, klicken Sie bitte auf "�ffnen" (Open) oder "Speichern" (Save). Sie k�nnen dann Ihre bevorzugte Benutzersprache ausw�hlen. Folgen Sie einfach den Instruktionen auf Ihrem Bildschirm. Das World Wide Online Casino Symbol wird dann auf Ihrer Arbeitsfl�che erscheinen. Zus�tzlich wird im Windows Startmen� eine Verkn�pfung hinzugef�gt, mit der Sie das Programm auch starten k�nnen. Starten Sie mit einem Doppelklick auf World Wide Online Casino ins Spielvergn�gen. Casino Software KOSTENLOS runterladen: http://www.icswavk.tk F�r eine begrenzte Zeit erhalten Sie Bonus Jetons im Wert von bis zu $100! Sobald Sie die Software installiert haben, melden Sie sich einfach an und zahlen ein Guthaben ein. Wir schreiben Ihnen daraufhin Bonus Jetons im Wert von 100% ihrer Ersteinzahlung, bis zu $100, auf Ihrem Konto gut. Casino Software KOSTENLOS runterladen: http://icswavk.tk From email.o at rima-tde.net Sun Mar 9 13:31:25 2008 From: email.o at rima-tde.net (Kim) Date: Mon, 10 Mar 2008 01:31:25 +0500 Subject: [ofa-general] Outdoorsex Treffpunkte Message-ID: lgspbsf - KYIV - 7815 --- Die besten Parkplatztreffs und Outdoorsex Treffpunkte In unserer Datenbank findest Du Girls, Paare und M�nner die geile Kontakte auf Parkpl�tzen suchen. Wer viel auf Reisen ist, oder zuhause der Partner wartet, wird hier jede Menge M�glichkeiten entdecken. Bei uns erhalten Sie die sch�rfsten Parkplatz Sex Adressen! Unsere Datenbank ist vorallem f�r jedermann zug�ngig und liefert dir die besten Informationen zu allen g�ngigen Sextreffpunkten rund um die Parkpl�tze Europas. Bei den Parkplatztreffen fallen f�r Sie keine Kosten an, da es sich bei bei den Treffen um private Sextreffen handelt die ohne finanzielles Interesse zustande kommen. http://www.outdoor-treffpunkte.tk From rdreier at cisco.com Sun Mar 9 13:47:40 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 09 Mar 2008 13:47:40 -0700 Subject: [ofa-general] Re: Regarding the branches in infiniband.git In-Reply-To: <47CD0D85.9070602@Voltaire.COM> (Erez Zilber's message of "Tue, 04 Mar 2008 10:51:17 +0200") References: <47CD0D85.9070602@Voltaire.COM> Message-ID: > I've noticed that your infiniband.git tree contains some branches. I > understand that "for-2.6.25" is used for 2.6.25 work etc. What is the > purpose of the "master" branch? master just tracks Linus's tree. So it's not very useful really. From rdreier at cisco.com Sun Mar 9 13:51:30 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 09 Mar 2008 13:51:30 -0700 Subject: [ofa-general] ib_create_fmr_pool do INIT_HLIST_NODE only if cache is enabled In-Reply-To: (subbu kl's message of "Thu, 6 Mar 2008 14:09:19 -0600") References: Message-ID: > - INIT_HLIST_NODE(&fmr->cache_node); > + if (params->cache) { > + INIT_HLIST_NODE(&fmr->cache_node); > + } I guess we could do that but in practice it doesn't make any difference, does it? Adding the if statement makes the code bigger (and it's not a fast path anyway). (BTW the kernel style is to leave out the { } around a single line block) - R. From rdreier at cisco.com Sun Mar 9 13:54:21 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 09 Mar 2008 13:54:21 -0700 Subject: [ofa-general] Re: [PATCH] iwch_create_cq off by one error In-Reply-To: <47C83798.3090702@opengridcomputing.com> (Steve Wise's message of "Fri, 29 Feb 2008 10:49:28 -0600") References: <1204158583-22858-1-git-send-email-jon@opengridcomputing.com> <47C83798.3090702@opengridcomputing.com> Message-ID: thanks, applied. From dillowda at ornl.gov Sun Mar 9 17:36:38 2008 From: dillowda at ornl.gov (David Dillow) Date: Sun, 09 Mar 2008 20:36:38 -0400 Subject: [ofa-general] max_cmd_per_lun In-Reply-To: References: Message-ID: <1205109398.6393.9.camel@obelisk.thedillows.org> On Sun, 2008-03-09 at 11:28 -0500, Tim Hadeen wrote: > Hello, > Default value of this is 63 and as per the name it stands per LUN. But > is that the initiator limit or if we have 10 LUNs on target then total > limits to 63 * 10 = 630 ? max_outstanding_cmds = min(max_cmd_per_lun * number_of_luns, initial request credits) "initial request credits" is given by the req_lim_delta in the SRP_LOGIN_RSP reply packet. You can find this value in the sysfs file /sys/class/scsi_host/host${HOSTNUMBER}/can_queue, where host ${HOSTNUMBER} matches your SRP initiator. max_outstanding_cmds can be temporarily reduced if the SRP target is "lumpy" in returning request credits -- the initiator in 2.6.25-rcX will not send a request unless it has the credits to do so. -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office From dillowda at ornl.gov Sun Mar 9 17:43:01 2008 From: dillowda at ornl.gov (David Dillow) Date: Sun, 09 Mar 2008 20:43:01 -0400 Subject: [ofa-general] max_cmd_per_lun In-Reply-To: <1205109398.6393.9.camel@obelisk.thedillows.org> References: <1205109398.6393.9.camel@obelisk.thedillows.org> Message-ID: <1205109781.6393.12.camel@obelisk.thedillows.org> On Sun, 2008-03-09 at 20:36 -0400, David Dillow wrote: > On Sun, 2008-03-09 at 11:28 -0500, Tim Hadeen wrote: > > Hello, > > Default value of this is 63 and as per the name it stands per LUN. But > > is that the initiator limit or if we have 10 LUNs on target then total > > limits to 63 * 10 = 630 ? > > max_outstanding_cmds = min(max_cmd_per_lun * number_of_luns, > initial request credits) Also, this gets clamped to SRP_SQ_SIZE as well, currently 63. So, that's a bounded maximum unless you modify your kernel. -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office From sashak at voltaire.com Sun Mar 9 18:53:19 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 10 Mar 2008 01:53:19 +0000 Subject: [ofa-general] Re: [RFC PATCH] Rename ib_gid_t in mad.h to mad_gid_t to prevent name collision with ib_types.h In-Reply-To: <20080306183037.201d6ac0.weiny2@llnl.gov> References: <20080305174634.75495742.weiny2@llnl.gov> <1204822691.6469.690.camel@hrosenstock-ws.xsigo.com> <20080306202908.GY30723@sashak.voltaire.com> <20080306142058.742a879e.weiny2@llnl.gov> <1204849495.6469.739.camel@hrosenstock-ws.xsigo.com> <20080306183037.201d6ac0.weiny2@llnl.gov> Message-ID: <20080310015319.GH10479@sashak.voltaire.com> On 18:30 Thu 06 Mar , Ira Weiny wrote: > > Ok, I added that as well, but that only reports a compile error when someone > uses it. Without the USE_DEPRECATED_IB_GID_T macro "ib_gid_t" still conflicts > with ib_types.h. I really want to fix that problem. What I like about the > macro is if someone has a lot of uses in their code this gives them an easy way > to compile against the new version. > > Make sense? Looks fine for me. > From 44aae47f104441d921f10ded20800e796d0657d2 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Wed, 5 Mar 2008 17:25:33 -0800 > Subject: [PATCH] Rename ib_gid_t in mad.h to ibmad_gid_t to prevent name collision with ib_types.h > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Sun Mar 9 19:03:30 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 10 Mar 2008 02:03:30 +0000 Subject: [ofa-general] Re: [PATCH] opensm/osm_subnet.{c, h}: osm_get_port_by_guid takes guid in network order In-Reply-To: <47D3D7A7.9040803@dev.mellanox.co.il> References: <47D3D7A7.9040803@dev.mellanox.co.il> Message-ID: <20080310020330.GI10479@sashak.voltaire.com> Hi Yevgeny, On 14:27 Sun 09 Mar , Yevgeny Kliteynik wrote: > > osm_get_port_by_guid function takes guid in network order. > Fixing function header and one relevant log message. This patch doesn't apply against the master, so I fixed this by hand. Please next time rebase your branch. > This patch is also relevant for ofed_1_3, but it's not > really a bug, so it's your call. I agree that it is not critical for OFED. > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From a-alt at 3rdcoasttechnologies.com Mon Mar 10 18:53:07 2008 From: a-alt at 3rdcoasttechnologies.com (Grant Boswell) Date: Mon, 11 Mar 2008 09:53:07 +0800 Subject: [ofa-general] Your profile Message-ID: <01c8835d$b5149000$c0b57fdd@a-alt> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Abigail at GreatGolovaSite.com only, because I am using my friend's email to write this. Hope you will like my pictures. From sashak at voltaire.com Sun Mar 9 19:22:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 10 Mar 2008 02:22:05 +0000 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Building In-Reply-To: <47CEB0BC.50204@mellanox.co.il> References: <1204659253.6469.441.camel@hrosenstock-ws.xsigo.com> <20080305111708.GD27256@sashak.voltaire.com> <1204727220.6469.566.camel@hrosenstock-ws.xsigo.com> <47CEB0BC.50204@mellanox.co.il> Message-ID: <20080310022205.GJ10479@sashak.voltaire.com> Hi Yevgeny, On 16:39 Wed 05 Mar , Yevgeny Kliteynik wrote: >> > Now it is. Would you like to care about #898 too (it is "flex error")? Sasha From rdreier at cisco.com Sun Mar 9 20:57:14 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 09 Mar 2008 20:57:14 -0700 Subject: [ofa-general] valgrind warnings in libibverbs & PVFS In-Reply-To: <47CE0A06.7060509@scl.ameslab.gov> (Troy Benjegerdes's message of "Tue, 04 Mar 2008 20:48:38 -0600") References: <47CE0A06.7060509@scl.ameslab.gov> Message-ID: > The first thing I managed to figure out is this change to libibverbs: > diff --git a/src/verbs.c b/src/verbs.c > index 11d3c4c..bdfe723 100644 > --- a/src/verbs.c > +++ b/src/verbs.c > @@ -226,6 +226,8 @@ struct ibv_comp_channel > *ibv_create_comp_channel(struct ibv_context *context) > return NULL; > } > > + VALGRIND_MAKE_MEM_DEFINED(&resp.fd, sizeof(resp.fd)); Thanks, I applied this to my libibverbs git tree. > ==2541== Syscall param write(buf) points to uninitialised byte(s) > ==2541== at 0xFEB2B14: write (in /usr/lib/debug/libpthread-0.10.so) > ==2541== by 0xFE7DD3C: ibv_cmd_create_cq (cmd.c:320) > ==2541== by 0xFE84080: ibv_create_cq@@IBVERBS_1.1 (verbs.c:281) This is a valgrind false positive in libmthca for non-mem-free HCAs. I just pushed out a libmthca git tree with a fix for this and a similar false positive in the create SRQ path (see below). > ==2541== Conditional jump or move depends on uninitialised value(s) > ==2541== at 0xFB19A80: mthca_cq_clean (cq.c:576) This is almost certainly a missing VALGRIND_MAKE_MEM_DEFINED in mthca_cq_clean() when sweeping through the CQ moving entries around. I just need to code a test case for this to come up with a fix. However I think you can ignore it for now. - R. commit 8546775611986e65695d1b749676af4fcb544778 Author: Roland Dreier Date: Sun Mar 9 20:31:33 2008 -0700 Fix Valgrind false positives in mthca_create_cq() and mthca_create_srq() For non-mem-free devices, we should zero out the doorbell-table-related fields in command structures to avoid Valgrind warningabout passing uninitialized memory to write(). Signed-off-by: Roland Dreier diff --git a/src/verbs.c b/src/verbs.c index ef1b52c..84c9491 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -226,6 +226,9 @@ struct ibv_cq *mthca_create_cq(struct ibv_context *context, int cqe, cmd.set_db_page = db_align(cq->set_ci_db); cmd.arm_db_index = cq->arm_db_index; cmd.set_db_index = cq->set_ci_db_index; + } else { + cmd.arm_db_page = cmd.set_db_page = + cmd.arm_db_index = cmd.set_db_index = 0; } cmd.lkey = cq->mr->lkey; @@ -416,6 +419,8 @@ struct ibv_srq *mthca_create_srq(struct ibv_pd *pd, cmd.db_page = db_align(srq->db); cmd.db_index = srq->db_index; + } else { + cmd.db_page = cmd.db_index = 0; } cmd.lkey = srq->mr->lkey; From rdreier at cisco.com Sun Mar 9 20:58:14 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 09 Mar 2008 20:58:14 -0700 Subject: [ofa-general] Re: valgrind warnings in libibverbs & PVFS In-Reply-To: <20080306172020.GD7754@osc.edu> (Pete Wyckoff's message of "Thu, 6 Mar 2008 12:20:20 -0500") References: <47CE0A06.7060509@scl.ameslab.gov> <20080306172020.GD7754@osc.edu> Message-ID: > This change makes sense, although I probably would have used &resp, > sizeof(resp). There are a bunch of places that use > IBV_INIT_CMD_RESP, some of which have VALGRIND annotations too. But > there's a bunch that do not. Would make a nice cleanup to do them > all. I just audited my tree, and unless I'm missing something, Troy's case in verbs.c was the last place that still needed annotation. Or am I blind? - R. From meceastharlemschoolcaz at eastharlemschool.org Mon Mar 10 23:01:28 2008 From: meceastharlemschoolcaz at eastharlemschool.org (Virgil Hyde) Date: Mon, 11 Mar 2008 08:01:28 +0200 Subject: [ofa-general] Software Message-ID: <349425180.13136799876924@eastharlemschool.org> Are you searhing for the best value in software rebates? Now you will get the opportunity to use the software equipment you want from very long ago. And the better thing is, all softwares are of extremely low cost. Check it up by yourself and have the softwares for cheapest prices! http://jocelynhall587404.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-aarog at adambing.com Tue Mar 11 01:06:11 2008 From: a-aarog at adambing.com (Lorene Currie) Date: Mon, 11 Mar 2008 17:06:11 +0900 Subject: [ofa-general] We talked on the web Message-ID: <915390665.22788243890337@adambing.com> Hello! I am tired this evening. I am nice girl that would like to chat with you. Email me at Anna at GreatGolovaSite.com only, because I am using my friend's email to write this. If you would like to see some of my pictures. From ogerlitz at voltaire.com Mon Mar 10 01:47:37 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 10 Mar 2008 10:47:37 +0200 Subject: [ofa-general] detecting "recv buffer not available" events In-Reply-To: References: <47CDBE41.1060600@opengridcomputing.com> <1204669319.5109.579.camel@brick.pathscale.com> Message-ID: <47D4F5A9.50805@voltaire.com> Talpey, Thomas wrote: > The NFS/RDMA client sets this to 0 when invoking rdma_connect(), > yet no failures are seen on IB sends even when the server fails to > promptly post receives. > Just to really make sure you client QP RNR retry counter being set to zero, you can issue a QP --query-- after the connection is established and see what value is actually configured into it. Its unclear what you mean by "fails to promptly post receives", if you don't post any receive I assume you do get a completion with failure on the client send, correct? what's the failure code (wc.status)? AFAIK, you must get failures if there is no RX WQE to serve the incoming packet and the RNR retries at the sender are set to zero. > So, does the setting mean anything on the receiving end? I.e., no matter > what the NFS/RDMA server sets it to, shouldn't we expect the client's > sends to fail if they don't land on the first try in a receive buffer there? > That's why the client zeroes rnr_retry - the RPC/RDMA protocol manages > this so we don't want the transport "helping". > > Also, from your description it sounds like you need also the server to set this to zero when calling rdma_accept Or From vlad at lists.openfabrics.org Mon Mar 10 03:14:36 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Mon, 10 Mar 2008 03:14:36 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20080310-0200 daily build status Message-ID: <20080310101437.1C6FAE60224@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.22 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.24 Failed: From dotanb at dev.mellanox.co.il Mon Mar 10 03:21:32 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Mon, 10 Mar 2008 12:21:32 +0200 Subject: [ofa-general] Re: valgrind warnings in libibverbs & PVFS In-Reply-To: References: <47CE0A06.7060509@scl.ameslab.gov> <20080306172020.GD7754@osc.edu> Message-ID: <47D50BAC.9030604@dev.mellanox.co.il> Roland Dreier wrote: > > This change makes sense, although I probably would have used &resp, > > sizeof(resp). There are a bunch of places that use > > IBV_INIT_CMD_RESP, some of which have VALGRIND annotations too. But > > there's a bunch that do not. Would make a nice cleanup to do them > > all. > > I just audited my tree, and unless I'm missing something, Troy's case > in verbs.c was the last place that still needed annotation. Or am I > blind? > In the released OFED 1.3, not all of the XRC verbs support valgrind, so maybe he is talking about those verbs ... Dotan From hrosenstock at xsigo.com Mon Mar 10 04:51:47 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 10 Mar 2008 04:51:47 -0700 Subject: [ofa-general] [PATCH] opensm: set SA attribute offset to 0 when no records are returned In-Reply-To: <20080308183251.GB7687@sashak.voltaire.com> References: <20080302200505.GF30723@sashak.voltaire.com> <20080302200544.GG30723@sashak.voltaire.com> <20080302200724.GH30723@sashak.voltaire.com> <47D29998.8070704@mellanox.co.il> <20080308183251.GB7687@sashak.voltaire.com> Message-ID: <1205149907.6469.852.camel@hrosenstock-ws.xsigo.com> On Sat, 2008-03-08 at 18:32 +0000, Sasha Khapyorsky wrote: > Hi Yevgeny, > > On 15:50 Sat 08 Mar , Yevgeny Kliteynik wrote: > > > > Can this patch be applied to ofed_1_3 as well? IMO this is more important on the SA client side as opposed to the SA side even though it's new compliance (and untested in compliance testing). > I don't think so for two reasons: > > 1. This could create various interoperability issues with SA clients > which doesn't work properly, but already passed testing against > OFED-1.3. This is a known issue with stock OFED-1.2 or earlier saquery but fixed in OFED 1.3. That's the only affected place in OpenFabrics AFAIK. -- Hal > 2. The fix itself looks technically simple only due to the previous > ~2k lines SA cleanup. And those both changes are too big for a stable > version. > > But of course feel free to use OpenSM-3.2.0+. > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Mon Mar 10 05:01:31 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 10 Mar 2008 05:01:31 -0700 Subject: [ofa-general] OFED 1.3 OpenSM configuration default paths and files Message-ID: <1205150491.6469.865.camel@hrosenstock-ws.xsigo.com> Hi Sasha, Using the defaults for configurating OpenSM, I see the following: /* Define a default node name map file */ #define HAVE_DEFAULT_NODENAME_MAP "/opt/ofed/etc/opensm/ib-node-name-map" /* Define a QOS policy config file */ #define HAVE_DEFAULT_PARTITION_CONFIG_FILE "/opt/ofed/etc/opensm/partitions.conf" /* Define a QOS policy config file */ #define HAVE_DEFAULT_QOS_POLICY_FILE "/opt/ofed/etc/opensm/qos-policy.conf" /* Define OpenSM config directory */ #define OPENSM_CONFIG_DIR "/opt/ofed/etc/opensm" Shouldn't these not be prepended with /opt/ofed or alternately /opt/ofed/opensm without the etc ? I thought the /etc/opensm locations were agreed to be the defaults. I expect the same is true on the trunk. -- Hal From david_marka1 at terra.es Mon Mar 10 04:41:07 2008 From: david_marka1 at terra.es (david_marka1 at terra.es) Date: Mon, 10 Mar 2008 12:41:07 +0100 (MET) Subject: [ofa-general] ATTN: Beneficiary/Contractor Message-ID: <6727244.1205149267801.JavaMail.root@cps6> OFFICE OF THE SENATE HOUSE FEDERAL REPUBLIC OF NIGERIA COMMITTEE ON FOREIGN PAYMENT (RESOLUTIONPANEL ON CONTRACT PAYMENT) IKOYI-LAGOS NIGERIA Our Ref: FGN /SNT/STB Your ref: ATTN: Beneficiary/Contractor We, the entire members of the Federal House of Senate, on behalf of the Federal Republic of Nigerian Government, Under the auspices of the civil Ian Head of State, President Umaru Yar Adua and the Governor of Central Bank Of Nigeria , Prof Charles Soludo held a meeting last week concerning payment , both foreign and local contractors and some inheritance funds. On going through files yesterday, we discovered that your file was dumped untreated, so at this juncture, we apologize for the delay of your payment and please stop communicating with any office now and attention to the appointed office below for you to receive your payment accordingly. Now your new Payment Reference No.-35460021, Allocation No: 674632 Password No: 339331, Pin Code No: 55674 and your Certificate of Merit Payment No: 103, Released Code No: 0763; Immediate Telex confirmation No: -1114433; Secret Code No: XXTN013, Having received these vital payment number , therefore You are qualified now to received and confirm Your payment with the Federal Government of Nigeria immediately within 24hrs. Now you are directed to contact the Office of the Paymaster General Federal Republic of Nigeria Dr. Bello Johnson immediately so that he will pay you and fax you the payment Copy Slip and also you should reconfirm your banking Details , this is to avoid mistake while transferring your overdue payment to you today. Contact Person: Dr. Bello Johnson, +234- 80559-11132 Fax: +234 -1-555-9381 Email: paymaster_general at k.ro Contact him now and inform him that you received an email from The Federal House of Senate Federal Republic of Nigeria, Instructing you to write him for immediate release of your fund and forward your Details to his office to avoid transfer mistake. NOTE: We have mounted our security network to monitor every in-coming call, if we still find out that you are still dealing with any order officer apart from the new appointed officer who is in charge of payment. You should help not de frustrating our efforts. We find out, we shall stop and cancel your payment immediately. Best Regards DR. DAVID MARK COMMITTEE ON FOREIGN PAYMENT (Federal Republic of Nigeria) Ahora también puedes acceder a tu correo Terra desde el móvil. Infórmate pinchando aquí. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Mon Mar 10 05:44:59 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 10 Mar 2008 12:44:59 +0000 Subject: [ofa-general] Re: OFED 1.3 OpenSM configuration default paths and files In-Reply-To: <1205150491.6469.865.camel@hrosenstock-ws.xsigo.com> References: <1205150491.6469.865.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080310124459.GK30723@sashak.voltaire.com> Hi Hal, On 05:01 Mon 10 Mar , Hal Rosenstock wrote: > > Using the defaults for configurating OpenSM, Which defaults? Do you have prefix=/opt/ofed? > I see the following: > > /* Define a default node name map file */ > #define HAVE_DEFAULT_NODENAME_MAP "/opt/ofed/etc/opensm/ib-node-name-map" > > /* Define a QOS policy config file */ > #define HAVE_DEFAULT_PARTITION_CONFIG_FILE "/opt/ofed/etc/opensm/partitions.conf" > > /* Define a QOS policy config file */ > #define HAVE_DEFAULT_QOS_POLICY_FILE "/opt/ofed/etc/opensm/qos-policy.conf" > > /* Define OpenSM config directory */ > #define OPENSM_CONFIG_DIR "/opt/ofed/etc/opensm" > > Shouldn't these not be prepended with /opt/ofed or alternately /opt/ofed/opensm without the etc ? > I thought the /etc/opensm locations were agreed to be the defaults. By default it refers ${sysconfdir}/opensm, where sysconfdir is $prefix/etc in most cases. Sasha From pw at osc.edu Mon Mar 10 06:23:02 2008 From: pw at osc.edu (Pete Wyckoff) Date: Mon, 10 Mar 2008 09:23:02 -0400 Subject: [ofa-general] Re: valgrind warnings in libibverbs & PVFS In-Reply-To: References: <47CE0A06.7060509@scl.ameslab.gov> <20080306172020.GD7754@osc.edu> Message-ID: <20080310132302.GA29707@osc.edu> rdreier at cisco.com wrote on Sun, 09 Mar 2008 20:58 -0700: > > This change makes sense, although I probably would have used &resp, > > sizeof(resp). There are a bunch of places that use > > IBV_INIT_CMD_RESP, some of which have VALGRIND annotations too. But > > there's a bunch that do not. Would make a nice cleanup to do them > > all. > > I just audited my tree, and unless I'm missing something, Troy's case > in verbs.c was the last place that still needed annotation. Or am I > blind? I think you found them all. My comment was made when looking at v1.1.1, missing a couple of recent patches from Dotan addressing the same issue. -- Pete From hrosenstock at xsigo.com Mon Mar 10 06:56:23 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 10 Mar 2008 06:56:23 -0700 Subject: [ofa-general] Re: OFED 1.3 OpenSM configuration default paths and files In-Reply-To: <20080310124459.GK30723@sashak.voltaire.com> References: <1205150491.6469.865.camel@hrosenstock-ws.xsigo.com> <20080310124459.GK30723@sashak.voltaire.com> Message-ID: <1205157383.6469.893.camel@hrosenstock-ws.xsigo.com> Hi Sasha, On Mon, 2008-03-10 at 12:44 +0000, Sasha Khapyorsky wrote: > Hi Hal, > > On 05:01 Mon 10 Mar , Hal Rosenstock wrote: > > > > Using the defaults for configurating OpenSM, > > Which defaults? Do you have prefix=/opt/ofed? Yes. > > I see the following: > > > > /* Define a default node name map file */ > > #define HAVE_DEFAULT_NODENAME_MAP "/opt/ofed/etc/opensm/ib-node-name-map" > > > > /* Define a QOS policy config file */ > > #define HAVE_DEFAULT_PARTITION_CONFIG_FILE "/opt/ofed/etc/opensm/partitions.conf" > > > > /* Define a QOS policy config file */ > > #define HAVE_DEFAULT_QOS_POLICY_FILE "/opt/ofed/etc/opensm/qos-policy.conf" > > > > /* Define OpenSM config directory */ > > #define OPENSM_CONFIG_DIR "/opt/ofed/etc/opensm" > > > > Shouldn't these not be prepended with /opt/ofed or alternately /opt/ofed/opensm without the etc ? > > I thought the /etc/opensm locations were agreed to be the defaults. > > By default it refers ${sysconfdir}/opensm, where sysconfdir is > $prefix/etc in most cases. I'm not sure how I can get the desired path/filenames now (with OFED 1.3 and beyond). Seems like in some cases prefix is wanted and others not (at least for me) :-( -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From chas at cmf.nrl.navy.mil Mon Mar 10 07:21:45 2008 From: chas at cmf.nrl.navy.mil (chas williams - CONTRACTOR) Date: Mon, 10 Mar 2008 10:21:45 -0400 Subject: [ofa-general] Re: OFED 1.3 OpenSM configuration default paths and files In-Reply-To: <1205157383.6469.893.camel@hrosenstock-ws.xsigo.com> Message-ID: <200803101421.m2AELjPU017276@cmf.nrl.navy.mil> In message <1205157383.6469.893.camel at hrosenstock-ws.xsigo.com>,Hal Rosenstock writes: >> By default it refers ${sysconfdir}/opensm, where sysconfdir is >> $prefix/etc in most cases. > >I'm not sure how I can get the desired path/filenames now (with OFED 1.3 >and beyond). Seems like in some cases prefix is wanted and others not >(at least for me) :-( i think you might be able to use ./configure --sysconfdir=/ --prefix=/opt/ofed From hrosenstock at xsigo.com Mon Mar 10 07:37:09 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 10 Mar 2008 07:37:09 -0700 Subject: [ofa-general] OpenSM installing with OFED 1.3 Message-ID: <1205159829.6469.908.camel@hrosenstock-ws.xsigo.com> Hi Sasha, I worked around this issue but when I first updated from OFED 1.2 to 1.3 I got link errors for opensm with open/close_node_name_map (in the new complib). I thought this was changed to use the one built in the tree rather than installed. Maybe this is an order thing where complib isn't built until after opensm. -- Hal From sashak at voltaire.com Mon Mar 10 07:59:41 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 10 Mar 2008 14:59:41 +0000 Subject: [ofa-general] Re: OFED 1.3 OpenSM configuration default paths and files In-Reply-To: <200803101421.m2AELjPU017276@cmf.nrl.navy.mil> References: <1205157383.6469.893.camel@hrosenstock-ws.xsigo.com> <200803101421.m2AELjPU017276@cmf.nrl.navy.mil> Message-ID: <20080310145941.GM30723@sashak.voltaire.com> On 10:21 Mon 10 Mar , chas williams - CONTRACTOR wrote: > > i think you might be able to use ./configure --sysconfdir=/ --prefix=/opt/ofed Yes, like this, but with --sysconfdir=/etc Sasha From kliteyn at dev.mellanox.co.il Mon Mar 10 07:56:23 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 10 Mar 2008 16:56:23 +0200 Subject: [ofa-general] [PATCH] opensm/osm_qos_parser: fixed compilation on byacc Message-ID: <47D54C17.1010802@dev.mellanox.co.il> Hi Sasha. Fixing compilation with byacc (bug 932). Please apply to ofed_1_3 and master. -- Yevgeny Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/osm_qos_parser.l | 1 + opensm/opensm/osm_qos_parser.y | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l index 41f8720..32e8ab3 100644 --- a/opensm/opensm/osm_qos_parser.l +++ b/opensm/opensm/osm_qos_parser.l @@ -56,6 +56,7 @@ static void save_pos(); extern int column_num; extern int line_num; extern FILE * __qos_parser_in; +extern YYSTYPE __qos_parser_lval; boolean_t in_description = FALSE; boolean_t in_list_of_hex_num_ranges = FALSE; diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y index 6595091..040c355 100644 --- a/opensm/opensm/osm_qos_parser.y +++ b/opensm/opensm/osm_qos_parser.y @@ -149,6 +149,7 @@ extern char * __qos_parser_text; extern int __qos_parser_lex (void); extern FILE * __qos_parser_in; extern int errno; +int __qos_parser_parse(); #define RESET_BUFFER __parser_tmp_struct_reset() -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Mar 10 08:01:16 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 10 Mar 2008 17:01:16 +0200 Subject: [ofa-general] [PATCH] opensm/configure.in: make lex/yacc presence mandatory Message-ID: <47D54D3C.2030706@dev.mellanox.co.il> Hi Sasha, When configure checks for Lex & Yacc, it uses AC_PROG_YACC & AC_PROG_LEX macros, which only issue a warning message if a program wasn't found. Make lex/yacc presence mandatory for configure completion. Please apply to ofed_1_3 and master. Signed-off-by: Yevgeny Kliteynik --- opensm/configure.in | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/opensm/configure.in b/opensm/configure.in index 84c1092..9a2ad1b 100644 --- a/opensm/configure.in +++ b/opensm/configure.in @@ -28,6 +28,18 @@ AC_PROG_MAKE_SET AC_PROG_YACC AC_PROG_LEX +AC_CHECK_PROGS(_YACC_,$YACC,none) +if test "$_YACC_" = "none" +then + AC_MSG_ERROR([No bison/byacc/yacc found.]) +fi + +AC_CHECK_PROGS(_LEX_,$LEX,none) +if test "$_LEX_" = "none" +then + AC_MSG_ERROR([No flex/lex found.]) +fi + dnl Checks for libraries AC_CHECK_LIB(pthread, pthread_mutex_init, [], AC_MSG_ERROR([pthread_mutex_init() not found. libosmcomp requires libpthread.])) -- 1.5.1.4 From tziporet at mellanox.co.il Mon Mar 10 08:12:56 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Mon, 10 Mar 2008 17:12:56 +0200 Subject: [ofa-general] OFED meeting agenda for today (March 10) Message-ID: <6C2C79E72C305246B504CBA17B5500C90380C4D0@mtlexch01.mtl.com> OFED meeting agenda: -------------------- 1. Feedback on OFED 1.3 release 2. Decide on fix release criteria and process (see suggestion below) 3. When do we need to have a new kernel branch for OFED 1.4 (based on kernel 2.6.25) 4. OFED meeting frequency - do we wish to continue with bi-weekly, or move to monthly meeting for this period Fix/point release criteria and process: --------------------------------------- Changes type: 1. No changes to kernel base (we should stay with 2.6.24) 2. No API changes (both in kernel and in user libs) 3. Low level driver changes - in the responsibility of the HW vendor, meaning: The HW vendor can enhance/fix bugs in the low level driver The QA for the change is the responsibility of the HW vendor 4. Core and ULPs: Critical and high priority bug fixes only 5. Add backports to support a new OS (e.g. SLES10 SP2, FC8, etc.) 6. Replace MPI revision - I am not sure here - need MPI people to respond on this Release frequency: 1. Every 4-6 weeks (unless a blocker bug arises) Release verification: 1. All vendors should run at least basic QA/verification cycle 2. If there is a change in a low level driver it's the responsibility of the HW vendor to run full QA with this HW. From hrosenstock at xsigo.com Mon Mar 10 08:16:54 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 10 Mar 2008 08:16:54 -0700 Subject: [ofa-general] Re: OFED 1.3 OpenSM configuration default paths and files In-Reply-To: <20080310145941.GM30723@sashak.voltaire.com> References: <1205157383.6469.893.camel@hrosenstock-ws.xsigo.com> <200803101421.m2AELjPU017276@cmf.nrl.navy.mil> <20080310145941.GM30723@sashak.voltaire.com> Message-ID: <1205162214.6469.914.camel@hrosenstock-ws.xsigo.com> On Mon, 2008-03-10 at 14:59 +0000, Sasha Khapyorsky wrote: > On 10:21 Mon 10 Mar , chas williams - CONTRACTOR wrote: > > > > i think you might be able to use ./configure --sysconfdir=/ --prefix=/opt/ofed > > Yes, like this, but with --sysconfdir=/etc Yes, that looks like it; thanks. -- Hal > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From meier3 at llnl.gov Mon Mar 10 08:48:53 2008 From: meier3 at llnl.gov (Timothy A. Meier) Date: Mon, 10 Mar 2008 08:48:53 -0700 Subject: [ofa-general] Re: [PATCH] opensm: provide methods for getting the OpenSM and Log In-Reply-To: <20080309200507.GF10479@sashak.voltaire.com> References: <47CDF5A0.1080509@llnl.gov> <20080309200507.GF10479@sashak.voltaire.com> Message-ID: <47D55865.5050407@llnl.gov> Hi Sasha, Sasha Khapyorsky wrote: > Hi Tim, > > On 17:21 Tue 04 Mar , Timothy A. Meier wrote: >> I wrote this patch so I could avoid passing around a pointer to the opensm, >> just to >> get access to the log. Also, it provides a method to get a pointer to the >> opensm >> as well, because there are several instances where this is needed for a >> trivial reason, >> and many functions include it as one of their arguments. > > You can pass pointer to opensm object to a function where you structure > is created (*_init() or so) and to store it there if you like. Basically > in OpenSM we are avoiding to use global non-constant data and I don't > see any really good reason to change this. > I understand, and agree in principal with the philosophy. In practice, however, this leads to many instances where a pointer to the opensm object is just being casually added to the argument list of a function. This is often done, just to get access to the log, or to pass the pointer through to another function that "might" need it. In these cases, a dependency is needlessly created in the higher level function. * I just feel that if a function has a legitimate need for the opensm object, it shouldn't impose that requirement on its entire calling tree. >> Since the opensm >> is a >> singleton, > > It could change some days. > > Sasha > No matter how it is created or initialized, the basic design assumes one and only one OpenSM object. In addition, the OpenSM object contains other objects (such as the log) that are also unique. If there comes a time when more than one opensm per thread of execution is required, I suspect many things will need to be re-examined. == How about trying this out with just the log? I think we will find this solves problems without introducing any. I'll whip up a patch for the log, and have you look at it... -- Timothy A. Meier Computer Scientist ICCD/High Performance Computing meier3 at llnl.gov From chu11 at llnl.gov Mon Mar 10 08:59:51 2008 From: chu11 at llnl.gov (Al Chu) Date: Mon, 10 Mar 2008 08:59:51 -0700 Subject: [ofa-general] [PATCH] opensm: multi lid routing balancing for updn/minhop In-Reply-To: <20080309154153.GD10479@sashak.voltaire.com> References: <1204585724.761.77.camel@cardanus.llnl.gov> <1204586117.761.79.camel@cardanus.llnl.gov> <1204657308.761.94.camel@cardanus.llnl.gov> <20080309154153.GD10479@sashak.voltaire.com> Message-ID: <1205164791.17718.56.camel@cardanus.llnl.gov> On Sun, 2008-03-09 at 15:41 +0000, Sasha Khapyorsky wrote: > On 11:01 Tue 04 Mar , Al Chu wrote: > > From 391e2e3cdd08b205bdb94203660c0aacb8212d5a Mon Sep 17 00:00:00 2001 > > From: Albert L. Chu > > Date: Mon, 3 Mar 2008 10:39:43 -0800 > > Subject: [PATCH] support balanced multi-lid routing > > > > > > Signed-off-by: Albert L. Chu > > Applied. Thanks. > > I just have a few stylistic comments below. I applied the patch as is > with hope that we can fix it later in subsequent patches (fixed some > already). Thanks. I'll keep the below in mind for the future. > An error code (ie 'ERR XXXX') should be unique, so it is better to not > change this. My bad on this one. I assumed the error codes were sequential, so I just picked the next one (the previous error codes were 3A09 and 3A0A). I'll do a grep next time :-) Al > [snip...] > > > diff --git a/opensm/include/opensm/osm_switch.h b/opensm/include/opensm/osm_switch.h > > index e2fe86d..dbb2552 100644 > > --- a/opensm/include/opensm/osm_switch.h > > +++ b/opensm/include/opensm/osm_switch.h > > @@ -158,6 +158,33 @@ typedef struct _osm_switch { > > * Switch object > > *********/ > > > > +/****s* OpenSM: Switch/osm_switch_guid_count_t > > +* NAME > > +* osm_switch_guid_count_t > > +* > > +* DESCRIPTION > > +* Stores system and node guids and the number of > > Tab character instead of spaces. > > [snip...] > > > --- a/opensm/opensm/osm_switch.c > > +++ b/opensm/opensm/osm_switch.c > > @@ -219,16 +219,99 @@ osm_switch_get_fwd_tbl_block(IN const osm_switch_t * const p_sw, > > > > /********************************************************************** > > **********************************************************************/ > > +static osm_switch_guid_count_t * > > +osm_switch_find_guid_common(IN const osm_switch_t * const p_sw, > > + IN osm_switch_guid_count_t * remote_guids, > > + IN uint16_t * p_num_remote_guids, > > + IN uint8_t port_num, > > + IN int find_sys_guid, > > + IN int find_node_guid) > > We are using osm_ prefixes only for public functions, for statics names > can be shorter. > > [snip...] > > > diff --git a/opensm/opensm/osm_ucast_mgr.c b/opensm/opensm/osm_ucast_mgr.c > > index 1aa5ea9..d7fc4d3 100644 > > --- a/opensm/opensm/osm_ucast_mgr.c > > +++ b/opensm/opensm/osm_ucast_mgr.c > > @@ -209,31 +209,21 @@ __osm_ucast_mgr_process_port(IN osm_ucast_mgr_t * const p_mgr, > > in providing better routing in LMC > 0 situations > > */ > > uint16_t lids_per_port = 1 << p_mgr->p_subn->opt.lmc; > > - uint64_t *remote_sys_guids = NULL; > > - uint64_t *remote_node_guids = NULL; > > - uint16_t num_used_sys = 0; > > - uint16_t num_used_nodes = 0; > > + osm_switch_guid_count_t *remote_guids = NULL; > > + uint16_t num_used_guids = 0; > > + osm_switch_guid_count_t *p_remote_guid_used = NULL; > > > > OSM_LOG_ENTER(p_mgr->p_log); > > > > if (lids_per_port > 1) { > > - remote_sys_guids = malloc(sizeof(uint64_t) * lids_per_port); > > - if (remote_sys_guids == NULL) { > > - OSM_LOG(p_mgr->p_log, OSM_LOG_ERROR, "ERR 3A09: " > > + remote_guids = malloc(sizeof(osm_switch_guid_count_t) * lids_per_port); > > + if (remote_guids == NULL) { > > + osm_log(p_mgr->p_log, OSM_LOG_ERROR, > > + "__osm_ucast_mgr_process_port: ERR 3A0B: " > > An error code (ie 'ERR XXXX') should be unique, so it is better to not > change this. > > Sasha -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From kliteyn at mellanox.co.il Mon Mar 10 09:03:44 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 10 Mar 2008 18:03:44 +0200 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <20080305161431.GO30723@sashak.voltaire.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> <47CEC012.5040203@mellanox.co.il> <20080305161431.GO30723@sashak.voltaire.com> Message-ID: <47D55BE0.6060503@mellanox.co.il> Sasha Khapyorsky wrote: > On 17:45 Wed 05 Mar , Yevgeny Kliteynik wrote: > >> I'm not aware of any switch FW dependency. >> As for HCAs - I'll collect the relevant info and update >> the list and the doc. >> HCAs: QoS supported by ConnectX. The latest FW release doesn't support QoS. QoS-enabled FW release (2_5_000) is planned for May. If someone wishes to get QoS-enabled FW before the official release, he should contact Mellanox FAE. Switches: QoS supported by InfiniScale III. -- Yevgeny -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Mon Mar 10 09:12:31 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 10 Mar 2008 09:12:31 -0700 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <47D55BE0.6060503@mellanox.co.il> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> <47CEC012.5040203@mellanox.co.il> <20080305161431.GO30723@sashak.voltaire.com> <47D55BE0.6060503@mellanox.co.il> Message-ID: <1205165551.6469.933.camel@hrosenstock-ws.xsigo.com> On Mon, 2008-03-10 at 18:03 +0200, Yevgeny Kliteynik wrote: > > Sasha Khapyorsky wrote: > > On 17:45 Wed 05 Mar , Yevgeny Kliteynik wrote: > > > > > I'm not aware of any switch FW dependency. > > > As for HCAs - I'll collect the relevant info and update > > > the list and the doc. > > > > > HCAs: QoS supported by ConnectX. I thought there was QoS support on some other (than Connect-X) Mellanox HCAs. Is that not true ? > The latest FW release > doesn't support QoS. QoS-enabled FW release (2_5_000) is > planned for May. If someone wishes to get QoS-enabled FW > before the official release, he should contact Mellanox FAE. > > Switches: QoS supported by InfiniScale III. Any firmware release ? The OpenSM release notes mention 0.5.0 or later. -- Hal > -- Yevgeny > > From hrosenstock at xsigo.com Mon Mar 10 09:17:12 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 10 Mar 2008 09:17:12 -0700 Subject: [ofa-general] opensm_release_notes-3.1.10.txt Message-ID: <1205165832.6469.938.camel@hrosenstock-ws.xsigo.com> Hi, Connect-X (and its firmware version) should get added to Table 3 in the OpenSM release notes. -- Hal From weiny2 at llnl.gov Mon Mar 10 09:48:49 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Mon, 10 Mar 2008 09:48:49 -0700 Subject: [ofa-general] Re: [PATCH] opensm: provide methods for getting the OpenSM and Log In-Reply-To: <20080309200507.GF10479@sashak.voltaire.com> References: <47CDF5A0.1080509@llnl.gov> <20080309200507.GF10479@sashak.voltaire.com> Message-ID: <20080310094849.14c5eb3c.weiny2@llnl.gov> On Sun, 9 Mar 2008 20:05:07 +0000 Sasha Khapyorsky wrote: > Hi Tim, > > On 17:21 Tue 04 Mar , Timothy A. Meier wrote: > > > Since the opensm > > is a > > singleton, > > It could change some days. > Sasha, do you really think this is likely? What would the use case be? Ira From mi_chimera at actionconferencecall.com Mon Mar 10 09:15:17 2008 From: mi_chimera at actionconferencecall.com (jory patty) Date: Mon, 10 Mar 2008 16:15:17 +0000 Subject: [ofa-general] To: openib-general Message-ID: <000901c882d8$04196265$f388ea88@mtqnmswv> Voted the #1 most effective male enlargement supplement by FHM readers: V_P_X_L is guaranteed to add inches within just a few short weeks. In addition to permanent, massive gains to length, you can also expect: 1. up to 20-30% gains in girth 2. Longer sex, never subside penis 3. More volume of ejaculate Satisfaction and results are absolutely guaranteed. Why hesitate?Try the only Doctor endorsed Herbal enlargement supplement today! Click here: http://sudeutos.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sguvel at thy.com Tue Mar 11 11:45:06 2008 From: sguvel at thy.com (Rose Landry) Date: Mon, 11 Mar 2008 19:45:06 +0100 Subject: [ofa-general] Die Software kann billig sein. Ihr PC ist froh Message-ID: <235511902.86231764805972@thy.com> Die Software auf allen europaischen Sprachen, fur Windows und Macintosh vorherbestimmt. Die konnen Sie momentan bekommen. Nur bezahlen und auslasten. Hier prasentiert sind nicht teuere, aber echte und vollige Produkte der Software.Jetzt wird jedes Programm leicht aufgestellt. Dabei hilft die professionelle Konsultation des Anwenderdienstes. Wir garantieren schnelle Antworte und die Moglichkeit der Ruckzahlung. Wenn Sie die Software kaufen, kaufen Sie nur die vollkommen funktionierende Softwarehttp://geocities.com/elisa.patton/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From meier3 at llnl.gov Mon Mar 10 11:45:10 2008 From: meier3 at llnl.gov (Timothy A. Meier) Date: Mon, 10 Mar 2008 11:45:10 -0700 Subject: [ofa-general] [PATCH] opensm: added convenience function for the opensm log Message-ID: <47D581B6.9020709@llnl.gov> Hi Sasha, This patch very similar to the "accessor" patch I submitted last week, but focuses primarily on the Log. So, if all your method wants to do is write to the opensm log, the OPENSM_LOG() macro should help solve the problem of "where do I get the log, or the opensm object?". The OSM_LOG() macro can be maintained for the cases when there are multiple logs. What do you think? It tries to hide the opensm object. -- Timothy A. Meier Computer Scientist ICCD/High Performance Computing meier3 at llnl.gov From -rwilke at accountingprincipals.com Mon Mar 10 10:26:23 2008 From: -rwilke at accountingprincipals.com (Kevin Aldridge) Date: Mon, 10 Mar 2008 17:26:23 +0000 Subject: [ofa-general] Re: clouded the excitement Message-ID: <000a01c882e2$071fc709$0089ae8a@bfegntss> ElaineCrapsGodard -------------- next part -------------- An HTML attachment was scrubbed... URL: From -texan at acee.net Mon Mar 10 10:35:19 2008 From: -texan at acee.net (Birgit Bagley) Date: Mon, 10 Mar 2008 17:35:19 +0000 Subject: [ofa-general] Re: were really just Message-ID: <000a01c882e4$075af188$e3e59287@ufcebs> DarioRouletteAlfred -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at dev.mellanox.co.il Mon Mar 10 12:52:57 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Mon, 10 Mar 2008 21:52:57 +0200 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <1205165551.6469.933.camel@hrosenstock-ws.xsigo.com> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> <47CEC012.5040203@mellanox.co.il> <20080305161431.GO30723@sashak.voltaire.com> <47D55BE0.6060503@mellanox.co.il> <1205165551.6469.933.camel@hrosenstock-ws.xsigo.com> Message-ID: <47D59199.1050301@mellanox.co.il> Hal Rosenstock wrote: > I thought there was QoS support on some other (than Connect-X) Mellanox > HCAs. Is that not true ? > > Only ConnectX supports QoS >> Switches: QoS supported by InfiniScale III. >> > > Any firmware release ? The OpenSM release notes mention 0.5.0 or later. > > I think any but will find out tomorrow From ardavis at ichips.intel.com Mon Mar 10 12:55:13 2008 From: ardavis at ichips.intel.com (Arlin Davis) Date: Mon, 10 Mar 2008 12:55:13 -0700 Subject: [ofa-general] max inline data query ? Message-ID: <47D59221.9000607@ichips.intel.com> Sorry, I know that we had this discussion before but could someone refresh my memory. Are there plans to add a query for max inline data size? Right now I have no other mechanism but to call ibv_create_qp, increasing inline size for a given sge count, until it fails. Thanks, -arlin From meier3 at llnl.gov Mon Mar 10 13:19:41 2008 From: meier3 at llnl.gov (Timothy A. Meier) Date: Mon, 10 Mar 2008 13:19:41 -0700 Subject: [ofa-general] [PATCH] opensm: added convenience function for the opensm log In-Reply-To: <47D581B6.9020709@llnl.gov> References: <47D581B6.9020709@llnl.gov> Message-ID: <47D597DD.5060201@llnl.gov> Hi Sasha, Oops, I forgot to attach the patch. Here it is. Timothy A. Meier wrote: > Hi Sasha, > > This patch very similar to the "accessor" patch I submitted last week, > but focuses > primarily on the Log. > > So, if all your method wants to do is write to the opensm log, the > OPENSM_LOG() macro > should help solve the problem of "where do I get the log, or the opensm > object?". > > The OSM_LOG() macro can be maintained for the cases when there are > multiple logs. > > What do you think? It tries to hide the opensm object. I guess I got a little carried away hiding things... ;^) -- Timothy A. Meier Computer Scientist ICCD/High Performance Computing 925.422.3341 meier3 at llnl.gov -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-opensm-added-convenience-function-for-the-opensm-lo.patch URL: From E-Cards at hallmark.com Mon Mar 10 13:06:58 2008 From: E-Cards at hallmark.com (hallmark.com) Date: Mon, 10 Mar 2008 16:06:58 -0400 (EDT) Subject: [ofa-general] You've received A Hallmark E-Card! Message-ID: <20080310200913.C2DB81DB410@master.techsupportme.com> An HTML attachment was scrubbed... URL: From changquing.tang at hp.com Mon Mar 10 13:42:16 2008 From: changquing.tang at hp.com (Tang, Changqing) Date: Mon, 10 Mar 2008 20:42:16 +0000 Subject: [ofa-general] max inline data query ? In-Reply-To: <47D59221.9000607@ichips.intel.com> References: <47D59221.9000607@ichips.intel.com> Message-ID: Does the inline size causes QP to use more pinned memory ? Is there any document to tell how a QP consumes physical memory, and what the relationship with inline data size is? Thanks. --CQ > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > Arlin Davis > Sent: Monday, March 10, 2008 2:55 PM > To: OpenFabrics General; Roland Dreier; Sean Hefty > Subject: [ofa-general] max inline data query ? > > Sorry, I know that we had this discussion before but could > someone refresh my memory. Are there plans to add a query for > max inline data size? > > Right now I have no other mechanism but to call > ibv_create_qp, increasing inline size for a given sge count, > until it fails. > > Thanks, > > -arlin > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From chu11 at llnl.gov Mon Mar 10 17:18:59 2008 From: chu11 at llnl.gov (Al Chu) Date: Mon, 10 Mar 2008 17:18:59 -0700 Subject: [ofa-general] [PATCH] Opensm: minor code cleanup Message-ID: <1205194739.17718.66.camel@cardanus.llnl.gov> Hey Sasha, As I began working on the switch balancing console option, I noticed that '0xff' was hard coded in a lot of places and OSM_NO_PATH is the macro used later on. This patch tries to clean some of this up by using OSM_NO_PATH consistently. Nothing fancy. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-use-OSM_NO_PATH-instead-of-hard-coded-0xff-when-appr.patch Type: text/x-patch Size: 3115 bytes Desc: not available URL: From a-4c at acstempe.com Tue Mar 11 17:38:01 2008 From: a-4c at acstempe.com (Jess Bartley) Date: Tue, 12 Mar 2008 08:38:01 +0800 Subject: [ofa-general] You told me that you will reply back Message-ID: <01c8841c$61833a80$add2227d@a-4c> Hello! I am tired today. I am nice girl that would like to chat with you. Email me at Sara at GreatGolovaSite.com only, because I am using my friend's email to write this. To see some pictures of me. From -ger at abtservicing.com Mon Mar 10 17:00:22 2008 From: -ger at abtservicing.com (Lea Denton) Date: Tue, 11 Mar 2008 00:00:22 +0000 Subject: [ofa-general] to see the value Message-ID: <000a01c88319$02de4d83$7363a6bb@qmfhagyc> Once again the strong winner hits! Our first and very promising pick of 08 To watch Tuesday, the 11th of March 2008 Analysis of OrderPro Logistics: Sym: OPLO At: 0.006 (Upwards trend, a perfect time to get in) 5day Est: 0.02 In recent Headlines: OrderPro Logistics Announces Warehouse Customer Expansion Within CMA Logistics Subsidiary Do your own research on this phenomenal new company, and you will see why it is a great addition to your 'portefollio'. Again, this one will fly up to the sky! We have never been wrong before! Get it up on your watch list on Tue the 11th, March 08 -------------- next part -------------- An HTML attachment was scrubbed... URL: From _fang at agl.aon.com Mon Mar 10 12:26:47 2008 From: _fang at agl.aon.com (Mike Bristed) Date: Mon, 10 Mar 2008 19:26:47 +0000 Subject: [ofa-general] like others, allows Message-ID: <000a01c882f3$07127d4e$faaf788d@gnxsxqe> Once again the strong winner hits! Our first and very promising pick of 08 To watch Tuesday, the 11th of March 2008 Analysis of OrderPro Logistics: Sym: OPLO At: 0.006 (Upwards trend, a perfect time to get in) 5day Est: 0.02 In recent Headlines: OrderPro Logistics Announces Warehouse Customer Expansion Within CMA Logistics Subsidiary Do your own research on this phenomenal new company, and you will see why it is a great addition to your 'portefollio'. Again, this one will fly up to the sky! We have never been wrong before! Get it up on your watch list on Tue the 11th, March 08 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Mon Mar 10 21:16:11 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 10 Mar 2008 21:16:11 -0700 Subject: [ofa-general] Re: [PATCH 1/2] IB/iSER: fix list iteration bug In-Reply-To: <47CD3B7A.2010106@voltaire.com> (Erez Zilber's message of "Tue, 04 Mar 2008 14:07:22 +0200") References: <877igkxffl.fsf@confield.dd.xiranet.com> <47CD3B7A.2010106@voltaire.com> Message-ID: thanks, applied. From rdreier at cisco.com Mon Mar 10 21:18:30 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 10 Mar 2008 21:18:30 -0700 Subject: [ofa-general] Re: [PATCH 2/2] IB/iSER: handle iser_device allocation error gracefully In-Reply-To: <47CD3C8A.20405@voltaire.com> (Erez Zilber's message of "Tue, 04 Mar 2008 14:11:54 +0200") References: <87zltgw0ut.fsf@confield.dd.xiranet.com> <47CD3C8A.20405@voltaire.com> Message-ID: thanks applied From rdreier at cisco.com Mon Mar 10 21:23:16 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 10 Mar 2008 21:23:16 -0700 Subject: [ofa-general] Re: [PATCH 2.6.25] RDMA/IWCM: don't access the cm_id after dereferencing it. In-Reply-To: <20080304224451.8548.33855.stgit@dell3.ogc.int> (Steve Wise's message of "Tue, 04 Mar 2008 16:44:52 -0600") References: <20080304224451.8548.33855.stgit@dell3.ogc.int> Message-ID: thanks, applied. From rdreier at cisco.com Mon Mar 10 21:33:11 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 10 Mar 2008 21:33:11 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get some more small post-2.6.25-rc5 fixes: Arne Redlich (2): IB/iser: Fix list iteration bug IB/iser: Handle iser_device allocation error gracefully Arthur Jones (1): MAINTAINERS: update ipath owner Jon Mason (2): RDMA/cxgb3: Return correct max_inline_data when creating a QP RDMA/cxgb3: Fix iwch_create_cq() off-by-one error Pete Wyckoff (2): Revert "IB/fmr_pool: ib_fmr_pool_flush() should flush all dirty FMRs" IB/fmr_pool: Flush all dirty FMRs from ib_fmr_pool_flush() Sean Hefty (1): IB/cm: Flush workqueue when removing device Steve Wise (1): RDMA/iwcm: Don't access a cm_id after dropping reference MAINTAINERS | 2 +- drivers/infiniband/core/cm.c | 3 +- drivers/infiniband/core/fmr_pool.c | 38 ++++++++++++--------- drivers/infiniband/core/iwcm.c | 5 ++- drivers/infiniband/hw/cxgb3/iwch_provider.c | 5 ++- drivers/infiniband/ulp/iser/iser_verbs.c | 47 ++++++++++++++------------- 6 files changed, 56 insertions(+), 44 deletions(-) diff --git a/MAINTAINERS b/MAINTAINERS index fed09b5..f229e16 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2143,7 +2143,7 @@ L: netdev at vger.kernel.org S: Maintained IPATH DRIVER: -P: Arthur Jones +P: Ralph Campbell M: infinipath at qlogic.com L: general at lists.openfabrics.org T: git git://git.qlogic.com/ipath-linux-2.6 diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c index b10ade9..4df4051 100644 --- a/drivers/infiniband/core/cm.c +++ b/drivers/infiniband/core/cm.c @@ -3759,6 +3759,7 @@ static void cm_remove_one(struct ib_device *device) port = cm_dev->port[i-1]; ib_modify_port(device, port->port_num, 0, &port_modify); ib_unregister_mad_agent(port->mad_agent); + flush_workqueue(cm.wq); cm_remove_port_fs(port); } kobject_put(&cm_dev->dev_obj); @@ -3813,6 +3814,7 @@ static void __exit ib_cm_cleanup(void) cancel_delayed_work(&timewait_info->work.work); spin_unlock_irq(&cm.lock); + ib_unregister_client(&cm_client); destroy_workqueue(cm.wq); list_for_each_entry_safe(timewait_info, tmp, &cm.timewait_list, list) { @@ -3820,7 +3822,6 @@ static void __exit ib_cm_cleanup(void) kfree(timewait_info); } - ib_unregister_client(&cm_client); class_unregister(&cm_class); idr_destroy(&cm.local_id_table); } diff --git a/drivers/infiniband/core/fmr_pool.c b/drivers/infiniband/core/fmr_pool.c index 7f00347..06d502c 100644 --- a/drivers/infiniband/core/fmr_pool.c +++ b/drivers/infiniband/core/fmr_pool.c @@ -139,7 +139,7 @@ static inline struct ib_pool_fmr *ib_fmr_cache_lookup(struct ib_fmr_pool *pool, static void ib_fmr_batch_release(struct ib_fmr_pool *pool) { int ret; - struct ib_pool_fmr *fmr, *next; + struct ib_pool_fmr *fmr; LIST_HEAD(unmap_list); LIST_HEAD(fmr_list); @@ -158,20 +158,6 @@ static void ib_fmr_batch_release(struct ib_fmr_pool *pool) #endif } - /* - * The free_list may hold FMRs that have been put there - * because they haven't reached the max_remap count. - * Invalidate their mapping as well. - */ - list_for_each_entry_safe(fmr, next, &pool->free_list, list) { - if (fmr->remap_count == 0) - continue; - hlist_del_init(&fmr->cache_node); - fmr->remap_count = 0; - list_add_tail(&fmr->fmr->list, &fmr_list); - list_move(&fmr->list, &unmap_list); - } - list_splice(&pool->dirty_list, &unmap_list); INIT_LIST_HEAD(&pool->dirty_list); pool->dirty_len = 0; @@ -384,6 +370,11 @@ void ib_destroy_fmr_pool(struct ib_fmr_pool *pool) i = 0; list_for_each_entry_safe(fmr, tmp, &pool->free_list, list) { + if (fmr->remap_count) { + INIT_LIST_HEAD(&fmr_list); + list_add_tail(&fmr->fmr->list, &fmr_list); + ib_unmap_fmr(&fmr_list); + } ib_dealloc_fmr(fmr->fmr); list_del(&fmr->list); kfree(fmr); @@ -407,8 +398,23 @@ EXPORT_SYMBOL(ib_destroy_fmr_pool); */ int ib_flush_fmr_pool(struct ib_fmr_pool *pool) { - int serial = atomic_inc_return(&pool->req_ser); + int serial; + struct ib_pool_fmr *fmr, *next; + + /* + * The free_list holds FMRs that may have been used + * but have not been remapped enough times to be dirty. + * Put them on the dirty list now so that the cleanup + * thread will reap them too. + */ + spin_lock_irq(&pool->pool_lock); + list_for_each_entry_safe(fmr, next, &pool->free_list, list) { + if (fmr->remap_count > 0) + list_move(&fmr->list, &pool->dirty_list); + } + spin_unlock_irq(&pool->pool_lock); + serial = atomic_inc_return(&pool->req_ser); wake_up_process(pool->thread); if (wait_event_interruptible(pool->force_wait, diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c index 223b1aa..81c9195 100644 --- a/drivers/infiniband/core/iwcm.c +++ b/drivers/infiniband/core/iwcm.c @@ -839,6 +839,7 @@ static void cm_work_handler(struct work_struct *_work) unsigned long flags; int empty; int ret = 0; + int destroy_id; spin_lock_irqsave(&cm_id_priv->lock, flags); empty = list_empty(&cm_id_priv->work_list); @@ -857,9 +858,9 @@ static void cm_work_handler(struct work_struct *_work) destroy_cm_id(&cm_id_priv->id); } BUG_ON(atomic_read(&cm_id_priv->refcount)==0); + destroy_id = test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags); if (iwcm_deref_id(cm_id_priv)) { - if (test_bit(IWCM_F_CALLBACK_DESTROY, - &cm_id_priv->flags)) { + if (destroy_id) { BUG_ON(!list_empty(&cm_id_priv->work_list)); free_cm_id(cm_id_priv); } diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index df1838f..b2ea921 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -189,7 +189,7 @@ static struct ib_cq *iwch_create_cq(struct ib_device *ibdev, int entries, int ve return ERR_PTR(-ENOMEM); } chp->rhp = rhp; - chp->ibcq.cqe = (1 << chp->cq.size_log2) - 1; + chp->ibcq.cqe = 1 << chp->cq.size_log2; spin_lock_init(&chp->lock); atomic_set(&chp->refcnt, 1); init_waitqueue_head(&chp->wait); @@ -819,8 +819,11 @@ static struct ib_qp *iwch_create_qp(struct ib_pd *pd, kfree(qhp); return ERR_PTR(-ENOMEM); } + attrs->cap.max_recv_wr = rqsize - 1; attrs->cap.max_send_wr = sqsize; + attrs->cap.max_inline_data = T3_MAX_INLINE; + qhp->rhp = rhp; qhp->attr.pd = php->pdid; qhp->attr.scq = ((struct iwch_cq *) attrs->send_cq)->cq.cqid; diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c index 714b8db..993f0a8 100644 --- a/drivers/infiniband/ulp/iser/iser_verbs.c +++ b/drivers/infiniband/ulp/iser/iser_verbs.c @@ -237,36 +237,32 @@ static int iser_free_ib_conn_res(struct iser_conn *ib_conn) static struct iser_device *iser_device_find_by_ib_device(struct rdma_cm_id *cma_id) { - struct list_head *p_list; - struct iser_device *device = NULL; + struct iser_device *device; mutex_lock(&ig.device_list_mutex); - p_list = ig.device_list.next; - while (p_list != &ig.device_list) { - device = list_entry(p_list, struct iser_device, ig_list); + list_for_each_entry(device, &ig.device_list, ig_list) /* find if there's a match using the node GUID */ if (device->ib_device->node_guid == cma_id->device->node_guid) - break; - } + goto inc_refcnt; - if (device == NULL) { - device = kzalloc(sizeof *device, GFP_KERNEL); - if (device == NULL) - goto out; - /* assign this device to the device */ - device->ib_device = cma_id->device; - /* init the device and link it into ig device list */ - if (iser_create_device_ib_res(device)) { - kfree(device); - device = NULL; - goto out; - } - list_add(&device->ig_list, &ig.device_list); + device = kzalloc(sizeof *device, GFP_KERNEL); + if (device == NULL) + goto out; + + /* assign this device to the device */ + device->ib_device = cma_id->device; + /* init the device and link it into ig device list */ + if (iser_create_device_ib_res(device)) { + kfree(device); + device = NULL; + goto out; } -out: - BUG_ON(device == NULL); + list_add(&device->ig_list, &ig.device_list); + +inc_refcnt: device->refcount++; +out: mutex_unlock(&ig.device_list_mutex); return device; } @@ -372,6 +368,12 @@ static void iser_addr_handler(struct rdma_cm_id *cma_id) int ret; device = iser_device_find_by_ib_device(cma_id); + if (!device) { + iser_err("device lookup/creation failed\n"); + iser_connect_error(cma_id); + return; + } + ib_conn = (struct iser_conn *)cma_id->context; ib_conn->device = device; @@ -380,7 +382,6 @@ static void iser_addr_handler(struct rdma_cm_id *cma_id) iser_err("resolve route failed: %d\n", ret); iser_connect_error(cma_id); } - return; } static void iser_route_handler(struct rdma_cm_id *cma_id) From _ko at accverschueren.com Mon Mar 10 19:54:14 2008 From: _ko at accverschueren.com (Sabrina Dillingham) Date: Tue, 11 Mar 2008 02:54:14 +0000 Subject: [ofa-general] knowing when, where Message-ID: <000701c88332$0176b526$3aa48e86@tdbbpb> Once again the strong winner hits! Our first and very promising pick of 08 To watch Tuesday, the 11th of March 2008 Analysis of OrderPro Logistics: Sym: OPLO At: 0.006 (Upwards trend, a perfect time to get in) 5day Est: 0.02 In recent Headlines: OrderPro Logistics Announces Warehouse Customer Expansion Within CMA Logistics Subsidiary Do your own research on this phenomenal new company, and you will see why it is a great addition to your 'portefollio'. Again, this one will fly up to the sky! We have never been wrong before! Get it up on your watch list on Tue the 11th, March 08 -------------- next part -------------- An HTML attachment was scrubbed... URL: From -leyva at a-zanimals.com Mon Mar 10 14:04:45 2008 From: -leyva at a-zanimals.com (Uta Baldwin) Date: Mon, 10 Mar 2008 21:04:45 +0000 Subject: [ofa-general] As the novelty of Message-ID: <000401c88301$04c28e8c$d937ec96@yblitogd> Nicole Once again the strong winner hits! Our first and very promising pick of 08 To watch Tuesday, the 11th of March 2008 Analysis of OrderPro Logistics: Sym: OPLO At: 0.006 (Upwards trend, a perfect time to get in) 5day Est: 0.02 In recent Headlines: OrderPro Logistics Announces Warehouse Customer Expansion Within CMA Logistics Subsidiary Do your own research on this phenomenal new company, and you will see why it is a great addition to your 'portefollio'. Again, this one will fly up to the sky! We have never been wrong before! Get it up on your watch list on Tue the 11th, March 08 Cover your eyes: Eye masks work, especially if they're big enough to cover the eyes completely. But people who toss and turn a lot may have trouble keeping them in place, and very light sleepers may find having something tied to their heads disturbing, says Robert D. Ballard, M.D., director of the sleep center at the National Jewish Medical and Research Center, in Denver. Soft cloth (terry or fleece) on the side that touches your face will make the mask comfortable and stable, and extra material around the nose bridge will close the pathways where ambient light might sneak through. Wash your mask once a week and it can last for year. From rajouri.jammu at gmail.com Mon Mar 10 22:52:26 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Mon, 10 Mar 2008 22:52:26 -0700 Subject: [ofa-general] max_mr limit Message-ID: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> The HCA I'm using reports max_mr = 131056. Our application needs large number of memory regions, much more than the 128K limit. Is there a way to increase this limit? I suppose it's bound by memory on the HCA. Is there an HCA that can use host memory to increase the number of MRs that can be supported. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dotanb at dev.mellanox.co.il Mon Mar 10 23:47:37 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 11 Mar 2008 08:47:37 +0200 Subject: [ofa-general] max inline data query ? In-Reply-To: References: <47D59221.9000607@ichips.intel.com> Message-ID: <47D62B09.2010009@dev.mellanox.co.il> Hi. If one wishes to use maximum inline data, he should open a QP with maximum number of s/g entries in the QP->SQ (ibv_qp_init_attr.cap.max_send_sge). The actual number of bytes which can be send as inline will be returned in the attribute ibv_qp_init_attr.cap.max_inline_data after ibv_create_qp will be finished. The inline size effects the size of memory which is being used (and pinned) for the QP buffers/queues (per WR in the QP->SQ). I'm sorry, but i don't really know what is exactly the size which is being consumed for a QP (I remember that in the past, Roland mentioned the exact calculation in one of the emails that he sent to this mailing list). thanks Dotan Tang, Changqing wrote: > Does the inline size causes QP to use more pinned memory ? Is there any document to tell > how a QP consumes physical memory, and what the relationship with inline data size is? > > Thanks. > > --CQ > > >> -----Original Message----- >> From: general-bounces at lists.openfabrics.org >> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of >> Arlin Davis >> Sent: Monday, March 10, 2008 2:55 PM >> To: OpenFabrics General; Roland Dreier; Sean Hefty >> Subject: [ofa-general] max inline data query ? >> >> Sorry, I know that we had this discussion before but could >> someone refresh my memory. Are there plans to add a query for >> max inline data size? >> >> Right now I have no other mechanism but to call >> ibv_create_qp, increasing inline size for a given sge count, >> until it fails. >> >> Thanks, >> >> -arlin >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> >> > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From yangdong at ncic.ac.cn Tue Mar 11 00:36:25 2008 From: yangdong at ncic.ac.cn (yangdong) Date: Tue, 11 Mar 2008 15:36:25 +0800 Subject: [ofa-general] AMP/RDMA performance expectation Message-ID: <47D63679.8070900@ncic.ac.cn> Hello! I implemented a message passing subsystem -- AMP for our cluster file system--DCFS3, I want AMP to run over Infiniband, I designed it in kernel(verbs). when client writing file, my protocol is implemented by RDMA WRITE, and reading file by RDMA READ, but perf of AMP is less than my expectation. Single thread (1t) read is 215MB/s, 4t is 340MB/s. And I see that the read file perf of NFS/RDMA is close to 600MB/s (1t). I made use of FMR and All physical reg ..and scatter/gather, but why the perf of AMP is lower? -- Dong Yang Institute of Computing Technology, Chinese Academy of Sciences Address: National Research Center for Intelligent Computing Systems (NCIC), P.O. Box 2704, Beijing 100080, P.R. China Phone: +86-10-62601005 From kyqeinfachguthfex at einfachguth.com Wed Mar 12 01:13:40 2008 From: kyqeinfachguthfex at einfachguth.com (Miriam Mcnamara) Date: Tue, 12 Mar 2008 10:13:40 +0200 Subject: [ofa-general] Legal software sales Message-ID: <636363437.93954101798647@einfachguth.com> Our purpose is to provide low price PC and Macintosh lawful soft and computer solutions for anyone's budget. Whether you are a corporate client, a small-scale enterprise owner, or shopping for your home personal computer, we think we can help you. SEE WHAT WE HAVE TO PROPOSE http://shttgoccqd78.blogspot.com Most demanding materials are: *Corel KPT 6: Retail price for this time - $199.00; Our only for this time - $39.95 *Adobe Premiere V1.5 Professional PC: Retail price this day - $290.00; Our for this day just - $49.95 *Adobe Audition V 1.0 PC: Retail price now - $499.00; Our this day - $39.95 *Adobe Photoshop Elements 4.0: Retail price for this day - $69.99; Our only - $39.95 *Microsoft Visual Basic RAD Professional v1.01: Retail price today - $45.00; Our only - $19.95 *Adobe Encore DVD V 1.5 PC: Retail price now - $700.00; Our this day only - $49.95 *Macromedia FreeHand MX for PC: Retail price for today - $399.00; Our this day only - $39.95 *Crystal Reports 10: Retail price today - $450.00; Our only - $39.95 COME TO US JUST NOW! http://shttgoccqd78.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Tue Mar 11 01:25:45 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 11 Mar 2008 10:25:45 +0200 Subject: [ofa-general] max inline data query ? In-Reply-To: <47D62B09.2010009@dev.mellanox.co.il> References: <47D59221.9000607@ichips.intel.com> <47D62B09.2010009@dev.mellanox.co.il> Message-ID: <47D64209.6040008@voltaire.com> Dotan Barak wrote: > If one wishes to use maximum inline data, he should open a QP with > maximum number of s/g entries in the QP->SQ > (ibv_qp_init_attr.cap.max_send_sge). The actual number of bytes which > can be send as inline will be returned in the > attribute ibv_qp_init_attr.cap.max_inline_data after ibv_create_qp > will be finished. > Its correct that the maximal inline length can be get this way, but please note that the inline size as a function of the send s/g entries has "steps" so for some values of n=|s/g| f(n+1) = f(n) but n+1 would consume more memory. Basically, inline sending tends to improve latency (only one DMA transaction, to fetch the descriptor) but it increases the memory consumption and the CPU utilization (since the user space hw library does a copy). My take here is that it should be used with care, going simple you can create the QP with the s/g count you need per your design and do inline send only when possible under this count, etc. Or. From -sadiqahmed at adityabirla.com Tue Mar 11 00:13:26 2008 From: -sadiqahmed at adityabirla.com (Katharina Lilienthal) Date: Tue, 11 Mar 2008 07:13:26 +0000 Subject: [ofa-general] out his profile on Message-ID: <000701c88356$055d1e2a$23e79d81@vfahvha> Once again the strong winner hits! Our first and very promising pick of 08 To watch Tuesday, the 11th of March 2008 Analysis of OrderPro Logistics: Sym: OPLO At: 0.006 (Upwards trend, a perfect time to get in) 5day Est: 0.02 In recent Headlines: OrderPro Logistics Announces Warehouse Customer Expansion Within CMA Logistics Subsidiary Do your own research on this phenomenal new company, and you will see why it is a great addition to your 'portefollio'. Again, this one will fly up to the sky! We have never been wrong before! Get it up on your watch list on Tue the 11th, March 08 -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Tue Mar 11 03:10:12 2008 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky Mellanox) Date: Tue, 11 Mar 2008 03:10:12 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20080311-0200 daily build status Message-ID: <20080311101012.E7D2DE60183@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_3/linux-2.6.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod --with-nes-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.24 Failed: From huiweiwang at htigc.com Wed Mar 12 04:04:35 2008 From: huiweiwang at htigc.com (Norman Case) Date: Tue, 12 Mar 2008 12:04:35 +0100 Subject: [ofa-general] MS Office 2007, AutoCAD 2008, Adobe Acrobat 8 Message-ID: <906561551.69520677085428@htigc.com> Um die echte und vollige Software in kurzer Zeit zu bekommen, braucht man nur zu bezahlen und auszulasten. Sie haben dann die Programmen auf allen europaischen Sprachen uberlassen, die fur Windows und Macintosh vorherbestimmt sind. http://geocities.com/ty.guy58/Wollen Sie das Programm aufstellen? Benutzen Sie die Hilfe der professionellen Konsultation des Anwenderdienstes. Wir antworten auf Ihre Fragen schnell und garantieren die Moglichkeit der Ruckzahlung. Kaufen die vollkommen funktionierende Software -------------- next part -------------- An HTML attachment was scrubbed... URL: From _chatwin at actsw.amat.com Tue Mar 11 02:34:20 2008 From: _chatwin at actsw.amat.com (Ursula Cuyler) Date: Tue, 11 Mar 2008 09:34:20 +0000 Subject: [ofa-general] generation are starting Message-ID: <000901c8836a$07ef4883$af31d786@nqhrtqqh> Alexander Once again the strong winner hits! Our first and very promising pick of 08 To watch Tuesday, the 11th of March 2008 Analysis of OrderPro Logistics: Sym: OPLO At: 0.006 (Upwards trend, a perfect time to get in) 5day Est: 0.02 In recent Headlines: OrderPro Logistics Announces Warehouse Customer Expansion Within CMA Logistics Subsidiary Do your own research on this phenomenal new company, and you will see why it is a great addition to your 'portefollio'. Again, this one will fly up to the sky! We have never been wrong before! Get it up on your watch list on Tue the 11th, March 08 But a steady stream of sound, no matter the volume, usually isn't disruptive. From hrosenstock at xsigo.com Tue Mar 11 04:28:15 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 04:28:15 -0700 Subject: [ofa-general] Error installing infiniband-diags Message-ID: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> Hi Sasha, When installing (OFED 1.3) infiniband-diags via make install, I get: ./config/install-sh -m 444 scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied make[3]: *** [install-data-hook] Error 1 Is it trying to move/remove the source script (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? -- Hal From hrosenstock at xsigo.com Tue Mar 11 04:31:41 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 04:31:41 -0700 Subject: [ofa-general] Error installing infiniband-diags In-Reply-To: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> Message-ID: <1205235101.6469.1046.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 04:28 -0700, Hal Rosenstock wrote: > Hi Sasha, > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > ./config/install-sh -m 444 > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > make[3]: *** [install-data-hook] Error 1 > > Is it trying to move/remove the source script > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? The following works however: ./config/install-sh -c -m 444 scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm Should -c be added ? -- Hal > -- Hal > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Tue Mar 11 04:41:34 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 04:41:34 -0700 Subject: [ofa-general] [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script Message-ID: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> infiniband-diags: Fix install of IBswcountlimits.pm script Signed-off-by: Hal Rosenstock diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index ca66e2d..d4abf9e 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -117,4 +117,4 @@ dist-hook: # install this to a default location. install-data-hook: - $(top_srcdir)/config/install-sh -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm + $(top_srcdir)/config/install-sh -c -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm From hrosenstock at xsigo.com Tue Mar 11 04:53:24 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 04:53:24 -0700 Subject: [ofa-general] [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> Message-ID: <1205236404.6469.1052.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 04:41 -0700, Hal Rosenstock wrote: > infiniband-diags: Fix install of IBswcountlimits.pm script Please install for both master and ofed_1_3. > Signed-off-by: Hal Rosenstock > > diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am > index ca66e2d..d4abf9e 100644 > --- a/infiniband-diags/Makefile.am > +++ b/infiniband-diags/Makefile.am > @@ -117,4 +117,4 @@ dist-hook: > > # install this to a default location. > install-data-hook: > - $(top_srcdir)/config/install-sh -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm > + $(top_srcdir)/config/install-sh -c -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From notebook at ktsnotebook.com Tue Mar 11 05:04:31 2008 From: notebook at ktsnotebook.com (Ktservis Notebook Servisi) Date: Tue, 11 Mar 2008 14:04:31 +0200 Subject: [ofa-general] =?iso-8859-1?q?Notebook_Servis_ve_Yedek_Par=E7a?= Message-ID: <4184-22008321112431777@xx-> KTSERVIS NOTEBOOK SERV�S� "��Z�M YOKSA - �CRET YOK" Marka Modelden Ba��ms�z Notebook Servis ve Yedek Par�a Merkezi H�zl� ��z�mler NOTEBOOK LCD EKRAN DE����M� Piyasan�n s�rekli eksikli�ini hissetti�i de�i�ik boyutlarda 6 ay garantili s�f�r orjinal Notebook Lcd Ekranlar� stoklar�m�zda. NOTEBOOK ADAPT�R En yayg�n markalar olan Compaq,HP,Dell,Sony,Fujitsu-Siemens,IBM,Toshiba,Asus ba�ta olmak �zere orjinal ve muadil notebook adapt�rleri stoklar�m�zda bulunmaktad�r. Ktservis Bili�im Hizmetleri Ltd �ti Ra�it R�za Sok. no:10 Mecidiyek�y - �stanbul Tel : 0 212 356 17 70 - 272 75 53 e-mail : info at ktsnotebook.com www : http://www.ktsnotebook.com Soru, istek ve g�r��leriniz i�in l�tfen e-mail g�nderiniz.G�nderileriniz en h�zl� �ekilde mutlaka cevaplanacakt�r. From a-annvik at actpartners.com Wed Mar 12 05:41:32 2008 From: a-annvik at actpartners.com (Alexandra Bray) Date: Tue, 12 Mar 2008 20:41:32 +0800 Subject: [ofa-general] I was looking for you Message-ID: <01c88481$747c2600$23533c3b@a-annvik> Hello! I am tired tonight. I am nice girl that would like to chat with you. Email me at Emily at WilderGoLovan.com only, because I am using my friend's email to write this. Will send some of my pictures From sashak at voltaire.com Tue Mar 11 06:12:36 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 13:12:36 +0000 Subject: [ofa-general] Re: [PATCH] opensm: provide methods for getting the OpenSM and Log In-Reply-To: <47D55865.5050407@llnl.gov> References: <47CDF5A0.1080509@llnl.gov> <20080309200507.GF10479@sashak.voltaire.com> <47D55865.5050407@llnl.gov> Message-ID: <20080311131236.GP10479@sashak.voltaire.com> Hi Tim, On 08:48 Mon 10 Mar , Timothy A. Meier wrote: > I understand, and agree in principal with the philosophy. Ok. > In practice, however, this leads to many instances where a pointer to the > opensm object is just being casually added to the argument list of a > function. It is not really needed in most cases. In OpenSM most functions work with some objects, just keep the reference to osm_opensm_t there. > * I just feel that if a function has a legitimate need for the opensm > object, > it shouldn't impose that requirement on its entire calling tree. See above. > No matter how it is created or initialized, the basic design assumes one > and only > one OpenSM object. I don't think so, it is rather opposite IMO. Of course there could be some code which violates this, but looking over history I think the basic design was multi instance ready. > In addition, the OpenSM object contains other objects > (such as > the log) that are also unique. It also can be replaced "on the fly" (I even played with it in a past). > If there comes a time when more than one opensm per thread of execution is > required, > I suspect many things will need to be re-examined. That is true. :( Sasha From sashak at voltaire.com Tue Mar 11 06:15:39 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 13:15:39 +0000 Subject: [ofa-general] Re: [PATCH] opensm: provide methods for getting the OpenSM and Log In-Reply-To: <20080310094849.14c5eb3c.weiny2@llnl.gov> References: <47CDF5A0.1080509@llnl.gov> <20080309200507.GF10479@sashak.voltaire.com> <20080310094849.14c5eb3c.weiny2@llnl.gov> Message-ID: <20080311131539.GQ10479@sashak.voltaire.com> On 09:48 Mon 10 Mar , Ira Weiny wrote: > > Sasha, do you really think this is likely? I don't really know. But it is possible. > What would the use case be? Yet in theory - GRM (global router manager) with multiple SMs in one process. Sasha From -moritoshi at advmetal.com Tue Mar 11 04:17:31 2008 From: -moritoshi at advmetal.com (bertrando morgan) Date: Tue, 11 Mar 2008 11:17:31 +0000 Subject: [ofa-general] Re: clovis Message-ID: <000601c88378$04b25d54$66054794@rbmlabmv> The best watches around. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at dev.mellanox.co.il Tue Mar 11 06:26:32 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 11 Mar 2008 15:26:32 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> Message-ID: <47D68888.3090709@mellanox.co.il> Rajouri Jammu wrote: > The HCA I'm using reports max_mr = 131056. > > Our application needs large number of memory regions, much more than > the 128K limit. > > Is there a way to increase this limit? > I suppose it's bound by memory on the HCA. Is there an HCA that can > use host memory to increase the number of MRs that can be supported. > Which HCA you are using? In mlx4 and mthca drivers there are modules parameters to change the amount of memory regions (by enlarging the number of MPTs and MTTs) for example: mlx4: log_num_mpt: log maximum number of memory protection table entries per HCA (default is 17; max is 20) log_num_mtt: log maximum number of memory translation table segments per HCA (default is 20; max is 20) mthca: num_mpt - maximum number of memory protection table entries per HCA (int) num_mtt - maximum number of memory translation table segments per HCA (int) Tziporet From -watson at abhct.com Tue Mar 11 04:12:46 2008 From: -watson at abhct.com (Katrin Reed) Date: Tue, 11 Mar 2008 11:12:46 +0000 Subject: [ofa-general] holding hands with Message-ID: <000a01c88377$05e22e0b$de239aa9@oneleprs> Once again the strong winner hits! Our first and very promising pick of 08 To watch Tuesday, the 11th of March 2008 Analysis of OrderPro Logistics: Sym: OPLO At: 0.006 (Upwards trend, a perfect time to get in) 5day Est: 0.02 In recent Headlines: OrderPro Logistics Announces Warehouse Customer Expansion Within CMA Logistics Subsidiary Do your own research on this phenomenal new company, and you will see why it is a great addition to your 'portefollio'. Again, this one will fly up to the sky! We have never been wrong before! Get it up on your watch list on Tue the 11th, March 08 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Tue Mar 11 07:10:02 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 11 Mar 2008 16:10:02 +0200 (IST) Subject: [ofa-general] [PATCH-for 2.6.25] ib/ipoib: don't drop multicast sends when it can be avoided Message-ID: When set_multicast_list is called the multicast task is restarted and the IPOIB_MCAST_STARTED bit is cleared. As a result for some window of time, multicast packets are not transmitted nor queued but rather dropped by ipoib_mcast_send. These dropped are painful specifically under two flows: first, bonding fail-over which both calls set_multicast_list on the new active slave and sends Gratuitous ARP through that slave, and second, IP_DROP_MEMBERSHIP code which both calls set_multicast_list on the device and issues IGMP leave. On both these cases, depending on the scheduling of the multicast task, the packets would be dropped. As a result in the bonding case, the failover would not be detected by the peers until their neighbouring subsystem will renew the neighbour (few tens of seconds). In the IGMP case, the IP router doesn't get an IGMP leave and would only learn on that from further probes on the group (also delay of at least few tens of seconds). Fix this by allowing transmission (or queuing) depending on the IPOIB_FLAG_OPER_UP flag. Signed-off-by: Olga Shern Signed-off-by: Or Gerlitz Index: linux-2.6.25-rc4/drivers/infiniband/ulp/ipoib/ipoib_multicast.c =================================================================== --- linux-2.6.25-rc4.orig/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2008-03-11 11:51:38.000000000 +0200 +++ linux-2.6.25-rc4/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2008-03-11 15:10:17.000000000 +0200 @@ -650,7 +650,7 @@ void ipoib_mcast_send(struct net_device */ spin_lock(&priv->lock); - if (!test_bit(IPOIB_MCAST_STARTED, &priv->flags) || + if (!test_bit(IPOIB_FLAG_OPER_UP, &priv->flags) || !priv->broadcast || !test_bit(IPOIB_MCAST_FLAG_ATTACHED, &priv->broadcast->flags)) { ++dev->stats.tx_dropped; From japelwoodsyk at elwood.com Wed Mar 12 07:20:53 2008 From: japelwoodsyk at elwood.com (Harriet Fraser) Date: Tue, 12 Mar 2008 21:20:53 +0700 Subject: [ofa-general] Software Message-ID: <088455898.93798769148352@elwood.com> Searching for the best price in software discounts? Now you will get the chance to use the software environment you wanted for a long time. And the best thing for you is, all softwares are dirt cheap. Check it up by yourself & take the softwares for lowest prices ever seen! http://rtdpdnspds57.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at mellanox.co.il Tue Mar 11 07:28:21 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Tue, 11 Mar 2008 16:28:21 +0200 Subject: [ofa-general] OFED March 10 teleconfrance meeting summary Message-ID: <6C2C79E72C305246B504CBA17B5500C90380CC85@mtlexch01.mtl.com> OFED meeting summary: ===================== 1. Decide on fix release criteria and process (see below) 2. OFED 1.4: - Target date: Sep 2008 - Will start with kernel 2.6.25 once it released - Next meeting will be devoted to define the features set. 3. OFED meeting frequency - decided to continue with bi-weekly meeting Meeting Details: ================ 1. Maintenance release criteria and process: -------------------------------------------- Changes guidelines: 1. No changes to kernel base (we should stay with 2.6.24) 2. No API changes (both in kernel and in user libs) 3. Low level driver changes - in the responsibility of the HW vendor, meaning: The HW vendor can enhance/fix bugs in the low level driver The QA for the change is the responsibility of the HW vendor 4. Core and ULPs (including MPI): Critical and high priority bug fixes only 5. Add backports to support a new OS (e.g. SLES10 SP2, FC8, etc.) Release frequency: * Will have first release few weeks after the plug-fest (depending on issues found in the interop events) * In general we wish to have a maintenance release every two months. Release verification: 1. All vendors should run at least basic QA/verification cycle 2. If there is a change in a low level driver it's the responsibility of the HW vendor to run full QA with this HW. Release process: 1. Publish the maintenance release target date 3 weeks prior to the release 2. Patches will be sent to Vlad against OFED 1.3 git repositories (this can be done ongoing throughout the period between the releases) 3. A release will be built and tested by all companies in the usual method. 2. OFED 1.4: ------------ a. Relations between Linux kernel.org and OFED kernel components: There is an agreement that it is important to drive any kernel change to kernel.org (except for modules with legal limitation like SDP). However, there is a concern that waiting for acceptance to kernel.org may hold vendors progress. So it seems that sending a patch to the kernel is a must, but acceptance to a specific kernel should not be a pre-requisite for inclusion in OFED. Since we have not get to agreement on this subject we decided to raise it in Sonoma again and decide there. Johann - can such a BOF scheduled for Sonoma? b. OFED 1.4 schedule and features: - Since we do a release every half a year and Aug is not a good month for release we decided on Sep 08 - First version of release features will be discussed in next meeting. - Final list will be closed in Sonoma based on customers' requests and vendors' plans. From info at paleodu.com Tue Mar 11 08:18:31 2008 From: info at paleodu.com (=?windows-1255?B?4Pfx6Pjp7SA=?=) Date: Tue, 11 Mar 2008 10:18:31 -0500 Subject: [ofa-general] =?windows-1255?b?7uvw8ekg+Ovp4eQg4e7n6fgg7ujl+PMg?= =?windows-1255?q?!?= Message-ID: <20080311151915.3E5FFE60B46@openfabrics.org> An HTML attachment was scrubbed... URL: From sashak at voltaire.com Tue Mar 11 08:59:46 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 15:59:46 +0000 Subject: [ofa-general] Re: OpenSM installing with OFED 1.3 In-Reply-To: <1205159829.6469.908.camel@hrosenstock-ws.xsigo.com> References: <1205159829.6469.908.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080311155946.GT30723@sashak.voltaire.com> Hi Hal, On 07:37 Mon 10 Mar , Hal Rosenstock wrote: > > I worked around this issue but when I first updated from OFED 1.2 to 1.3 How? with rpm? > I got link errors for opensm with open/close_node_name_map (in the new > complib). I thought this was changed to use the one built in the tree > rather than installed. Right, and it is how things work (at least in my setup). > Maybe this is an order thing where complib isn't > built until after opensm. No, complib build is going before opensm. Sasha From weiny2 at llnl.gov Tue Mar 11 08:50:42 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 11 Mar 2008 08:50:42 -0700 Subject: [ofa-general] Error installing infiniband-diags In-Reply-To: <1205235101.6469.1046.camel@hrosenstock-ws.xsigo.com> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> <1205235101.6469.1046.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080311085042.2366e01c.weiny2@llnl.gov> On Tue, 11 Mar 2008 04:31:41 -0700 Hal Rosenstock wrote: > On Tue, 2008-03-11 at 04:28 -0700, Hal Rosenstock wrote: > > Hi Sasha, > > > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > > > ./config/install-sh -m 444 > > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > > make[3]: *** [install-data-hook] Error 1 > > > > Is it trying to move/remove the source script > > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? > > The following works however: > ./config/install-sh -c -m 444 scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > Should -c be added ? Not according my help page: 08:48:11 > ./infiniband-diags/config/install-sh --help Usage: ./infiniband-diags/config/install-sh [OPTION]... [-T] SRCFILE DSTFILE ... Options: -c (ignored) ... Is your install program different? I see the same thing in "man install" on my machines. Ira From chu11 at llnl.gov Tue Mar 11 08:53:20 2008 From: chu11 at llnl.gov (Al Chu) Date: Tue, 11 Mar 2008 08:53:20 -0700 Subject: [ofa-general] Error installing infiniband-diags In-Reply-To: <1205235101.6469.1046.camel@hrosenstock-ws.xsigo.com> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> <1205235101.6469.1046.camel@hrosenstock-ws.xsigo.com> Message-ID: <1205250800.17718.73.camel@cardanus.llnl.gov> On Tue, 2008-03-11 at 04:31 -0700, Hal Rosenstock wrote: > On Tue, 2008-03-11 at 04:28 -0700, Hal Rosenstock wrote: > > Hi Sasha, > > > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > > > ./config/install-sh -m 444 > > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > > make[3]: *** [install-data-hook] Error 1 > > > > Is it trying to move/remove the source script > > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? > > The following works however: > ./config/install-sh -c -m 444 scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > Should -c be added ? Hey Hal, What libtool version are you working with? In mine -c does nothing. >grep ignored install-sh -c (ignored) I am running with libtool-1.5.22-6.1 from RHEL5. Al > -- Hal > > > -- Hal > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From -nag at actioninmailing.com Tue Mar 11 07:17:57 2008 From: -nag at actioninmailing.com (dieter shaw) Date: Tue, 11 Mar 2008 14:17:57 +0000 Subject: [ofa-general] Re: dilip Message-ID: <000901c88391$068fc05e$d40a789b@uplsiw> The best watches around. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.hefty at intel.com Tue Mar 11 09:20:50 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 11 Mar 2008 09:20:50 -0700 Subject: [ofa-general] max inline data query ? In-Reply-To: <47D64209.6040008@voltaire.com> References: <47D59221.9000607@ichips.intel.com> <47D62B09.2010009@dev.mellanox.co.il> <47D64209.6040008@voltaire.com> Message-ID: <000001c88393$dfae6a80$51fc070a@amr.corp.intel.com> I think what's missing is a max_inline_send field in the device attributes that the user can query. Whether there's a trade-off between sge's and supported inline data size is implementation specific. - Sean From ardavis at ichips.intel.com Tue Mar 11 09:45:38 2008 From: ardavis at ichips.intel.com (Arlin Davis) Date: Tue, 11 Mar 2008 09:45:38 -0700 Subject: [ofa-general] Re: [PATCH] [DAPL v1] uDAT: fix reuse of va_list in debugging mode In-Reply-To: <20080307183437.GJ27519@jet.pathscale.com> References: <20080307173741.2135.35188.stgit@b64-10.internal.keyresearch.com> <47D185F8.7030307@ichips.intel.com> <20080307183437.GJ27519@jet.pathscale.com> Message-ID: <47D6B732.4000700@ichips.intel.com> Patrick Marchand Latifi wrote: > Hi Arlin, > > On Fri, Mar 07, 2008 at 10:14:16AM -0800, Arlin Davis wrote: >> Patrick Marchand Latifi wrote: >>> Make sure we reinitialize the va_list since va_list is undefined >>> if a function traverses the va_list with va_arg. >>> >>> This patch fixes the uDAT debugging case when both stdout and >>> syslog output is wanted. >>> committed for both v1 and v2. Thanks. From sashak at voltaire.com Tue Mar 11 10:26:01 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 17:26:01 +0000 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_parser: fixed compilation on byacc In-Reply-To: <47D54C17.1010802@dev.mellanox.co.il> References: <47D54C17.1010802@dev.mellanox.co.il> Message-ID: <20080311172601.GW30723@sashak.voltaire.com> On 16:56 Mon 10 Mar , Yevgeny Kliteynik wrote: > Hi Sasha. > > Fixing compilation with byacc (bug 932). > Please apply to ofed_1_3 and master. > > -- Yevgeny > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From sashak at voltaire.com Tue Mar 11 10:26:19 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 17:26:19 +0000 Subject: [ofa-general] Re: [PATCH] opensm/configure.in: make lex/yacc presence mandatory In-Reply-To: <47D54D3C.2030706@dev.mellanox.co.il> References: <47D54D3C.2030706@dev.mellanox.co.il> Message-ID: <20080311172619.GX30723@sashak.voltaire.com> On 17:01 Mon 10 Mar , Yevgeny Kliteynik wrote: > > Hi Sasha, > > When configure checks for Lex & Yacc, it uses AC_PROG_YACC & AC_PROG_LEX > macros, which only issue a warning message if a program wasn't found. > Make lex/yacc presence mandatory for configure completion. > > Please apply to ofed_1_3 and master. > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From sashak at voltaire.com Tue Mar 11 10:31:19 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 17:31:19 +0000 Subject: [ofa-general] Re: [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080311173119.GY30723@sashak.voltaire.com> Hi Hal, On 04:41 Tue 11 Mar , Hal Rosenstock wrote: > infiniband-diags: Fix install of IBswcountlimits.pm script > > Signed-off-by: Hal Rosenstock > > diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am > index ca66e2d..d4abf9e 100644 > --- a/infiniband-diags/Makefile.am > +++ b/infiniband-diags/Makefile.am > @@ -117,4 +117,4 @@ dist-hook: > > # install this to a default location. > install-data-hook: > - $(top_srcdir)/config/install-sh -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm > + $(top_srcdir)/config/install-sh -c -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm '-c' does nothing (ignored) for me. Sasha From sashak at voltaire.com Tue Mar 11 10:34:21 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 17:34:21 +0000 Subject: [ofa-general] Re: Error installing infiniband-diags In-Reply-To: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080311173421.GZ30723@sashak.voltaire.com> On 04:28 Tue 11 Mar , Hal Rosenstock wrote: > Hi Sasha, > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > ./config/install-sh -m 444 > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > make[3]: *** [install-data-hook] Error 1 > > Is it trying to move/remove the source script > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? I don't know. Maybe your install-sh tries to move scripts/IBswcountlimits.pm to new place. Add '-xv' at first line of the ./config/install-sh and see. Or send me this script. Sasha From sashak at voltaire.com Tue Mar 11 10:36:46 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 17:36:46 +0000 Subject: [ofa-general] Re: opensm_release_notes-3.1.10.txt In-Reply-To: <1205165832.6469.938.camel@hrosenstock-ws.xsigo.com> References: <1205165832.6469.938.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080311173646.GA30723@sashak.voltaire.com> Hi Hal, On 09:17 Mon 10 Mar , Hal Rosenstock wrote: > > Connect-X (and its firmware version) should get added to Table 3 in the > OpenSM release notes. Ok. Let's clarify switch FW version too. Patches are welcomed. Sasha From sashak at voltaire.com Tue Mar 11 10:46:21 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 17:46:21 +0000 Subject: [ofa-general] Re: [PATCH] Opensm: minor code cleanup In-Reply-To: <1205194739.17718.66.camel@cardanus.llnl.gov> References: <1205194739.17718.66.camel@cardanus.llnl.gov> Message-ID: <20080311174621.GB30723@sashak.voltaire.com> On 17:18 Mon 10 Mar , Al Chu wrote: > Hey Sasha, > > As I began working on the switch balancing console option, I noticed > that '0xff' was hard coded in a lot of places and OSM_NO_PATH is the > macro used later on. This patch tries to clean some of this up by using > OSM_NO_PATH consistently. Nothing fancy. > > Al > > -- > Albert Chu > chu11 at llnl.gov > 925-422-5311 > Computer Scientist > High Performance Systems Division > Lawrence Livermore National Laboratory > From 06be52ceafc980cead2473d093db1a74f5921c35 Mon Sep 17 00:00:00 2001 > From: Albert L. Chu > Date: Mon, 10 Mar 2008 17:08:50 -0700 > Subject: [PATCH] use OSM_NO_PATH instead of hard coded 0xff when appropriate > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From rajouri.jammu at gmail.com Tue Mar 11 10:38:26 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Tue, 11 Mar 2008 10:38:26 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <47D68888.3090709@mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> Message-ID: <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> I think it's Arabel. Both drivers are loaded (ib_mthca and mlx4_core). How do I tell which driver's settings I should modify? What will be the max_mr value if log_num_mpt = 20? On Tue, Mar 11, 2008 at 6:26 AM, Tziporet Koren wrote: > Rajouri Jammu wrote: > > The HCA I'm using reports max_mr = 131056. > > > > Our application needs large number of memory regions, much more than > > the 128K limit. > > > > Is there a way to increase this limit? > > I suppose it's bound by memory on the HCA. Is there an HCA that can > > use host memory to increase the number of MRs that can be supported. > > > Which HCA you are using? > In mlx4 and mthca drivers there are modules parameters to change the > amount of memory regions (by enlarging the number of MPTs and MTTs) > > for example: > mlx4: > log_num_mpt: log maximum number of memory protection table > entries per HCA (default is 17; max is 20) > log_num_mtt: log maximum number of memory translation table > segments per HCA (default is 20; max is 20) > > mthca: > num_mpt - maximum number of memory protection table > entries per HCA (int) > num_mtt - maximum number of memory translation table > segments per HCA (int) > > Tziporet > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Tue Mar 11 10:39:12 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 10:39:12 -0700 Subject: [ofa-general] Error installing infiniband-diags In-Reply-To: <20080311085042.2366e01c.weiny2@llnl.gov> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> <1205235101.6469.1046.camel@hrosenstock-ws.xsigo.com> <20080311085042.2366e01c.weiny2@llnl.gov> Message-ID: <1205257152.27847.49.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 08:50 -0700, Ira Weiny wrote: > On Tue, 11 Mar 2008 04:31:41 -0700 > Hal Rosenstock wrote: > > > On Tue, 2008-03-11 at 04:28 -0700, Hal Rosenstock wrote: > > > Hi Sasha, > > > > > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > > > > > ./config/install-sh -m 444 > > > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > > > make[3]: *** [install-data-hook] Error 1 > > > > > > Is it trying to move/remove the source script > > > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? > > > > The following works however: > > ./config/install-sh -c -m 444 scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > > > Should -c be added ? > > Not according my help page: but it makes it work so that's not definitive > 08:48:11 > ./infiniband-diags/config/install-sh --help > Usage: ./infiniband-diags/config/install-sh [OPTION]... [-T] SRCFILE DSTFILE > > ... > > Options: > -c (ignored) > > ... > > Is your install program different? I see the same thing in "man install" > on my machines. config/install-sh --help config/install-sh: --help does not exist > Ira From hrosenstock at xsigo.com Tue Mar 11 10:42:37 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 10:42:37 -0700 Subject: [ofa-general] Error installing infiniband-diags In-Reply-To: <1205250800.17718.73.camel@cardanus.llnl.gov> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> <1205235101.6469.1046.camel@hrosenstock-ws.xsigo.com> <1205250800.17718.73.camel@cardanus.llnl.gov> Message-ID: <1205257357.27847.52.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 08:53 -0700, Al Chu wrote: > On Tue, 2008-03-11 at 04:31 -0700, Hal Rosenstock wrote: > > On Tue, 2008-03-11 at 04:28 -0700, Hal Rosenstock wrote: > > > Hi Sasha, > > > > > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > > > > > ./config/install-sh -m 444 > > > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > > > make[3]: *** [install-data-hook] Error 1 > > > > > > Is it trying to move/remove the source script > > > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? > > > > The following works however: > > ./config/install-sh -c -m 444 scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > > > Should -c be added ? > > Hey Hal, > > What libtool version are you working with? libtool --version ltmain.sh (GNU libtool) 1.5.6 (1.1220.2.95 2004/04/11 05:50:42) Debian: 224 $ > In mine -c does nothing. > > >grep ignored install-sh > -c (ignored) > > I am running with libtool-1.5.22-6.1 from RHEL5. My install says the same thing (-c ignored) but it does make a difference. -- Hal > Al > > > -- Hal > > > > > -- Hal > > > _______________________________________________ > > > general mailing list > > > general at lists.openfabrics.org > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Tue Mar 11 10:44:47 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 10:44:47 -0700 Subject: [ofa-general] Re: OpenSM installing with OFED 1.3 In-Reply-To: <20080311155946.GT30723@sashak.voltaire.com> References: <1205159829.6469.908.camel@hrosenstock-ws.xsigo.com> <20080311155946.GT30723@sashak.voltaire.com> Message-ID: <1205257487.27847.56.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 15:59 +0000, Sasha Khapyorsky wrote: > Hi Hal, > > On 07:37 Mon 10 Mar , Hal Rosenstock wrote: > > > > I worked around this issue but when I first updated from OFED 1.2 to 1.3 > > How? with rpm? No; building from the tree using the ofed_1_3 branch. > > I got link errors for opensm with open/close_node_name_map (in the new > > complib). I thought this was changed to use the one built in the tree > > rather than installed. > > Right, and it is how things work (at least in my setup). > > Maybe this is an order thing where complib isn't > > built until after opensm. > > No, complib build is going before opensm. Then there's some other problem which causes this. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From dwroeggim at roeggi.de Wed Mar 12 10:44:29 2008 From: dwroeggim at roeggi.de (Edith Pruitt) Date: Wed, 13 Mar 2008 01:44:29 +0800 Subject: [ofa-general] Medications that you need. Message-ID: <01c884ab$c6d23c80$062e19da@dwroeggim> Buy Must Have medications at Canada based pharmacy. No prescription at all! Same quality! Save your money, buy pills immediately! http://geocities.com/alvadonovan20 We provide confidential and secure purchase! From hrosenstock at xsigo.com Tue Mar 11 10:48:01 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 10:48:01 -0700 Subject: [ofa-general] Re: [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <20080311173119.GY30723@sashak.voltaire.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> <20080311173119.GY30723@sashak.voltaire.com> Message-ID: <1205257681.27847.59.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 17:31 +0000, Sasha Khapyorsky wrote: > Hi Hal, > > On 04:41 Tue 11 Mar , Hal Rosenstock wrote: > > infiniband-diags: Fix install of IBswcountlimits.pm script > > > > Signed-off-by: Hal Rosenstock > > > > diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am > > index ca66e2d..d4abf9e 100644 > > --- a/infiniband-diags/Makefile.am > > +++ b/infiniband-diags/Makefile.am > > @@ -117,4 +117,4 @@ dist-hook: > > > > # install this to a default location. > > install-data-hook: > > - $(top_srcdir)/config/install-sh -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm > > + $(top_srcdir)/config/install-sh -c -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm > > '-c' does nothing (ignored) for me. Even though that's what man install says in my setup, it does make a difference in fixing the permission issue. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Tue Mar 11 11:00:29 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 11:00:29 -0700 Subject: [ofa-general] Re: opensm_release_notes-3.1.10.txt In-Reply-To: <20080311173646.GA30723@sashak.voltaire.com> References: <1205165832.6469.938.camel@hrosenstock-ws.xsigo.com> <20080311173646.GA30723@sashak.voltaire.com> Message-ID: <1205258429.27847.63.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 17:36 +0000, Sasha Khapyorsky wrote: > Hi Hal, > > On 09:17 Mon 10 Mar , Hal Rosenstock wrote: > > > > Connect-X (and its firmware version) should get added to Table 3 in the > > OpenSM release notes. QoS qualification should also be noted. > Ok. Let's clarify switch FW version too. Huh ? FW version above is relative to Connect-X not switch. > Patches are welcomed. I think Mellanox is in the best position to add this line in. Yevgeny ? -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sashak at voltaire.com Tue Mar 11 11:17:19 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 18:17:19 +0000 Subject: [ofa-general] Re: [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <1205257681.27847.59.camel@hrosenstock-ws.xsigo.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> <20080311173119.GY30723@sashak.voltaire.com> <1205257681.27847.59.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080311181719.GC30723@sashak.voltaire.com> On 10:48 Tue 11 Mar , Hal Rosenstock wrote: > > > > '-c' does nothing (ignored) for me. > > Even though that's what man install says in my setup, it does make a > difference in fixing the permission issue. But we need to know what the issue is, not just add things blindly. Sasha From hrosenstock at xsigo.com Tue Mar 11 11:16:10 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 11:16:10 -0700 Subject: [ofa-general] Re: Error installing infiniband-diags In-Reply-To: <20080311173421.GZ30723@sashak.voltaire.com> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> <20080311173421.GZ30723@sashak.voltaire.com> Message-ID: <1205259370.27847.66.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 17:34 +0000, Sasha Khapyorsky wrote: > On 04:28 Tue 11 Mar , Hal Rosenstock wrote: > > Hi Sasha, > > > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > > > ./config/install-sh -m 444 > > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > > make[3]: *** [install-data-hook] Error 1 > > > > Is it trying to move/remove the source script > > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? > > I don't know. Maybe your install-sh tries to move > scripts/IBswcountlimits.pm to new place. Add '-xv' at first line of the > ./config/install-sh Looking at install-sh, I'm not sure what you mean by this exactly. -- Hal > and see. Or send me this script. > > Sasha From sashak at voltaire.com Tue Mar 11 11:32:54 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 11 Mar 2008 18:32:54 +0000 Subject: [ofa-general] Re: Error installing infiniband-diags In-Reply-To: <1205259370.27847.66.camel@hrosenstock-ws.xsigo.com> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> <20080311173421.GZ30723@sashak.voltaire.com> <1205259370.27847.66.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080311183254.GD30723@sashak.voltaire.com> On 11:16 Tue 11 Mar , Hal Rosenstock wrote: > > > > > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > > > > > ./config/install-sh -m 444 > > > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > > > make[3]: *** [install-data-hook] Error 1 > > > > > > Is it trying to move/remove the source script > > > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? > > > > I don't know. Maybe your install-sh tries to move > > scripts/IBswcountlimits.pm to new place. Add '-xv' at first line of the > > ./config/install-sh > > Looking at install-sh, I'm not sure what you mean by this exactly. I meant to add verbosity - to make first line of the script to be: #!/bin/sh -xv , rerun it (without '-c') and to see what is going on here. Sasha From hrosenstock at xsigo.com Tue Mar 11 11:26:26 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 11:26:26 -0700 Subject: [ofa-general] Re: Error installing infiniband-diags In-Reply-To: <20080311183254.GD30723@sashak.voltaire.com> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> <20080311173421.GZ30723@sashak.voltaire.com> <1205259370.27847.66.camel@hrosenstock-ws.xsigo.com> <20080311183254.GD30723@sashak.voltaire.com> Message-ID: <1205259986.27847.72.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 18:32 +0000, Sasha Khapyorsky wrote: > On 11:16 Tue 11 Mar , Hal Rosenstock wrote: > > > > > > > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > > > > > > > ./config/install-sh -m 444 > > > > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > > > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > > > > make[3]: *** [install-data-hook] Error 1 > > > > > > > > Is it trying to move/remove the source script > > > > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? > > > > > > I don't know. Maybe your install-sh tries to move > > > scripts/IBswcountlimits.pm to new place. Add '-xv' at first line of the > > > ./config/install-sh > > > > Looking at install-sh, I'm not sure what you mean by this exactly. > > I meant to add verbosity - to make first line of the script to be: > > #!/bin/sh -xv > > , rerun it (without '-c') and to see what is going on here. without -c it does a mv (from the source dir) which fails and with -c it does a cp. without -c: + '[' x '!=' x ']' + '[' x = x ']' basename "$dst" ++ basename //opt/ofed/sbin/IBswcountlimits.pm + dstfile=IBswcountlimits.pm + '[' xIBswcountlimits.pm = x ']' + : + dsttmp=//opt/ofed/sbin/_inst.29162_ + rmtmp=//opt/ofed/sbin/_rm.29162_ + trap 'status=$?; rm -f "$dsttmp" "$rmtmp" && exit $status' 0 + trap '(exit $?); exit' 1 2 13 15 + mv scripts/IBswcountlimits.pm //opt/ofed/sbin/_inst.29162_ mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied status=$?; rm -f "$dsttmp" "$rmtmp" && exit $status + status=1 + rm -f //opt/ofed/sbin/_inst.29162_ //opt/ofed/sbin/_rm.29162_ with -c: + dstfile=IBswcountlimits.pm + '[' xIBswcountlimits.pm = x ']' + : + dsttmp=//opt/ofed/sbin/_inst.6997_ + rmtmp=//opt/ofed/sbin/_rm.6997_ + trap 'status=$?; rm -f "$dsttmp" "$rmtmp" && exit $status' 0 + trap '(exit $?); exit' 1 2 13 15 + cp scripts/IBswcountlimits.pm //opt/ofed/sbin/_inst.6997_ + '[' x '!=' x ']' + : + '[' x '!=' x ']' + : + '[' x '!=' x ']' + : + '[' 'xchmod 444' '!=' x ']' + chmod 444 //opt/ofed/sbin/_inst.6997_ > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From ralph.campbell at qlogic.com Tue Mar 11 11:50:59 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 11 Mar 2008 11:50:59 -0700 Subject: [ofa-general] [PATCH 1/5] IB/ipath - fix IB compliance problems with link state vs physical state In-Reply-To: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> References: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> Message-ID: <20080311185059.20841.88023.stgit@eng-46.mv.qlogic.com> Subnet manager SetPortinfo messages distingush between changing the link state (DOWN, ARM, ACTIVE) and the link physical state (POLL, SLEEP, DISABLED). These are somewhat independent commands and affect when link width and speed changes take effect. Without this patch, a link DOWN physical state NOP command was causing the link width and speed settings to take effect which should only happen when the link physical state is goes down (either by a SMP or some link physical error like link errors exceeding the threshold). Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_common.h | 2 +- drivers/infiniband/hw/ipath/ipath_driver.c | 28 +++++++++++-------------- drivers/infiniband/hw/ipath/ipath_kernel.h | 1 + drivers/infiniband/hw/ipath/ipath_mad.c | 7 +++--- drivers/infiniband/hw/ipath/ipath_registers.h | 2 +- 5 files changed, 18 insertions(+), 22 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_common.h b/drivers/infiniband/hw/ipath/ipath_common.h index 4146210..591901a 100644 --- a/drivers/infiniband/hw/ipath/ipath_common.h +++ b/drivers/infiniband/hw/ipath/ipath_common.h @@ -75,7 +75,7 @@ #define IPATH_IB_LINKDOWN 0 #define IPATH_IB_LINKARM 1 #define IPATH_IB_LINKACTIVE 2 -#define IPATH_IB_LINKINIT 3 +#define IPATH_IB_LINKDOWN_ONLY 3 #define IPATH_IB_LINKDOWN_SLEEP 4 #define IPATH_IB_LINKDOWN_DISABLE 5 #define IPATH_IB_LINK_LOOPBACK 6 /* enable local loopback */ diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index d5ff6ca..ca4d0ac 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -851,8 +851,7 @@ void ipath_disarm_piobufs(struct ipath_devdata *dd, unsigned first, * -ETIMEDOUT state can have multiple states set, for any of several * transitions. */ -static int ipath_wait_linkstate(struct ipath_devdata *dd, u32 state, - int msecs) +int ipath_wait_linkstate(struct ipath_devdata *dd, u32 state, int msecs) { dd->ipath_state_wanted = state; wait_event_interruptible_timeout(ipath_state_wait, @@ -1656,8 +1655,8 @@ void ipath_cancel_sends(struct ipath_devdata *dd, int restore_sendctrl) static void ipath_set_ib_lstate(struct ipath_devdata *dd, int which) { static const char *what[4] = { - [0] = "DOWN", - [INFINIPATH_IBCC_LINKCMD_INIT] = "INIT", + [0] = "NOP", + [INFINIPATH_IBCC_LINKCMD_DOWN] = "DOWN", [INFINIPATH_IBCC_LINKCMD_ARMED] = "ARMED", [INFINIPATH_IBCC_LINKCMD_ACTIVE] = "ACTIVE" }; @@ -1672,9 +1671,9 @@ static void ipath_set_ib_lstate(struct ipath_devdata *dd, int which) (dd, dd->ipath_kregs->kr_ibcstatus) >> INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT) & INFINIPATH_IBCS_LINKTRAININGSTATE_MASK]); - /* flush all queued sends when going to DOWN or INIT, to be sure that + /* flush all queued sends when going to DOWN to be sure that * they don't block MAD packets */ - if (!linkcmd || linkcmd == INFINIPATH_IBCC_LINKCMD_INIT) + if (linkcmd == INFINIPATH_IBCC_LINKCMD_DOWN) ipath_cancel_sends(dd, 1); ipath_write_kreg(dd, dd->ipath_kregs->kr_ibcctrl, @@ -1687,6 +1686,13 @@ int ipath_set_linkstate(struct ipath_devdata *dd, u8 newstate) int ret; switch (newstate) { + case IPATH_IB_LINKDOWN_ONLY: + ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_DOWN << + INFINIPATH_IBCC_LINKCMD_SHIFT); + /* don't wait */ + ret = 0; + goto bail; + case IPATH_IB_LINKDOWN: ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKINITCMD_POLL << INFINIPATH_IBCC_LINKINITCMD_SHIFT); @@ -1709,16 +1715,6 @@ int ipath_set_linkstate(struct ipath_devdata *dd, u8 newstate) ret = 0; goto bail; - case IPATH_IB_LINKINIT: - if (dd->ipath_flags & IPATH_LINKINIT) { - ret = 0; - goto bail; - } - ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_INIT << - INFINIPATH_IBCC_LINKCMD_SHIFT); - lstate = IPATH_LINKINIT; - break; - case IPATH_IB_LINKARM: if (dd->ipath_flags & IPATH_LINKARMED) { ret = 0; diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 4cc0f95..ecf3f7f 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -767,6 +767,7 @@ void ipath_kreceive(struct ipath_portdata *); int ipath_setrcvhdrsize(struct ipath_devdata *, unsigned); int ipath_reset_device(int); void ipath_get_faststats(unsigned long); +int ipath_wait_linkstate(struct ipath_devdata *, u32, int); int ipath_set_linkstate(struct ipath_devdata *, u8); int ipath_set_mtu(struct ipath_devdata *, u16); int ipath_set_lid(struct ipath_devdata *, u32, u8); diff --git a/drivers/infiniband/hw/ipath/ipath_mad.c b/drivers/infiniband/hw/ipath/ipath_mad.c index d98d5f1..b34b91d 100644 --- a/drivers/infiniband/hw/ipath/ipath_mad.c +++ b/drivers/infiniband/hw/ipath/ipath_mad.c @@ -555,10 +555,7 @@ static int recv_subn_set_portinfo(struct ib_smp *smp, /* FALLTHROUGH */ case IB_PORT_DOWN: if (lstate == 0) - if (get_linkdowndefaultstate(dd)) - lstate = IPATH_IB_LINKDOWN_SLEEP; - else - lstate = IPATH_IB_LINKDOWN; + lstate = IPATH_IB_LINKDOWN_ONLY; else if (lstate == 1) lstate = IPATH_IB_LINKDOWN_SLEEP; else if (lstate == 2) @@ -568,6 +565,8 @@ static int recv_subn_set_portinfo(struct ib_smp *smp, else goto err; ipath_set_linkstate(dd, lstate); + ipath_wait_linkstate(dd, IPATH_LINKINIT | IPATH_LINKARMED | + IPATH_LINKACTIVE, 1000); break; case IB_PORT_ARMED: ipath_set_linkstate(dd, IPATH_IB_LINKARM); diff --git a/drivers/infiniband/hw/ipath/ipath_registers.h b/drivers/infiniband/hw/ipath/ipath_registers.h index 6d2a17f..92ad73a 100644 --- a/drivers/infiniband/hw/ipath/ipath_registers.h +++ b/drivers/infiniband/hw/ipath/ipath_registers.h @@ -185,7 +185,7 @@ #define INFINIPATH_IBCC_LINKINITCMD_SLEEP 3 #define INFINIPATH_IBCC_LINKINITCMD_SHIFT 16 #define INFINIPATH_IBCC_LINKCMD_MASK 0x3ULL -#define INFINIPATH_IBCC_LINKCMD_INIT 1 /* move to 0x11 */ +#define INFINIPATH_IBCC_LINKCMD_DOWN 1 /* move to 0x11 */ #define INFINIPATH_IBCC_LINKCMD_ARMED 2 /* move to 0x21 */ #define INFINIPATH_IBCC_LINKCMD_ACTIVE 3 /* move to 0x31 */ #define INFINIPATH_IBCC_LINKCMD_SHIFT 18 From Ralph at pathscale.com Tue Mar 11 11:50:54 2008 From: Ralph at pathscale.com (Ralph at pathscale.com) Date: Tue, 11 Mar 2008 11:50:54 -0700 Subject: [ofa-general] [PATCH 0/5] IB/ipath -- patches in for-roland for 2.6.25 Message-ID: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> The following patches fix a few minor bugs in the ib_ipath driver so I think they are appropriate for 2.6.25. These can also be pulled into Roland's infiniband.git repo using: git pull git://git.qlogic.com/ipath-linux-2.6 for-roland From ralph.campbell at qlogic.com Tue Mar 11 11:51:04 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 11 Mar 2008 11:51:04 -0700 Subject: [ofa-general] [PATCH 2/5] IB/ipath -- potentially wrong rnr retry counter returned in ipath_query_qp In-Reply-To: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> References: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> Message-ID: <20080311185104.20841.94999.stgit@eng-46.mv.qlogic.com> From: Patrick Marchand Latifi There can be a case where the requester's rnr retry counter (s_rnr_retry) is less than the number of rnr retries allowed per QP (s_rnr_retry_cnt). This can happen if the s_rnr_retry counter is being decremented and an ipath_query_qp call is issued during that time frame. The fix is to always return the number of rnr retries allowed per QP instead of the requester's rnr counter. Found by code review. Signed-off-by: Patrick Marchand Latifi Acked-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_qp.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 80dc623..8214c09 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -647,7 +647,7 @@ int ipath_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, attr->port_num = 1; attr->timeout = qp->timeout; attr->retry_cnt = qp->s_retry_cnt; - attr->rnr_retry = qp->s_rnr_retry; + attr->rnr_retry = qp->s_rnr_retry_cnt; attr->alt_port_num = 0; attr->alt_timeout = 0; From ralph.campbell at qlogic.com Tue Mar 11 11:51:09 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 11 Mar 2008 11:51:09 -0700 Subject: [ofa-general] [PATCH 3/5] IB/ipath -- fix RC QP initialization In-Reply-To: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> References: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> Message-ID: <20080311185109.20841.74830.stgit@eng-46.mv.qlogic.com> From: Patrick Marchand Latifi This patch fixes the initialization of the RC QP since we would rely on the queue pair type (ibqp->qp_type) being set but this field is only initialized when we return from ipath_create_qp (it is initialized by the user-level verbs library). The fix is to not depend on this field to initialize the send and the receive state of the RC QP. Signed-off-by: Patrick Marchand Latifi --- drivers/infiniband/hw/ipath/ipath_qp.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 8214c09..553d900 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -329,8 +329,9 @@ struct ipath_qp *ipath_lookup_qpn(struct ipath_qp_table *qpt, u32 qpn) /** * ipath_reset_qp - initialize the QP state to the reset state * @qp: the QP to reset + * @type: the QP type */ -static void ipath_reset_qp(struct ipath_qp *qp) +static void ipath_reset_qp(struct ipath_qp *qp, enum ib_qp_type type) { qp->remote_qpn = 0; qp->qkey = 0; @@ -342,7 +343,7 @@ static void ipath_reset_qp(struct ipath_qp *qp) qp->s_psn = 0; qp->r_psn = 0; qp->r_msn = 0; - if (qp->ibqp.qp_type == IB_QPT_RC) { + if (type == IB_QPT_RC) { qp->s_state = IB_OPCODE_RC_SEND_LAST; qp->r_state = IB_OPCODE_RC_SEND_LAST; } else { @@ -534,7 +535,7 @@ int ipath_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, switch (new_state) { case IB_QPS_RESET: - ipath_reset_qp(qp); + ipath_reset_qp(qp, ibqp->qp_type); break; case IB_QPS_ERR: @@ -839,7 +840,7 @@ struct ib_qp *ipath_create_qp(struct ib_pd *ibpd, goto bail_qp; } qp->ip = NULL; - ipath_reset_qp(qp); + ipath_reset_qp(qp, init_attr->qp_type); break; default: From ralph.campbell at qlogic.com Tue Mar 11 11:51:15 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 11 Mar 2008 11:51:15 -0700 Subject: [ofa-general] [PATCH 4/5] IB/ipath -- fix work completion entry put on the wrong cq In-Reply-To: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> References: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> Message-ID: <20080311185114.20841.41575.stgit@eng-46.mv.qlogic.com> From: Patrick Marchand Latifi A work completion entry could be placed on the wrong completion queue. This can happen when the RC QP is placed in the error state. Signed-off-by: Patrick Marchand Latifi Acked-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_qp.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 553d900..087ed31 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -415,7 +415,7 @@ int ipath_error_qp(struct ipath_qp *qp, enum ib_wc_status err) wc.wr_id = qp->r_wr_id; wc.opcode = IB_WC_RECV; wc.status = err; - ipath_cq_enter(to_icq(qp->ibqp.send_cq), &wc, 1); + ipath_cq_enter(to_icq(qp->ibqp.recv_cq), &wc, 1); } wc.status = IB_WC_WR_FLUSH_ERR; From ralph.campbell at qlogic.com Tue Mar 11 11:51:20 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 11 Mar 2008 11:51:20 -0700 Subject: [ofa-general] [PATCH 5/5] IB/ipath -- reset the retry counter for RDMA_READ_RESPONSE_MIDDLE packets In-Reply-To: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> References: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> Message-ID: <20080311185120.20841.3818.stgit@eng-46.mv.qlogic.com> From: Patrick Marchand Latifi Reset the retry counter once we get a successful RDMA_READ_RESPONSE_MIDDLE packet. This fix will prevent the requester from reporting a retry exceeded error too early. Signed-off-by: Patrick Marchand Latifi --- drivers/infiniband/hw/ipath/ipath_rc.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_rc.c b/drivers/infiniband/hw/ipath/ipath_rc.c index 459e46e..40f3e37 100644 --- a/drivers/infiniband/hw/ipath/ipath_rc.c +++ b/drivers/infiniband/hw/ipath/ipath_rc.c @@ -1196,6 +1196,10 @@ static inline void ipath_rc_rcv_resp(struct ipath_ibdev *dev, list_move_tail(&qp->timerwait, &dev->pending[dev->pending_index]); spin_unlock(&dev->pending_lock); + + if (opcode == OP(RDMA_READ_RESPONSE_MIDDLE)) + qp->s_retry = qp->s_retry_cnt; + /* * Update the RDMA receive state but do the copy w/o * holding the locks and blocking interrupts. From alasshinesixseven at mail.com Tue Mar 11 12:04:44 2008 From: alasshinesixseven at mail.com (Terry Eliza) Date: Tue, 11 Mar 2008 21:04:44 +0200 Subject: [ofa-general] Online medlcaL shop - trotcarry Message-ID: <000b01c883aa$c4704470$1263525c@causeyourstears> Hello, suchscent tillsmoke rustybed Online medshop - low prllce http://celinotheodoricaw56.blogspot.com waistand furycame wherenine mayquite debtcur longthin ninegreek stopshops From murray at tradeworx.com Tue Mar 11 13:00:45 2008 From: murray at tradeworx.com (Murray Smigel) Date: Tue, 11 Mar 2008 16:00:45 -0400 Subject: [ofa-general] installing and running ofed3 on debian Message-ID: <47D6E4ED.10208@tradeworx.com> Hi, I am running debian etch on x86-64 hardware. I am using Mellanox ConnectX HCAs. At the moment I just have two machines connected by a cable for testing. I extracted the sources from the OFED-3.0 release SRPMS and built them. I took a fresh 2.6.24 linux source tree and built a kernel after replacing the drivers/infiniband and include/rdma directories with those from the ofa_kernel-1.3. The build and install went smoothly. After the boot with the new kernel: nasnu2:/etc/udev/rules.d# dmesg | grep mlx mlx4_core: Mellanox ConnectX core driver v0.01 (May 1, 2007) mlx4_core: Initializing 0000:0b:00.0 modprobe ib_uverbs modprobe ib_umad modprobe ib_uverbs modprobe ib_ucm nasnu2:/usr/local/bin# lsmod | grep mlx mlx4_core 72608 0 nasnu2:/usr/local/bin# lsmod | grep ib_ ib_ucm 20040 0 ib_cm 35240 1 ib_ucm ib_sa 25888 1 ib_cm ib_uverbs 38192 1 ib_ucm ib_umad 19496 0 ib_mad 40356 3 ib_cm,ib_sa,ib_umad ib_core 61504 6 ib_ucm,ib_cm,ib_sa,ib_uverbs,ib_umad,ib_mad I created a udev script 90-ib.rules in /etc/udev/rules.d KERNEL=="umad*", NAME="infiniband/%k" KERNEL=="issm*", NAME="infiniband/%k" KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666" KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666" KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666" and restarted udev. When I try to start opensm I get: Mar 11 15:51:59 034721 [8D3B2320] 0x03 -> OpenSM 3.2.0 Mar 11 15:51:59 034777 [8D3B2320] 0x80 -> OpenSM 3.2.0 Mar 11 15:51:59 035360 [8D3B2320] 0x80 -> Entering DISCOVERING state Mar 11 15:51:59 035484 [8D3B2320] 0x02 -> osm_vendor_bind: Binding to port 0x0 Mar 11 15:51:59 035519 [8D3B2320] 0x01 -> osm_vendor_open_port: ERR 542A: umad_get_ca() failed Mar 11 15:51:59 035529 [8D3B2320] 0x01 -> osm_vendor_bind: ERR 5424: Unable to open port 0x0 Mar 11 15:51:59 035538 [8D3B2320] 0x01 -> osm_sm_mad_ctrl_bind: ERR 3118: Vendor specific bind failed Mar 11 15:51:59 035546 [8D3B2320] 0x01 -> osm_sm_bind: ERR 2E10: SM MAD Controller bind failed (IB_ERROR) Mar 11 15:51:59 035571 [8D3B2320] 0x01 -> osm_sa_mad_ctrl_unbind: ERR 1A11: No previous bind Mar 11 15:51:59 035873 [8D3B2320] 0x80 -> Exiting SM Can you please give me some advice as to what I am missing? Thanks, murray smigel From hrosenstock at xsigo.com Tue Mar 11 13:03:08 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 11 Mar 2008 13:03:08 -0700 Subject: [ofa-general] OFED March 10 teleconfrance meeting summary In-Reply-To: <6C2C79E72C305246B504CBA17B5500C90380CC85@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C90380CC85@mtlexch01.mtl.com> Message-ID: <1205265789.27847.85.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-11 at 16:28 +0200, Tziporet Koren wrote: A couple elaborations on the below: > OFED meeting summary: > ===================== > 1. Decide on fix release criteria and process (see below) > 2. OFED 1.4: > - Target date: Sep 2008 > - Will start with kernel 2.6.25 once it released That was one proposal but I think we need to make sure the dates work out. There was discussion of working backwards from the target date. Given that 2.6.25 is now at rc5, this will probably be OK in terms of the OFED 1.4 timeline. > - Next meeting will be devoted to define the features set. > 3. OFED meeting frequency - decided to continue with bi-weekly meeting > > Meeting Details: > ================ > > 1. Maintenance release criteria and process: > -------------------------------------------- > Changes guidelines: > 1. No changes to kernel base (we should stay with 2.6.24) > 2. No API changes (both in kernel and in user libs) > 3. Low level driver changes - in the responsibility of the HW vendor, > meaning: > The HW vendor can enhance/fix bugs in the low level driver > The QA for the change is the responsibility of the HW vendor > 4. Core and ULPs (including MPI): Critical and high priority bug fixes > only > 5. Add backports to support a new OS (e.g. SLES10 SP2, FC8, etc.) > > Release frequency: > * Will have first release few weeks after the plug-fest (depending on > issues > found in the interop events) > * In general we wish to have a maintenance release every two months. > > Release verification: > 1. All vendors should run at least basic QA/verification cycle > 2. If there is a change in a low level driver it's the responsibility of > the > HW vendor to run full QA with this HW. > > Release process: > 1. Publish the maintenance release target date 3 weeks prior to the > release > 2. Patches will be sent to Vlad against OFED 1.3 git repositories > (this can be done ongoing throughout the period between the releases) > 3. A release will be built and tested by all companies in the usual > method. > > > 2. OFED 1.4: > ------------ > a. Relations between Linux kernel.org and OFED kernel components: > There is an agreement that it is important to drive any kernel change > > to kernel.org (except for modules with legal limitation like SDP). > However, there is a concern that waiting for acceptance to kernel.org > > may hold vendors progress. So it seems that sending a patch to the > kernel > is a must, There was a proposal to allow one OFED release prior to doing this. -- Hal > but acceptance to a specific kernel should not be a > pre-requisite > for inclusion in OFED. > Since we have not get to agreement on this subject we decided to > raise it in > Sonoma again and decide there. > Johann - can such a BOF scheduled for Sonoma? > b. OFED 1.4 schedule and features: > - Since we do a release every half a year and Aug is not a good month > for release > we decided on Sep 08 > - First version of release features will be discussed in next > meeting. > - Final list will be closed in Sonoma based on customers' requests > and vendors' plans. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From aprontidyrimpuppy at mail.com Tue Mar 11 09:36:50 2008 From: aprontidyrimpuppy at mail.com (Marcelyn Brant) Date: Tue, 11 Mar 2008 17:36:50 +0100 Subject: [ofa-general] Online medlcaL shop - gravefair Message-ID: <000601c88396$1b182780$e38328bd@winkyoudapple> Hello, sincestudy pianohare lowwept Online medshop - low prllce http://consueloatp65.blogspot.com ifvifwait roofshold spoutstoop ahemsave whatsbled bridethen eastshape stoodpegs From weiny2 at llnl.gov Tue Mar 11 13:45:51 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 11 Mar 2008 13:45:51 -0700 Subject: [ofa-general] Error installing infiniband-diags In-Reply-To: <1205257152.27847.49.camel@hrosenstock-ws.xsigo.com> References: <1205234895.6469.1043.camel@hrosenstock-ws.xsigo.com> <1205235101.6469.1046.camel@hrosenstock-ws.xsigo.com> <20080311085042.2366e01c.weiny2@llnl.gov> <1205257152.27847.49.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080311134551.5074fbe0.weiny2@llnl.gov> On Tue, 11 Mar 2008 10:39:12 -0700 Hal Rosenstock wrote: > On Tue, 2008-03-11 at 08:50 -0700, Ira Weiny wrote: > > On Tue, 11 Mar 2008 04:31:41 -0700 > > Hal Rosenstock wrote: > > > > > On Tue, 2008-03-11 at 04:28 -0700, Hal Rosenstock wrote: > > > > Hi Sasha, > > > > > > > > When installing (OFED 1.3) infiniband-diags via make install, I get: > > > > > > > > ./config/install-sh -m 444 > > > > scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > > > mv: cannot remove `scripts/IBswcountlimits.pm': Permission denied > > > > make[3]: *** [install-data-hook] Error 1 > > > > > > > > Is it trying to move/remove the source script > > > > (scripts/IBswcountlimits.pm) ? Any idea on what could be wrong ? > > > > > > The following works however: > > > ./config/install-sh -c -m 444 scripts/IBswcountlimits.pm //usr/local/lib/perl/5.8.4/IBswcountlimits.pm > > > > > > Should -c be added ? > > > > Not according my help page: > > but it makes it work so that's not definitive Well from your other email it looks like it should be added. I wonder if this is a deprecated feature though? Al and I thought your version would tell you what it does. Ira > > > 08:48:11 > ./infiniband-diags/config/install-sh --help > > Usage: ./infiniband-diags/config/install-sh [OPTION]... [-T] SRCFILE DSTFILE > > > > ... > > > > Options: > > -c (ignored) > > > > ... > > > > Is your install program different? I see the same thing in "man install" > > on my machines. > > config/install-sh --help > config/install-sh: --help does not exist > > > Ira From rdreier at cisco.com Tue Mar 11 13:57:55 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 11 Mar 2008 13:57:55 -0700 Subject: [ofa-general] Re: [PATCH 0/5] IB/ipath -- patches in for-roland for 2.6.25 In-Reply-To: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> (Ralph@pathscale.com's message of "Tue, 11 Mar 2008 11:50:54 -0700") References: <20080311185054.20841.44147.stgit@eng-46.mv.qlogic.com> Message-ID: thanks, applied all 5. Do you have any pending patches for 2.6.26? In particular it would be nice to fix drivers/infiniband/hw/ipath/ipath_iba6110.c: In function 'slave_or_pri_blk': drivers/infiniband/hw/ipath/ipath_iba6110.c:843: warning: overflow in implicit constant conversion and also the sparse warnings drivers/infiniband/hw/ipath/ipath_intr.c:835:7: warning: incorrect type in assignment (different base types) drivers/infiniband/hw/ipath/ipath_intr.c:835:7: expected restricted unsigned long long [usertype] val drivers/infiniband/hw/ipath/ipath_intr.c:835:7: got unsignedlong long drivers/infiniband/hw/ipath/ipath_intr.c:836:33: warning: incorrect type in assignment (different base types) drivers/infiniband/hw/ipath/ipath_intr.c:836:33: expected restricted unsigned long long volatile [usertype] drivers/infiniband/hw/ipath/ipath_intr.c:836:33: got unsigned long [unsigned] [long] drivers/infiniband/hw/ipath/ipath_intr.c:62:3: warning: dereference of noderef expression drivers/infiniband/hw/ipath/ipath_intr.c:64:8: warning: dereference of noderef expression which all look like real bugs. - R. From rdreier at cisco.com Tue Mar 11 14:12:50 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 11 Mar 2008 14:12:50 -0700 Subject: [ofa-general] [PATCH-for 2.6.25] ib/ipoib: don't drop multicast sends when it can be avoided In-Reply-To: (Or Gerlitz's message of "Tue, 11 Mar 2008 16:10:02 +0200 (IST)") References: Message-ID: thanks, applied. From exfh at boxcarchildren.com Tue Mar 11 15:38:09 2008 From: exfh at boxcarchildren.com (Jessie Foley) Date: Tue, 11 Mar 2008 16:38:09 -0600 Subject: [ofa-general] Smart in bed games Message-ID: <01c88396$4a016e80$e008cabd@exfh> She wants you more now Details attached -------------- next part -------------- A non-text attachment was scrubbed... Name: v.zip Type: application/zip Size: 487 bytes Desc: not available URL: From daniel at pocock.com.au Tue Mar 11 16:03:31 2008 From: daniel at pocock.com.au (Daniel Pocock) Date: Tue, 11 Mar 2008 23:03:31 +0000 Subject: [ofa-general] SRP target sporadic behaviour with Solaris, VMware Message-ID: <47D70FC3.2030302@pocock.com.au> Hi, I've recently set up the SRP target module on Linux (2.6.22). Trying to access the target from various initiators (Fedora, Debian, Solaris 10, VMWare ESX 3.5) gives mixed results. The Linux clients, despite having limited configuration tools, worked immediately. I've opened a thread on the Sun forums to discuss the Solaris 10 issue: http://forum.java.sun.com/thread.jspa?threadID=5273631 On VMware: - I had to reboot my new VMware ESX server a few times before it found my 500GB target. - VMWare completely rejects a target if it doesn't have a partition table - I ran parted on Linux and then VMWare was OK - Also, the messages in VMWare gave me the impression it would clobber the whole volume, rather than just a single partition - so to avoid the possibility of losing my other partitions, I made a special target representing the intended partition rather than the entire volume. Now I have a VMware partition table nested within a partition. - VMware only seems to show one target at a time - I had created a few test targets, but I could only see one of them. Is this what other people see? ibsrpdm on the other Linux hosts shows all the targets. Any suggestions? Regards, Daniel From daniel at pocock.com.au Tue Mar 11 16:25:44 2008 From: daniel at pocock.com.au (Daniel Pocock) Date: Tue, 11 Mar 2008 23:25:44 +0000 Subject: [ofa-general] SRP target/LVM HA configuration Message-ID: <47D714F8.9090805@pocock.com.au> I'm contemplating a HA configuration based on SRP and LVM (or maybe EVMS). There are many good resources based on NFS and drbd (see http://www.linux-ha.org/HaNFS) but it would be more flexible to work with block level (e.g SRP) rather than file level (NFS). Obviously, SRP/RDMA offers a major performance benefit compared with drbd (which uses IP). Basically, I envisage the primary server having access to the secondary (passive) server's disk using SRP, and putting both the local (primary) disk and SRP (secondary) disk into RAID1. The RAID1 set would contain a volume group and multiple volumes - which would, in turn, be SRP targets (for VMware to use) or possibly NFS shares. This leads me to a few issues: - Read operations - would it be better for the primary to read from both disks, or just it's own disk? Using drbd, the secondary disk is not read unless the primary is down. However, given the performance of SRP, I suspect that reading from both the local and SRP disk would give a boost to performance. - Does it make sense to use md or LVM to combine a local disk and an SRP disk into RAID1 (or potentially RAID5)? Are there technical challenges there, given that one target is slightly faster than the other? - Fail-over - when the secondary detects that the primary is down, can it dynamically take the place of the failed SRP target? Will the end-user initiators (e.g. VMWare, see diagram below) be confused when the changeover occurs? Is there the possibility of data inconsistency if some write operations had been acknowledged by the primary but not propagated to the secondary's disk at the moment when the failure occurred? - Recovery - when the old primary comes back online as a secondary, it will need to resync it's disk - is a partial resync possible, or is full rebuild mandatory? Diagram: Disk--Primary Server-------------------SRP Initiator (e.g. VMware ESX) | +------NFS client | . SRP . (RAID1 of primary's . disk and secondary's . disk) . (fail-over path to storage | . when primary is down) Disk--Secondary Server. . . . . . From rdreier at cisco.com Tue Mar 11 18:37:49 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 11 Mar 2008 18:37:49 -0700 Subject: [ofa-general] Bug in send gather mode if we switch to CM? Message-ID: Hi Eli, It seems that the current 2.6.25-rc IPoIB code has a bug introduced with the send gather support, namely that the post_send() in ipoib_cm.c assumes that num_sge is 1, but it might not be if gather support is ever used. Since we don't have the checksum offload stuff yet, this is impossible to trigger, but it is a trap waiting to hit us when we merge the checksum offload patches. Does the patch below look correct to you? I have tentatively queued it up for 2.6.25. - R. commit 4200406b8fbbf309f4fffb339bd16c4553ae0c30 Author: Roland Dreier Date: Tue Mar 11 18:35:20 2008 -0700 IPoIB/cm: Set tx_wr.num_sge in connected mode post_send() Commit 7143740d ("IPoIB: Add send gather support") made it possible for tx_wr.num_sge to be != 1 -- this happens if send gather support is enabled. However, the code in the connected mode post_send() function assumes the old invariant, namely that tx_wr.num_sge is always 1. Fix this by explicitly setting tx_wr.num_sge to 1 in the CM post_send(). Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 52b1beb..4e8d028 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -637,6 +637,7 @@ static inline int post_send(struct ipoib_dev_priv *priv, priv->tx_sge[0].addr = addr; priv->tx_sge[0].length = len; + priv->tx_wr.num_sge = 1; priv->tx_wr.wr_id = wr_id | IPOIB_OP_CM; return ib_post_send(tx->qp, &priv->tx_wr, &bad_wr); From temice at icqmail.com Tue Mar 11 19:30:33 2008 From: temice at icqmail.com (Jess Obrien) Date: Tue, 11 Mar 2008 18:30:33 -0800 Subject: [ofa-general] You ever try it? Message-ID: <01c883a5$fe2d4000$dc6a137b@temice> Be the guy the girls are talking about Details attached -------------- next part -------------- A non-text attachment was scrubbed... Name: p.zip Type: application/zip Size: 270 bytes Desc: not available URL: From rdreier at cisco.com Tue Mar 11 20:20:38 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 11 Mar 2008 20:20:38 -0700 Subject: [ofa-general] [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: (Roland Dreier's message of "Tue, 11 Mar 2008 18:37:49 -0700") References: Message-ID: Commit 7143740d ("IPoIB: Add send gather support") made struct ipoib_tx_buf significantly larger, since the mapping member changed from a single u64 to an array with MAX_SKB_FRAGS + 1 entries. This means that allocating tx_rings with kzalloc() may fail because there is not enough contiguous memory for the new, much bigger size. Fix this regression by allocating the rings with vmalloc() instead. Signed-off-by: Roland Dreier --- I've also tentatively queued this up for 2.6.25, since it fixes a regression introduced by making the tx_ring much bigger. While writing this patch, I noticed that we seem to waste a lot of memory for connected mode tx_rings, since there's no way we would ever use gather sends with the current code. Does it make sense to use a different tx_buf structure for 2.6.26 to shrink the memory use back down? diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 4e8d028..0fd9f0a 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -38,6 +38,7 @@ #include #include #include +#include #include "ipoib.h" @@ -1031,13 +1032,13 @@ static int ipoib_cm_tx_init(struct ipoib_cm_tx *p, u32 qpn, struct ipoib_dev_priv *priv = netdev_priv(p->dev); int ret; - p->tx_ring = kzalloc(ipoib_sendq_size * sizeof *p->tx_ring, - GFP_KERNEL); + p->tx_ring = vmalloc(ipoib_sendq_size * sizeof *p->tx_ring); if (!p->tx_ring) { ipoib_warn(priv, "failed to allocate tx ring\n"); ret = -ENOMEM; goto err_tx; } + memset(p->tx_ring, 0, ipoib_sendq_size * sizeof *p->tx_ring); p->qp = ipoib_cm_create_tx_qp(p->dev, p); if (IS_ERR(p->qp)) { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index f96477a..0658f0b 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -41,6 +41,7 @@ #include #include #include +#include #include /* For ARPHRD_xxx */ @@ -887,13 +888,13 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port) goto out; } - priv->tx_ring = kzalloc(ipoib_sendq_size * sizeof *priv->tx_ring, - GFP_KERNEL); + priv->tx_ring = vmalloc(ipoib_sendq_size * sizeof *priv->tx_ring); if (!priv->tx_ring) { printk(KERN_WARNING "%s: failed to allocate TX ring (%d entries)\n", ca->name, ipoib_sendq_size); goto out_rx_ring_cleanup; } + memset(priv->tx_ring, 0, ipoib_sendq_size * sizeof *priv->tx_ring); /* priv->tx_head, tx_tail & tx_outstanding are already 0 */ From info at pleshood.com Tue Mar 11 21:49:27 2008 From: info at pleshood.com (=?windows-1255?B?4/gnIPnpIPHl7O8=?=) Date: Tue, 11 Mar 2008 23:49:27 -0500 Subject: [ofa-general] =?windows-1255?b?7uQg5Pnp6OQg7OT56eIg+OXl5+ntIOHy?= =?windows-1255?b?8fc=?= ? Message-ID: <20080312045019.D9E77E6088E@openfabrics.org> An HTML attachment was scrubbed... URL: From jackm at dev.mellanox.co.il Tue Mar 11 23:35:30 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 12 Mar 2008 08:35:30 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> Message-ID: <200803120835.31442.jackm@dev.mellanox.co.il> On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > I think it's Arabel. > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell which > driver's settings I should modify? > > What will be the max_mr value if log_num_mpt = 20? > To see which device you have, type the following command in your linux console: lspci | grep Mellanox For ConnectX (mlx4), you will see: InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) (lspci has not caught up with us yet). For Arbel (InfiniHost III), you will see either: InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA (rev a0) or, if you are running your arbel in Tavor compatibility mode: InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20) =========== If your installed HCA is a ConnectX, you should use the module parameters for mlx4. If your installed HCA is an InfiniHost III, you should use the module parameters for ib_mthca. - Jack From jackm at dev.mellanox.co.il Tue Mar 11 23:35:30 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 12 Mar 2008 08:35:30 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> Message-ID: <200803120835.31442.jackm@dev.mellanox.co.il> On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > I think it's Arabel. > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell which > driver's settings I should modify? > > What will be the max_mr value if log_num_mpt = 20? > To see which device you have, type the following command in your linux console: lspci | grep Mellanox For ConnectX (mlx4), you will see: InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) (lspci has not caught up with us yet). For Arbel (InfiniHost III), you will see either: InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA (rev a0) or, if you are running your arbel in Tavor compatibility mode: InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20) =========== If your installed HCA is a ConnectX, you should use the module parameters for mlx4. If your installed HCA is an InfiniHost III, you should use the module parameters for ib_mthca. - Jack From jackm at dev.mellanox.co.il Tue Mar 11 23:38:27 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 12 Mar 2008 08:38:27 +0200 Subject: [ofa-general] [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: References: Message-ID: <200803120838.28012.jackm@dev.mellanox.co.il> On Wednesday 12 March 2008 05:20, Roland Dreier wrote: > Commit 7143740d ("IPoIB: Add send gather support") made struct > ipoib_tx_buf significantly larger, since the mapping member changed > from a single u64 to an array with MAX_SKB_FRAGS + 1 entries.  This > means that allocating tx_rings with kzalloc() may fail because there > is not enough contiguous memory for the new, much bigger size.  Fix > this regression by allocating the rings with vmalloc() instead. > I think that this patch should be limited to 64-bit systems only. On 32-bit systems, kernel virtual memory is very limited. Won't this cause problems on 32-bit systems? - Jack From dotanb at dev.mellanox.co.il Wed Mar 12 00:11:39 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Wed, 12 Mar 2008 09:11:39 +0200 Subject: [ofa-general] AMP/RDMA performance expectation In-Reply-To: <47D63679.8070900@ncic.ac.cn> References: <47D63679.8070900@ncic.ac.cn> Message-ID: <47D7822B.8060202@dev.mellanox.co.il> Hi. Do you see performance changes between file reads/writes? (can you check the performance of your code without doing the memory registration? to check if the algorithm that you are using fro sending data hurts the performance) thanks Dotan yangdong wrote: > Hello! I implemented a message passing subsystem -- AMP for our cluster > file system--DCFS3, > I want AMP to run over Infiniband, I designed it in kernel(verbs). > when client writing file, my protocol is implemented by RDMA WRITE, and > reading file by RDMA > READ, but perf of AMP is less than my expectation. Single thread (1t) > read is 215MB/s, 4t is 340MB/s. > And I see that the read file perf of NFS/RDMA is close to 600MB/s (1t). > I made use of FMR and All physical reg ..and scatter/gather, but why the > perf of AMP is lower? > > > > From bart.vanassche at gmail.com Wed Mar 12 00:10:45 2008 From: bart.vanassche at gmail.com (Bart Van Assche) Date: Wed, 12 Mar 2008 08:10:45 +0100 Subject: [ofa-general] SRP target sporadic behaviour with Solaris, VMware In-Reply-To: <47D70FC3.2030302@pocock.com.au> References: <47D70FC3.2030302@pocock.com.au> Message-ID: On Wed, Mar 12, 2008 at 12:03 AM, Daniel Pocock wrote: > > I've recently set up the SRP target module on Linux (2.6.22). > > Trying to access the target from various initiators (Fedora, Debian, > Solaris 10, VMWare ESX 3.5) gives mixed results. > > The Linux clients, despite having limited configuration tools, worked > immediately. > > I've opened a thread on the Sun forums to discuss the Solaris 10 issue: > > http://forum.java.sun.com/thread.jspa?threadID=5273631 > > On VMware: > - I had to reboot my new VMware ESX server a few times before it found > my 500GB target. > - VMWare completely rejects a target if it doesn't have a partition > table - I ran parted on Linux and then VMWare was OK > - Also, the messages in VMWare gave me the impression it would clobber > the whole volume, rather than just a single partition - so to avoid the > possibility of losing my other partitions, I made a special target > representing the intended partition rather than the entire volume. Now > I have a VMware partition table nested within a partition. > - VMware only seems to show one target at a time - I had created a few > test targets, but I could only see one of them. Is this what other > people see? ibsrpdm on the other Linux hosts shows all the targets. My experience with SRP is as follows (with Linux 2.6.24 + SCST + SRPT as target): * Linux SRP initiator: works perfectly. * OpenSolaris SRP initiator: I could not get Sun's SRP initiator working on OpenSolaris. I even asked a Solaris expert to help me, but he couldn't get the SRP initiator working either. * VMware ESX 3.5 + Mellanox InfiniBand drivers (released in January 2008): until now I only have tested a setup with a single target. When doing a lot of I/O over the SRP connection, after about 10 minutes the virtual machine running on VMware starts logging communication errors. I reported this yesterday to Mellanox support, and Mellanox is currently working on this issue. Note: I had to upgrade the InfiniBand switch firmware before the ESX server was able to find the SRP target. Bart. From E-Cards at hallmark.com Tue Mar 11 12:48:08 2008 From: E-Cards at hallmark.com (hallmark.com) Date: 11 Mar 2008 19:48:08 -0000 Subject: [ofa-general] You've received A Hallmark E-Card! Message-ID: <20080311194808.1907.qmail@mail.student.seark.edu> An HTML attachment was scrubbed... URL: From bsi at blimpp.com Wed Mar 12 01:10:47 2008 From: bsi at blimpp.com (Lee Cramer) Date: Wed, 12 Mar 2008 16:10:47 +0800 Subject: [ofa-general] You ever try it? Message-ID: <01c8845b$a2472100$0883353c@bsi> Attract the mate of your dreams Details attached -------------- next part -------------- A non-text attachment was scrubbed... Name: p.zip Type: application/zip Size: 270 bytes Desc: not available URL: From augustine508 at yahoo.com Wed Mar 12 01:42:54 2008 From: augustine508 at yahoo.com (JUSTIN EKINS) Date: Wed, 12 Mar 2008 09:42:54 +0100 (CET) Subject: [ofa-general] Dear Friend Message-ID: <20080312084254.E956443C09@epsilon.artcom.pl> Dear Friend I am very happy to inform you about my surcess regarding the arrangment for the delivery of your bank draft but the manager of eko bank told me that before the check will get to you that it will expire so i told him to cash the $2.5,000.00. All the delivery arrangement of the $2.5,000.00 in cach has been made with BLUE-DART COURIER COMPANY and i have paid for the delivery fee together with the registration fee which cost me $480 except from the insurance fee $100 which you are to send to the BLUE-DART COURIER COMPANY to enable them carry out the transaction. Contact them with the bellow information and they will instruct you where to send the $100 to them,do comply with them and let me know when you recieve your fund. Name: DR JUSTIN EKINS Email bdc_company4 at yahoo.com 1.YOUR FULL NAME 2.YOUR HOME ADDRESS. 3.YOUR CURRENT HOME TELEPHONE NUMBER. 4.MOBILE PHONE NUMBER 5.A COPY OF YOUR PICTURE Best Regards Dr Ansell Wallas From dwsiciliaonlinem at siciliaonline.it Wed Mar 12 03:15:12 2008 From: dwsiciliaonlinem at siciliaonline.it (Nolan Ayala) Date: Wed, 12 Mar 2008 18:15:12 +0800 Subject: [ofa-general] Medications that you need. Message-ID: <01c8846d$03327800$01a79cdb@dwsiciliaonlinem> Buy Must Have medications at Canada based pharmacy. No prescription at all! Same quality! Save your money, buy pills immediately! http://geocities.com/ricocochran41 We provide confidential and secure purchase! From a-allens at access-receivables.com Wed Mar 12 04:45:45 2008 From: a-allens at access-receivables.com (Hilary Lott) Date: Wed, 12 Mar 2008 17:15:45 +0530 Subject: [ofa-general] Don't miss to see my pic Message-ID: <036641505.32644536287539@access-receivables.com> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Alexis at WilderGoLovan.com only, because I am using my friend's email to write this. Wanna see some pictures of me? From a-andyc at agivera.com Wed Mar 12 04:53:48 2008 From: a-andyc at agivera.com (Keisha Ayers) Date: Wed, 12 Mar 2008 12:53:48 +0100 Subject: [ofa-general] Do you want to see my pic? Message-ID: <01c88440$1d099600$aee1004d@a-andyc> Hello! I am tired tonight. I am nice girl that would like to chat with you. Email me at Pia at WilderGoLovan.com only, because I am using my friend's email to write this. Hope you like my pictures. From ogerlitz at voltaire.com Wed Mar 12 05:12:59 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 12 Mar 2008 14:12:59 +0200 Subject: [ofa-general] [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: References: Message-ID: <47D7C8CB.4030203@voltaire.com> Roland Dreier wrote: > Commit 7143740d ("IPoIB: Add send gather support") made struct > ipoib_tx_buf significantly larger, since the mapping member changed > from a single u64 to an array with MAX_SKB_FRAGS + 1 entries. This > means that allocating tx_rings with kzalloc() may fail because there > is not enough contiguous memory for the new, much bigger size. Fix > this regression by allocating the rings with vmalloc() instead. > Don't you need to free them with vfree() in this case? Or. From safak at baduk.or.kr Wed Mar 12 05:30:06 2008 From: safak at baduk.or.kr (safak at baduk.or.kr) Date: Wed, 12 Mar 2008 04:30:06 -0800 Subject: [ofa-general] Your ecard is waiting! Message-ID: <002e01c8843c$cda69610$197c4ba3@rpcf> Please click here to view your Crazy Funny Ecard Online http://122.162.211.215/ From vlad at dev.mellanox.co.il Wed Mar 12 06:07:59 2008 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Wed, 12 Mar 2008 15:07:59 +0200 Subject: [ofa-general] installing and running ofed3 on debian In-Reply-To: <47D6E4ED.10208@tradeworx.com> References: <47D6E4ED.10208@tradeworx.com> Message-ID: <47D7D5AF.9040109@dev.mellanox.co.il> Murray Smigel wrote: > Hi, > I am running debian etch on x86-64 hardware. I am using Mellanox > ConnectX HCAs. > At the moment I just have two machines connected by a cable for testing. > > I extracted the sources from the OFED-3.0 release SRPMS and built them. > I took a fresh 2.6.24 linux source tree and built a kernel after > replacing the > drivers/infiniband and include/rdma directories with those from the > ofa_kernel-1.3. > The build and install went smoothly. > > After the boot with the new kernel: > nasnu2:/etc/udev/rules.d# dmesg | grep mlx > mlx4_core: Mellanox ConnectX core driver v0.01 (May 1, 2007) > mlx4_core: Initializing 0000:0b:00.0 > > > modprobe ib_uverbs > modprobe ib_umad > modprobe ib_uverbs > modprobe ib_ucm > > nasnu2:/usr/local/bin# lsmod | grep mlx > mlx4_core 72608 0 > Try: > modprobe mlx4_ib Regards, Vladimir From eli at dev.mellanox.co.il Wed Mar 12 06:33:16 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Wed, 12 Mar 2008 15:33:16 +0200 Subject: [ofa-general] Re: Bug in send gather mode if we switch to CM? In-Reply-To: References: Message-ID: <1205328796.26105.19.camel@mtls03> On Tue, 2008-03-11 at 18:37 -0700, Roland Dreier wrote: > Hi Eli, > > It seems that the current 2.6.25-rc IPoIB code has a bug introduced > with the send gather support, namely that the post_send() in > ipoib_cm.c assumes that num_sge is 1, but it might not be if gather > support is ever used. Since we don't have the checksum offload stuff > yet, this is impossible to trigger, but it is a trap waiting to hit us > when we merge the checksum offload patches. > > Does the patch below look correct to you? I have tentatively queued > it up for 2.6.25. This looks to me like a bug even with gather support in CM - when you change from UD to CM mode, priv->tx_wr.num_sge might have a value larger than 1 and the HW could access invalid pointers if the next send is in the CM path. So I think this is a correct fix. > > - R. > > commit 4200406b8fbbf309f4fffb339bd16c4553ae0c30 > Author: Roland Dreier > Date: Tue Mar 11 18:35:20 2008 -0700 > > IPoIB/cm: Set tx_wr.num_sge in connected mode post_send() > > Commit 7143740d ("IPoIB: Add send gather support") made it possible > for tx_wr.num_sge to be != 1 -- this happens if send gather support is > enabled. However, the code in the connected mode post_send() function > assumes the old invariant, namely that tx_wr.num_sge is always 1. Fix > this by explicitly setting tx_wr.num_sge to 1 in the CM post_send(). > > Signed-off-by: Roland Dreier > > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > index 52b1beb..4e8d028 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > @@ -637,6 +637,7 @@ static inline int post_send(struct ipoib_dev_priv *priv, > priv->tx_sge[0].addr = addr; > priv->tx_sge[0].length = len; > > + priv->tx_wr.num_sge = 1; > priv->tx_wr.wr_id = wr_id | IPOIB_OP_CM; > > return ib_post_send(tx->qp, &priv->tx_wr, &bad_wr); From hrosenstock at xsigo.com Wed Mar 12 06:33:19 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 12 Mar 2008 06:33:19 -0700 Subject: [ofa-general] [PATCH] opensm/osm_qos_parser: fixed compilation on byacc In-Reply-To: <47D54C17.1010802@dev.mellanox.co.il> References: <47D54C17.1010802@dev.mellanox.co.il> Message-ID: <1205328799.27847.125.camel@hrosenstock-ws.xsigo.com> Hi Yevgeny, On Mon, 2008-03-10 at 16:56 +0200, Yevgeny Kliteynik wrote: > Hi Sasha. > > Fixing compilation with byacc (bug 932). > Please apply to ofed_1_3 and master. > > -- Yevgeny > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/opensm/osm_qos_parser.l | 1 + > opensm/opensm/osm_qos_parser.y | 1 + > 2 files changed, 2 insertions(+), 0 deletions(-) > > diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l > index 41f8720..32e8ab3 100644 > --- a/opensm/opensm/osm_qos_parser.l > +++ b/opensm/opensm/osm_qos_parser.l > @@ -56,6 +56,7 @@ static void save_pos(); > extern int column_num; > extern int line_num; > extern FILE * __qos_parser_in; > +extern YYSTYPE __qos_parser_lval; > > boolean_t in_description = FALSE; > boolean_t in_list_of_hex_num_ranges = FALSE; > diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y > index 6595091..040c355 100644 > --- a/opensm/opensm/osm_qos_parser.y > +++ b/opensm/opensm/osm_qos_parser.y > @@ -149,6 +149,7 @@ extern char * __qos_parser_text; > extern int __qos_parser_lex (void); > extern FILE * __qos_parser_in; > extern int errno; > +int __qos_parser_parse(); > > #define RESET_BUFFER __parser_tmp_struct_reset() Are the following warnings benign: osm_qos_parser.tab.c: In function `__qos_parser_parse': osm_qos_parser.tab.c:1723: warning: suggest parentheses around assignment used as truth value osm_qos_parser.tab.c:1771: warning: label `yyerrlab' defined but not used osm_qos_parser.tab.c:1766: warning: label `yynewerror' defined but not used ../include/complib/cl_qcomppool.h: At top level: osm_qos_parser_y.c:2: warning: `yysccsid' defined but not used Also, have you tested regressed this with some QoS setup ? Thanks. -- Hal From eli at dev.mellanox.co.il Wed Mar 12 06:41:15 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Wed, 12 Mar 2008 15:41:15 +0200 Subject: [ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: References: Message-ID: <1205329275.26105.28.camel@mtls03> On Tue, 2008-03-11 at 20:20 -0700, Roland Dreier wrote: > Commit 7143740d ("IPoIB: Add send gather support") made struct > ipoib_tx_buf significantly larger, since the mapping member changed > from a single u64 to an array with MAX_SKB_FRAGS + 1 entries. This > means that allocating tx_rings with kzalloc() may fail because there > is not enough contiguous memory for the new, much bigger size. Fix > this regression by allocating the rings with vmalloc() instead. > > Signed-off-by: Roland Dreier > --- > I've also tentatively queued this up for 2.6.25, since it fixes a > regression introduced by making the tx_ring much bigger. > > While writing this patch, I noticed that we seem to waste a lot of > memory for connected mode tx_rings, since there's no way we would ever > use gather sends with the current code. Does it make sense to use a > different tx_buf structure for 2.6.26 to shrink the memory use back down? I noticed that too and I am not sure what the solution should be. From one side, for CM mode we clear NETIF_F_SG but then we use an MTU of 65520 bytes and it might not be so trivial for the networking stack to provide SKB with a linear data so large. Maybe we could use a struct defined like this: struct ipoib_tx_buf { struct sk_buff *skb; u64 mapping[0]; }; and when allocating, allocate as much memory as needed to cover the required size of the "mapping" array. We'll have to use kmllaoc n times according to the size of the tx ring. This could be good also for 32 bit systems where vmalloc area is small. > > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > index 4e8d028..0fd9f0a 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > @@ -38,6 +38,7 @@ > #include > #include > #include > +#include > > #include "ipoib.h" > > @@ -1031,13 +1032,13 @@ static int ipoib_cm_tx_init(struct ipoib_cm_tx *p, u32 qpn, > struct ipoib_dev_priv *priv = netdev_priv(p->dev); > int ret; > > - p->tx_ring = kzalloc(ipoib_sendq_size * sizeof *p->tx_ring, > - GFP_KERNEL); > + p->tx_ring = vmalloc(ipoib_sendq_size * sizeof *p->tx_ring); > if (!p->tx_ring) { > ipoib_warn(priv, "failed to allocate tx ring\n"); > ret = -ENOMEM; > goto err_tx; > } > + memset(p->tx_ring, 0, ipoib_sendq_size * sizeof *p->tx_ring); > > p->qp = ipoib_cm_create_tx_qp(p->dev, p); > if (IS_ERR(p->qp)) { > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c > index f96477a..0658f0b 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c > @@ -41,6 +41,7 @@ > #include > #include > #include > +#include > > #include /* For ARPHRD_xxx */ > > @@ -887,13 +888,13 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port) > goto out; > } > > - priv->tx_ring = kzalloc(ipoib_sendq_size * sizeof *priv->tx_ring, > - GFP_KERNEL); > + priv->tx_ring = vmalloc(ipoib_sendq_size * sizeof *priv->tx_ring); > if (!priv->tx_ring) { > printk(KERN_WARNING "%s: failed to allocate TX ring (%d entries)\n", > ca->name, ipoib_sendq_size); > goto out_rx_ring_cleanup; > } > + memset(priv->tx_ring, 0, ipoib_sendq_size * sizeof *priv->tx_ring); > > /* priv->tx_head, tx_tail & tx_outstanding are already 0 */ > From murray at tradeworx.com Wed Mar 12 06:46:40 2008 From: murray at tradeworx.com (Murray Smigel) Date: Wed, 12 Mar 2008 09:46:40 -0400 Subject: [ofa-general] installing and running ofed3 on debian In-Reply-To: <47D7D5AF.9040109@dev.mellanox.co.il> References: <47D6E4ED.10208@tradeworx.com> <47D7D5AF.9040109@dev.mellanox.co.il> Message-ID: <47D7DEC0.1020105@tradeworx.com> Vladimir Sokolovsky wrote: > Murray Smigel wrote: > >> Hi, >> I am running debian etch on x86-64 hardware. I am using Mellanox >> ConnectX HCAs. >> At the moment I just have two machines connected by a cable for testing. >> >> I extracted the sources from the OFED-3.0 release SRPMS and built them. >> I took a fresh 2.6.24 linux source tree and built a kernel after >> replacing the >> drivers/infiniband and include/rdma directories with those from the >> ofa_kernel-1.3. >> The build and install went smoothly. >> >> After the boot with the new kernel: >> nasnu2:/etc/udev/rules.d# dmesg | grep mlx >> mlx4_core: Mellanox ConnectX core driver v0.01 (May 1, 2007) >> mlx4_core: Initializing 0000:0b:00.0 >> >> >> modprobe ib_uverbs >> modprobe ib_umad >> modprobe ib_uverbs >> modprobe ib_ucm >> >> nasnu2:/usr/local/bin# lsmod | grep mlx >> mlx4_core 72608 0 >> > > Try: > > modprobe mlx4_ib > ** Thanks, that fixed it. Stupid me... murray smigel > Regards, > Vladimir From rdreier at cisco.com Wed Mar 12 07:48:20 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 07:48:20 -0700 Subject: [ofa-general] [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: <200803120838.28012.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Wed, 12 Mar 2008 08:38:27 +0200") References: <200803120838.28012.jackm@dev.mellanox.co.il> Message-ID: > I think that this patch should be limited to 64-bit systems only. > On 32-bit systems, kernel virtual memory is very limited. Won't > this cause problems on 32-bit systems? How big is a tx_ring really going to be? 256 KB in extreme cases? I don't think using 256 KB of vmalloc space is a problem even on 32-bit systems, which have a minimum of 128 MB of vmalloc space. From rdreier at cisco.com Wed Mar 12 07:51:32 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 07:51:32 -0700 Subject: [ofa-general] [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: <47D7C8CB.4030203@voltaire.com> (Or Gerlitz's message of "Wed, 12 Mar 2008 14:12:59 +0200") References: <47D7C8CB.4030203@voltaire.com> Message-ID: > Don't you need to free them with vfree() in this case? Oops, of course I do. Thanks: commit 10313cbb92206450b450e14f2b3f6ccde42d9a34 Author: Roland Dreier Date: Wed Mar 12 07:51:03 2008 -0700 IPoIB: Allocate priv->tx_ring with vmalloc() Commit 7143740d ("IPoIB: Add send gather support") made struct ipoib_tx_buf significantly larger, since the mapping member changed from a single u64 to an array with MAX_SKB_FRAGS + 1 entries. This means that allocating tx_rings with kzalloc() may fail because there is not enough contiguous memory for the new, much bigger size. Fix this regression by allocating the rings with vmalloc() instead. Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 4e8d028..2490b2d 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -38,6 +38,7 @@ #include #include #include +#include #include "ipoib.h" @@ -1031,13 +1032,13 @@ static int ipoib_cm_tx_init(struct ipoib_cm_tx *p, u32 qpn, struct ipoib_dev_priv *priv = netdev_priv(p->dev); int ret; - p->tx_ring = kzalloc(ipoib_sendq_size * sizeof *p->tx_ring, - GFP_KERNEL); + p->tx_ring = vmalloc(ipoib_sendq_size * sizeof *p->tx_ring); if (!p->tx_ring) { ipoib_warn(priv, "failed to allocate tx ring\n"); ret = -ENOMEM; goto err_tx; } + memset(p->tx_ring, 0, ipoib_sendq_size * sizeof *p->tx_ring); p->qp = ipoib_cm_create_tx_qp(p->dev, p); if (IS_ERR(p->qp)) { @@ -1078,6 +1079,7 @@ err_id: ib_destroy_qp(p->qp); err_qp: p->qp = NULL; + vfree(p->tx_ring); err_tx: return ret; } @@ -1128,7 +1130,7 @@ timeout: if (p->qp) ib_destroy_qp(p->qp); - kfree(p->tx_ring); + vfree(p->tx_ring); kfree(p); } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index f96477a..5728204 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -41,6 +41,7 @@ #include #include #include +#include #include /* For ARPHRD_xxx */ @@ -887,13 +888,13 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port) goto out; } - priv->tx_ring = kzalloc(ipoib_sendq_size * sizeof *priv->tx_ring, - GFP_KERNEL); + priv->tx_ring = vmalloc(ipoib_sendq_size * sizeof *priv->tx_ring); if (!priv->tx_ring) { printk(KERN_WARNING "%s: failed to allocate TX ring (%d entries)\n", ca->name, ipoib_sendq_size); goto out_rx_ring_cleanup; } + memset(priv->tx_ring, 0, ipoib_sendq_size * sizeof *priv->tx_ring); /* priv->tx_head, tx_tail & tx_outstanding are already 0 */ @@ -903,7 +904,7 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port) return 0; out_tx_ring_cleanup: - kfree(priv->tx_ring); + vfree(priv->tx_ring); out_rx_ring_cleanup: kfree(priv->rx_ring); @@ -928,7 +929,7 @@ void ipoib_dev_cleanup(struct net_device *dev) ipoib_ib_dev_cleanup(dev); kfree(priv->rx_ring); - kfree(priv->tx_ring); + vfree(priv->tx_ring); priv->rx_ring = NULL; priv->tx_ring = NULL; From dwsesolutionsm at sesolutions.com Wed Mar 12 08:41:57 2008 From: dwsesolutionsm at sesolutions.com (Joy Proctor) Date: Wed, 12 Mar 2008 15:41:57 +0000 Subject: [ofa-general] Enjoy gambling in the most reliable online casino! Message-ID: <01c88457$9a8cd080$c2dd0a5c@dwsesolutionsm> Feel like gambling? Golden Gate Casino is worth your attention. All popular casino games, great welcome bonus, fast to download, easy to use and completely free software! Golden Gate Casino guarantees competent customer support for all players, quick response in case you have question or problem and instant payouts. Fair gaming only! http://geocities.com/wmcervantes989 Choose Golden Gate Casino! From ssufficool at rov.sbcounty.gov Wed Mar 12 08:57:39 2008 From: ssufficool at rov.sbcounty.gov (Sufficool, Stanley) Date: Wed, 12 Mar 2008 08:57:39 -0700 Subject: [ofa-general] SRP target/LVM HA configuration In-Reply-To: <47D714F8.9090805@pocock.com.au> Message-ID: I had looked at this configuration as well and decided to use the volume management at the clients to mirror the data. Windows LDM mirrored across 2 SRPT servers and Linux md RAID 1 mirrored. This provides transparent failover and the SRP client/host will rebuild the slices that went offline. > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > Daniel Pocock > Sent: Tuesday, March 11, 2008 4:26 PM > To: general at lists.openfabrics.org > Subject: [ofa-general] SRP target/LVM HA configuration > > > > > > > I'm contemplating a HA configuration based on SRP and LVM (or > maybe EVMS). > > There are many good resources based on NFS and drbd (see > http://www.linux-ha.org/HaNFS) but it would be more flexible to work > with block level (e.g SRP) rather than file level (NFS). Obviously, > SRP/RDMA offers a major performance benefit compared with drbd (which > uses IP). > > Basically, I envisage the primary server having access to the > secondary > (passive) server's disk using SRP, and putting both the local > (primary) > disk and SRP (secondary) disk into RAID1. The RAID1 set > would contain a > volume group and multiple volumes - which would, in turn, be > SRP targets > (for VMware to use) or possibly NFS shares. > > This leads me to a few issues: > > - Read operations - would it be better for the primary to > read from both > disks, or just it's own disk? Using drbd, the secondary disk is not > read unless the primary is down. However, given the > performance of SRP, > I suspect that reading from both the local and SRP disk would give a > boost to performance. > > - Does it make sense to use md or LVM to combine a local disk > and an SRP > disk into RAID1 (or potentially RAID5)? Are there technical > challenges > there, given that one target is slightly faster than the other? > > - Fail-over - when the secondary detects that the primary is > down, can > it dynamically take the place of the failed SRP target? Will the > end-user initiators (e.g. VMWare, see diagram below) be confused when > the changeover occurs? Is there the possibility of data > inconsistency > if some write operations had been acknowledged by the primary but not > propagated to the secondary's disk at the moment when the > failure occurred? > > - Recovery - when the old primary comes back online as a > secondary, it > will need to resync it's disk - is a partial resync possible, > or is full > rebuild mandatory? > > > Diagram: > > > Disk--Primary Server-------------------SRP Initiator (e.g. VMware ESX) > | +------NFS client > | . > SRP . > (RAID1 of primary's . > disk and secondary's . > disk) . (fail-over path to storage > | . when primary is down) > Disk--Secondary Server. . . . . . > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From kliteyn at dev.mellanox.co.il Wed Mar 12 09:12:10 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 12 Mar 2008 18:12:10 +0200 Subject: [ofa-general] [PATCH] opensm/osm_qos_parser: fixed compilation on byacc In-Reply-To: <1205328799.27847.125.camel@hrosenstock-ws.xsigo.com> References: <47D54C17.1010802@dev.mellanox.co.il> <1205328799.27847.125.camel@hrosenstock-ws.xsigo.com> Message-ID: <47D800DA.2040102@dev.mellanox.co.il> Hi Hal, Hal Rosenstock wrote: > Hi Yevgeny, > > On Mon, 2008-03-10 at 16:56 +0200, Yevgeny Kliteynik wrote: >> Hi Sasha. >> >> Fixing compilation with byacc (bug 932). >> Please apply to ofed_1_3 and master. >> >> -- Yevgeny >> >> Signed-off-by: Yevgeny Kliteynik >> --- >> opensm/opensm/osm_qos_parser.l | 1 + >> opensm/opensm/osm_qos_parser.y | 1 + >> 2 files changed, 2 insertions(+), 0 deletions(-) >> >> diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l >> index 41f8720..32e8ab3 100644 >> --- a/opensm/opensm/osm_qos_parser.l >> +++ b/opensm/opensm/osm_qos_parser.l >> @@ -56,6 +56,7 @@ static void save_pos(); >> extern int column_num; >> extern int line_num; >> extern FILE * __qos_parser_in; >> +extern YYSTYPE __qos_parser_lval; >> >> boolean_t in_description = FALSE; >> boolean_t in_list_of_hex_num_ranges = FALSE; >> diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y >> index 6595091..040c355 100644 >> --- a/opensm/opensm/osm_qos_parser.y >> +++ b/opensm/opensm/osm_qos_parser.y >> @@ -149,6 +149,7 @@ extern char * __qos_parser_text; >> extern int __qos_parser_lex (void); >> extern FILE * __qos_parser_in; >> extern int errno; >> +int __qos_parser_parse(); >> >> #define RESET_BUFFER __parser_tmp_struct_reset() > > Are the following warnings benign: > > osm_qos_parser.tab.c: In function `__qos_parser_parse': > osm_qos_parser.tab.c:1723: warning: suggest parentheses around assignment used as truth value > osm_qos_parser.tab.c:1771: warning: label `yyerrlab' defined but not used > osm_qos_parser.tab.c:1766: warning: label `yynewerror' defined but not used > ../include/complib/cl_qcomppool.h: At top level: > osm_qos_parser_y.c:2: warning: `yysccsid' defined but not used > > Also, have you tested regressed this with some QoS setup ? What version of byacc do you have? What distro? I checked it on RH4U4 with bison and byacc - no warnings, and parser works fine. -- Yevgeny > Thanks. > > -- Hal > From rdreier at cisco.com Wed Mar 12 09:47:44 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 09:47:44 -0700 Subject: [ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: <1205329275.26105.28.camel@mtls03> (Eli Cohen's message of "Wed, 12 Mar 2008 15:41:15 +0200") References: <1205329275.26105.28.camel@mtls03> Message-ID: > I noticed that too and I am not sure what the solution should be. From > one side, for CM mode we clear NETIF_F_SG but then we use an MTU of > 65520 bytes and it might not be so trivial for the networking stack to > provide SKB with a linear data so large. But that's the network stack's problem. Since we clear NETIF_F_SG (and since we don't have checksum offload for connected mode) there's no way that we would ever get an skb with any fragments when we enable CM. And the current IPoIB CM code has no handling for non-linear skbs. So I don't see any point in worrying about this. > Maybe we could use a struct defined like this: > > struct ipoib_tx_buf { > struct sk_buff *skb; > u64 mapping[0]; > }; > > and when allocating, allocate as much memory as needed to cover the > required size of the "mapping" array. We'll have to use kmllaoc n times > according to the size of the tx ring. This could be good also for 32 bit > systems where vmalloc area is small. We could do this for the main (datagram mode) tx_ring, but then we eat an extra pointer indirection for every send. I suspect that's worse than the TLB miss we take by using vmalloc(). - R. From chu11 at llnl.gov Wed Mar 12 09:57:15 2008 From: chu11 at llnl.gov (Al Chu) Date: Wed, 12 Mar 2008 09:57:15 -0700 Subject: [ofa-general] [PATCH] Opensm: switchbalance console option Message-ID: <1205341036.17718.87.camel@cardanus.llnl.gov> Hey Sasha, Here's a patch that does the previously discussed switchbalance console option. Algorithmically it does pretty much the exact same thing as the check_lft_balance script, but everything is faster of course b/c opensm already knows everything. Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-add-switchbalance-command-to-console.patch Type: text/x-patch Size: 5792 bytes Desc: not available URL: From weiny2 at llnl.gov Wed Mar 12 10:23:33 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 12 Mar 2008 10:23:33 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and 1.2.1 specification Message-ID: <20080312102333.49779c75.weiny2@llnl.gov> While making changes to the DataDetails for trap 144 I noticed that trap 256 and 259 were wrong. This patch should fix them acording to both the 1.2 and 1.2.1 spec. IRa >From 9ad1430729151fab371b98fce82e28b33c49f036 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Mon, 10 Mar 2008 13:09:45 -0700 Subject: [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and 1.2.1 specification Signed-off-by: Ira K. Weiny --- opensm/include/iba/ib_types.h | 12 +++++++----- 1 files changed, 7 insertions(+), 5 deletions(-) diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index a026ac7..f80d0d5 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -7160,13 +7160,13 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated struct _ntc_256 { // total: 54 ib_net16_t pad1; // 2 ib_net16_t lid; // 2 - ib_net16_t pad2; // 2 + ib_net16_t dr_slid; // 2 uint8_t method; // 1 - uint8_t pad3; // 1 + uint8_t pad2; // 1 ib_net16_t attr_id; // 2 ib_net32_t attr_mod; // 4 ib_net64_t mkey; // 8 - uint8_t dr_slid; // 1 + uint8_t pad3; // 1 uint8_t dr_trunc_hop; // 1 uint8_t dr_rtn_path[30]; // 30 } PACK_SUFFIX ntc_256; @@ -7189,9 +7189,11 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated ib_net16_t data_valid; // 2 ib_net16_t lid1; // 2 ib_net16_t lid2; // 2 - ib_net32_t key; // 4 + ib_net16_t key; // 4 uint8_t sl; // 1 - ib_net32_t qp1; // 4 + uint8_t qp1_msb; // 1 + ib_net16_t qp1_lsb; // 2 + uint8_t pad; // 1 uint8_t qp2_msb; // 1 ib_net16_t qp2_lsb; // 2 ib_gid_t gid1; // 16 -- 1.5.1 From weiny2 at llnl.gov Wed Mar 12 10:23:35 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 12 Mar 2008 10:23:35 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 Message-ID: <20080312102335.5eecc8d0.weiny2@llnl.gov> Sasha, I don't know if this is how you would like to define the masks or not. Let me know, Ira >From c5c6b2f6bd3e889a4eb6fa462507fcbf25a433e9 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Mon, 10 Mar 2008 11:19:29 -0700 Subject: [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 Signed-off-by: Ira K. Weiny --- opensm/include/iba/ib_types.h | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-) diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 649ef1c..a026ac7 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -7143,9 +7143,11 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated struct _ntc_144 { ib_net16_t pad1; - ib_net16_t lid; // lid where capability mask changed - ib_net16_t pad2; - ib_net32_t new_cap_mask; // new capability mask + ib_net16_t lid; // lid where change occured + uint8_t pad2; // reserved + uint8_t local_changes; // 7b reserved 1b local changes + ib_net32_t new_cap_mask; // new capability mask + ib_net16_t change_flgs; // 13b reserved 3b change flags } PACK_SUFFIX ntc_144; struct _ntc_145 { @@ -7205,6 +7207,14 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated } PACK_SUFFIX ib_mad_notice_attr_t; #include +/** + * Trap 144 masks + */ +#define TRAP_144_MASK_OTHER_LOCAL_CHANGES 0x01 +#define TRAP_144_MASK_LINK_SPEED_ENABLE_CHANGE (CL_HTON16(0x0004)) +#define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) +#define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) + /****f* IBA Base: Types/ib_notice_is_generic * NAME * ib_notice_is_generic -- 1.5.1 From murray at tradeworx.com Wed Mar 12 11:05:37 2008 From: murray at tradeworx.com (Murray Smigel) Date: Wed, 12 Mar 2008 14:05:37 -0400 Subject: [ofa-general] proper way to recover from poll CQ failed error Message-ID: <47D81B71.8090902@tradeworx.com> Hi, I am running OFED-3.0 using ConnectX adapters in a two machine direct connect mode. Most of the various pingpong tests seem ok, but when I run ibv_srq_pingpong -s 500 -n 1000 I get poll "CQ failed -2" when I start up the client side. Smaller values of -s worked fine. Once this happens, no other pingpong tests seem to work. I have then unloaded all the ib_* mlmx_* and iw_* modules, reloaded them and things still fail. I have to reboot the machines to get things back. 1) Is there a cleaner way to recover from this situation? 2) Is the initial failure an indication that something else is wrong? 3) Is the -s 1 latency I see with ibv_rc_pingpong of ~7 microseconds reasonable? Thanks, murray smigel From hrosenstock at xsigo.com Wed Mar 12 11:07:02 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 12 Mar 2008 11:07:02 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and 1.2.1 specification In-Reply-To: <20080312102333.49779c75.weiny2@llnl.gov> References: <20080312102333.49779c75.weiny2@llnl.gov> Message-ID: <1205345222.28072.61.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-12 at 10:23 -0700, Ira Weiny wrote: > While making changes to the DataDetails for trap 144 I noticed that trap 256 and 259 were wrong. > > This patch should fix them acording to both the 1.2 and 1.2.1 spec. > > IRa > > > >From 9ad1430729151fab371b98fce82e28b33c49f036 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Mon, 10 Mar 2008 13:09:45 -0700 > Subject: [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and > 1.2.1 specification > > Signed-off-by: Ira K. Weiny > --- > opensm/include/iba/ib_types.h | 12 +++++++----- > 1 files changed, 7 insertions(+), 5 deletions(-) > > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > index a026ac7..f80d0d5 100644 > --- a/opensm/include/iba/ib_types.h > +++ b/opensm/include/iba/ib_types.h > @@ -7160,13 +7160,13 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > struct _ntc_256 { // total: 54 > ib_net16_t pad1; // 2 > ib_net16_t lid; // 2 > - ib_net16_t pad2; // 2 > + ib_net16_t dr_slid; // 2 > uint8_t method; // 1 > - uint8_t pad3; // 1 > + uint8_t pad2; // 1 > ib_net16_t attr_id; // 2 > ib_net32_t attr_mod; // 4 > ib_net64_t mkey; // 8 > - uint8_t dr_slid; // 1 > + uint8_t pad3; // 1 > uint8_t dr_trunc_hop; // 1 > uint8_t dr_rtn_path[30]; // 30 > } PACK_SUFFIX ntc_256; > @@ -7189,9 +7189,11 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > ib_net16_t data_valid; // 2 > ib_net16_t lid1; // 2 > ib_net16_t lid2; // 2 > - ib_net32_t key; // 4 > + ib_net16_t key; // 4 Isn't key still 32 bits ? > uint8_t sl; // 1 > - ib_net32_t qp1; // 4 > + uint8_t qp1_msb; // 1 > + ib_net16_t qp1_lsb; // 2 > + uint8_t pad; // 1 > uint8_t qp2_msb; // 1 > ib_net16_t qp2_lsb; // 2 I think splitting up QPN like this would make use harder. -- Hal > ib_gid_t gid1; // 16 From hrosenstock at xsigo.com Wed Mar 12 11:13:10 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 12 Mar 2008 11:13:10 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <20080312102335.5eecc8d0.weiny2@llnl.gov> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> Message-ID: <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-12 at 10:23 -0700, Ira Weiny wrote: > Sasha, > > I don't know if this is how you would like to define the masks or not. > > Let me know, > Ira > > > >From c5c6b2f6bd3e889a4eb6fa462507fcbf25a433e9 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Mon, 10 Mar 2008 11:19:29 -0700 > Subject: [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 > > > Signed-off-by: Ira K. Weiny > --- > opensm/include/iba/ib_types.h | 16 +++++++++++++--- > 1 files changed, 13 insertions(+), 3 deletions(-) > > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > index 649ef1c..a026ac7 100644 > --- a/opensm/include/iba/ib_types.h > +++ b/opensm/include/iba/ib_types.h > @@ -7143,9 +7143,11 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > > struct _ntc_144 { > ib_net16_t pad1; > - ib_net16_t lid; // lid where capability mask changed > - ib_net16_t pad2; > - ib_net32_t new_cap_mask; // new capability mask > + ib_net16_t lid; // lid where change occured > + uint8_t pad2; // reserved > + uint8_t local_changes; // 7b reserved 1b local changes > + ib_net32_t new_cap_mask; // new capability mask > + ib_net16_t change_flgs; // 13b reserved 3b change flags Should this be padded out as in the 1.2.1 spec ? -- Hal > } PACK_SUFFIX ntc_144; > > struct _ntc_145 { > @@ -7205,6 +7207,14 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > } PACK_SUFFIX ib_mad_notice_attr_t; > #include > > +/** > + * Trap 144 masks > + */ > +#define TRAP_144_MASK_OTHER_LOCAL_CHANGES 0x01 > +#define TRAP_144_MASK_LINK_SPEED_ENABLE_CHANGE (CL_HTON16(0x0004)) > +#define TRAP_144_MASK_LINK_WIDTH_ENABLE_CHANGE (CL_HTON16(0x0002)) > +#define TRAP_144_MASK_NODE_DESCRIPTION_CHANGE (CL_HTON16(0x0001)) > + > /****f* IBA Base: Types/ib_notice_is_generic > * NAME > * ib_notice_is_generic From rdreier at cisco.com Wed Mar 12 11:28:45 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 11:28:45 -0700 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? Message-ID: The following is what I have queued up to ask Linus to pull soon. Are there any other patches that we want to get into 2.6.25? I know Jack sent a few patches. I'm planning on holding off on the driver version number changes until 2.6.26, since we've lived for a long time with the current info and I don't see any urgency in the change. I sent some comments about the warning message supression patches and didn't see a response, so I'm holding off on those as well. Other than that I don't know of anything that looks useful for 2.6.25, but let me know if I'm wrong. Or Gerlitz (1): IPoIB: Don't drop multicast sends when they can be queued Patrick Marchand Latifi (4): IB/ipath: Fix potentially wrong RNR retry counter returned in ipath_query_qp() IB/ipath: Fix RC QP initialization IB/ipath: Fix error completion put on send CQ instead of recv CQ IB/ipath: Reset the retry counter for RDMA_READ_RESPONSE_MIDDLE packets Ralph Campbell (1): IB/ipath: Fix IB compliance problems with link state vs physical state Roland Dreier (2): IPoIB/cm: Set tx_wr.num_sge in connected mode post_send() IPoIB: Allocate priv->tx_ring with vmalloc() From crankingtb3544 at derekware.com Wed Mar 12 11:29:40 2008 From: crankingtb3544 at derekware.com (Doreen Wimund) Date: Wed, 12 Mar 2008 10:29:40 -0800 Subject: [ofa-general] Jetzt bestellen und ein blaues Wunder erleben Message-ID: <01c8842b$fa6d8200$e979e1c9@crankingtb3544> Mussen Sie meds anonym kaufen? Hier ist wie. Viaaaagra - preis 0.8 Euro Ciaaaalis - preis 1.5 Euro Viaaaagra Professional - preis 2.4 Euro Ciaaaalis Professional - preis 3.17 Euro und viel mehr - Kostenlose, arztliche Telefon-Beratung - Kein peinlicher Arztbesuch erforderlich - Bequem und diskret online bestellen. - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - Visa verifizierter Onlineshop - Diskrete Verpackung und Zahlung - keine versteckte Kosten Nur fur kurze Zeit - vier Pillen umsonst erhalten http://ploytt.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Wed Mar 12 11:40:48 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 12 Mar 2008 11:40:48 -0700 Subject: [ofa-general] [PATCH] opensm/osm_qos_parser: fixed compilation on byacc In-Reply-To: <47D800DA.2040102@dev.mellanox.co.il> References: <47D54C17.1010802@dev.mellanox.co.il> <1205328799.27847.125.camel@hrosenstock-ws.xsigo.com> <47D800DA.2040102@dev.mellanox.co.il> Message-ID: <1205347248.28072.75.camel@hrosenstock-ws.xsigo.com> Hi Yevgeny, On Wed, 2008-03-12 at 18:12 +0200, Yevgeny Kliteynik wrote: > Hi Hal, > > Hal Rosenstock wrote: > > Hi Yevgeny, > > > > On Mon, 2008-03-10 at 16:56 +0200, Yevgeny Kliteynik wrote: > >> Hi Sasha. > >> > >> Fixing compilation with byacc (bug 932). > >> Please apply to ofed_1_3 and master. > >> > >> -- Yevgeny > >> > >> Signed-off-by: Yevgeny Kliteynik > >> --- > >> opensm/opensm/osm_qos_parser.l | 1 + > >> opensm/opensm/osm_qos_parser.y | 1 + > >> 2 files changed, 2 insertions(+), 0 deletions(-) > >> > >> diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l > >> index 41f8720..32e8ab3 100644 > >> --- a/opensm/opensm/osm_qos_parser.l > >> +++ b/opensm/opensm/osm_qos_parser.l > >> @@ -56,6 +56,7 @@ static void save_pos(); > >> extern int column_num; > >> extern int line_num; > >> extern FILE * __qos_parser_in; > >> +extern YYSTYPE __qos_parser_lval; > >> > >> boolean_t in_description = FALSE; > >> boolean_t in_list_of_hex_num_ranges = FALSE; > >> diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y > >> index 6595091..040c355 100644 > >> --- a/opensm/opensm/osm_qos_parser.y > >> +++ b/opensm/opensm/osm_qos_parser.y > >> @@ -149,6 +149,7 @@ extern char * __qos_parser_text; > >> extern int __qos_parser_lex (void); > >> extern FILE * __qos_parser_in; > >> extern int errno; > >> +int __qos_parser_parse(); > >> > >> #define RESET_BUFFER __parser_tmp_struct_reset() > > > > Are the following warnings benign: > > > > osm_qos_parser.tab.c: In function `__qos_parser_parse': > > osm_qos_parser.tab.c:1723: warning: suggest parentheses around assignment used as truth value > > osm_qos_parser.tab.c:1771: warning: label `yyerrlab' defined but not used > > osm_qos_parser.tab.c:1766: warning: label `yynewerror' defined but not used > > ../include/complib/cl_qcomppool.h: At top level: > > osm_qos_parser_y.c:2: warning: `yysccsid' defined but not used > > > > Also, have you tested regressed this with some QoS setup ? > > What version of byacc do you have? 1.9.1-1.1 > What distro? Debian 3.1 with some local fixes and updates -- Hal > I checked it on RH4U4 with bison and byacc - no warnings, and parser works fine. > > -- Yevgeny > > > Thanks. > > > > -- Hal > > > From eli at dev.mellanox.co.il Wed Mar 12 12:02:55 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Wed, 12 Mar 2008 21:02:55 +0200 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: References: Message-ID: <4e6a6b3c0803121202x3b0184f8i29067d867dfc9eeb@mail.gmail.com> On 3/12/08, Roland Dreier wrote: > The following is what I have queued up to ask Linus to pull soon. Are > there any other patches that we want to get into 2.6.25? Roland, what about these patches? Can we push them too? http://lists.openfabrics.org/pipermail/general/2008-February/047170.html http://lists.openfabrics.org/pipermail/general/2008-February/047171.html http://lists.openfabrics.org/pipermail/general/2008-February/047172.html > > I know Jack sent a few patches. I'm planning on holding off on the > driver version number changes until 2.6.26, since we've lived for a > long time with the current info and I don't see any urgency in the > change. I sent some comments about the warning message supression > patches and didn't see a response, so I'm holding off on those as > well. > > Other than that I don't know of anything that looks useful for 2.6.25, > but let me know if I'm wrong. > > Or Gerlitz (1): > IPoIB: Don't drop multicast sends when they can be queued > > Patrick Marchand Latifi (4): > IB/ipath: Fix potentially wrong RNR retry counter returned in ipath_query_qp() > IB/ipath: Fix RC QP initialization > IB/ipath: Fix error completion put on send CQ instead of recv CQ > IB/ipath: Reset the retry counter for RDMA_READ_RESPONSE_MIDDLE packets > > Ralph Campbell (1): > IB/ipath: Fix IB compliance problems with link state vs physical state > > Roland Dreier (2): > IPoIB/cm: Set tx_wr.num_sge in connected mode post_send() > IPoIB: Allocate priv->tx_ring with vmalloc() > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From rdreier at cisco.com Wed Mar 12 12:05:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 12:05:05 -0700 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: <4e6a6b3c0803121202x3b0184f8i29067d867dfc9eeb@mail.gmail.com> (Eli Cohen's message of "Wed, 12 Mar 2008 21:02:55 +0200") References: <4e6a6b3c0803121202x3b0184f8i29067d867dfc9eeb@mail.gmail.com> Message-ID: > Roland, what about these patches? Can we push them too? > http://lists.openfabrics.org/pipermail/general/2008-February/047170.html > http://lists.openfabrics.org/pipermail/general/2008-February/047171.html > http://lists.openfabrics.org/pipermail/general/2008-February/047172.html No, after -rc5 is way too late for that kind of change. I'm planning on merging them for 2.6.26 soon. - R. From hrosenstock at xsigo.com Wed Mar 12 12:16:58 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 12 Mar 2008 12:16:58 -0700 Subject: [ofa-general] [PATCH] opensm/osm_qos_parser: fixed compilation on byacc In-Reply-To: <47D800DA.2040102@dev.mellanox.co.il> References: <47D54C17.1010802@dev.mellanox.co.il> <1205328799.27847.125.camel@hrosenstock-ws.xsigo.com> <47D800DA.2040102@dev.mellanox.co.il> Message-ID: <1205349418.28072.91.camel@hrosenstock-ws.xsigo.com> Hi Yevgeny, On Wed, 2008-03-12 at 18:12 +0200, Yevgeny Kliteynik wrote: > Hi Hal, > > Hal Rosenstock wrote: > > Hi Yevgeny, > > > > On Mon, 2008-03-10 at 16:56 +0200, Yevgeny Kliteynik wrote: > >> Hi Sasha. > >> > >> Fixing compilation with byacc (bug 932). > >> Please apply to ofed_1_3 and master. > >> > >> -- Yevgeny > >> > >> Signed-off-by: Yevgeny Kliteynik > >> --- > >> opensm/opensm/osm_qos_parser.l | 1 + > >> opensm/opensm/osm_qos_parser.y | 1 + > >> 2 files changed, 2 insertions(+), 0 deletions(-) > >> > >> diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l > >> index 41f8720..32e8ab3 100644 > >> --- a/opensm/opensm/osm_qos_parser.l > >> +++ b/opensm/opensm/osm_qos_parser.l > >> @@ -56,6 +56,7 @@ static void save_pos(); > >> extern int column_num; > >> extern int line_num; > >> extern FILE * __qos_parser_in; > >> +extern YYSTYPE __qos_parser_lval; > >> > >> boolean_t in_description = FALSE; > >> boolean_t in_list_of_hex_num_ranges = FALSE; > >> diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y > >> index 6595091..040c355 100644 > >> --- a/opensm/opensm/osm_qos_parser.y > >> +++ b/opensm/opensm/osm_qos_parser.y > >> @@ -149,6 +149,7 @@ extern char * __qos_parser_text; > >> extern int __qos_parser_lex (void); > >> extern FILE * __qos_parser_in; > >> extern int errno; > >> +int __qos_parser_parse(); > >> > >> #define RESET_BUFFER __parser_tmp_struct_reset() > > > > Are the following warnings benign: > > > > osm_qos_parser.tab.c: In function `__qos_parser_parse': > > osm_qos_parser.tab.c:1723: warning: suggest parentheses around assignment used as truth value > > osm_qos_parser.tab.c:1771: warning: label `yyerrlab' defined but not used > > osm_qos_parser.tab.c:1766: warning: label `yynewerror' defined but not used > > ../include/complib/cl_qcomppool.h: At top level: > > osm_qos_parser_y.c:2: warning: `yysccsid' defined but not used > > > > Also, have you tested regressed this with some QoS setup ? > > What version of byacc do you have? What distro? > I checked it on RH4U4 with bison and byacc - no warnings, and parser works fine. What are the versions (of byacc and bison) there ? -- Hal > -- Yevgeny > > > Thanks. > > > > -- Hal > > > From cogitatesipbc602 at patentquality.com Wed Mar 12 12:33:20 2008 From: cogitatesipbc602 at patentquality.com (Monroe Swenson) Date: Wed, 12 Mar 2008 20:33:20 +0100 Subject: [ofa-general] A breakthrough in herbal Science Message-ID: <01c88480$4f3ae800$be033b54@cogitatesipbc602> A breakthrough in herbal Science has created a pill that has been designed specifically for penis enlargement. The tests that took place over a 6 month period showed that out of the 5,000 Males from around the world who participated, the average gain after 5 months of taking VPXL pills was 3.02 Inches! Amazing, PERMANENT RESULTS that will last. http://dgkgmukcgs73.blogspot.com From hrosenstock at xsigo.com Wed Mar 12 12:35:06 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 12 Mar 2008 12:35:06 -0700 Subject: [ofa-general] ibutils binaries configuration and installation questions Message-ID: <1205350506.28072.95.camel@hrosenstock-ws.xsigo.com> Hi Oren, Are the ibutils binaries meant to be run by root (like the other management utilities) ? If so, by default, they should be in sbin rather than bin. Also, is there a configure option for where the binaries go ? Thanks. -- Hal From rdreier at cisco.com Wed Mar 12 12:43:58 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 12:43:58 -0700 Subject: [ofa-general] [PATCH/RFC] IB/uverbs: Don't store struct file * for event files Message-ID: The file member of struct ib_uverbs_event_file was only used to keep track of whether the file had been closed or not. The only thing we ever did with the value was check if it was NULL or not. Simplify the code and get rid of the need to keep track of the struct file * we allocate by replacing the file member with an is_closed member. Signed-off-by: Roland Dreier --- drivers/infiniband/core/uverbs.h | 4 ++-- drivers/infiniband/core/uverbs_main.c | 9 ++++----- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h index c75eb6c..2cad8b4 100644 --- a/drivers/infiniband/core/uverbs.h +++ b/drivers/infiniband/core/uverbs.h @@ -81,13 +81,13 @@ struct ib_uverbs_device { struct ib_uverbs_event_file { struct kref ref; - struct file *file; struct ib_uverbs_file *uverbs_file; spinlock_t lock; - int is_async; wait_queue_head_t poll_wait; struct fasync_struct *async_queue; struct list_head event_list; + int is_async; + int is_closed; }; struct ib_uverbs_file { diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 7c2ac39..63335da 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -352,7 +352,7 @@ static int ib_uverbs_event_close(struct inode *inode, struct file *filp) struct ib_uverbs_event *entry, *tmp; spin_lock_irq(&file->lock); - file->file = NULL; + file->is_closed = 1; list_for_each_entry_safe(entry, tmp, &file->event_list, list) { if (entry->counter) list_del(&entry->obj_list); @@ -390,7 +390,7 @@ void ib_uverbs_comp_handler(struct ib_cq *cq, void *cq_context) return; spin_lock_irqsave(&file->lock, flags); - if (!file->file) { + if (file->is_closed) { spin_unlock_irqrestore(&file->lock, flags); return; } @@ -423,7 +423,7 @@ static void ib_uverbs_async_handler(struct ib_uverbs_file *file, unsigned long flags; spin_lock_irqsave(&file->async_file->lock, flags); - if (!file->async_file->file) { + if (!file->async_file->is_closed) { spin_unlock_irqrestore(&file->async_file->lock, flags); return; } @@ -509,6 +509,7 @@ struct file *ib_uverbs_alloc_event_file(struct ib_uverbs_file *uverbs_file, ev_file->uverbs_file = uverbs_file; ev_file->async_queue = NULL; ev_file->is_async = is_async; + ev_file->is_closed = 0; *fd = get_unused_fd(); if (*fd < 0) { @@ -522,8 +523,6 @@ struct file *ib_uverbs_alloc_event_file(struct ib_uverbs_file *uverbs_file, goto err_fd; } - ev_file->file = filp; - /* * fops_get() can't fail here, because we're coming from a * system call on a uverbs file, which will already have a -- 1.5.4.3 From rdreier at cisco.com Wed Mar 12 12:45:47 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 12:45:47 -0700 Subject: [ofa-general] [PATCH/RFC] IB/uverbs: Use alloc_file() instead of get_empty_filp() Message-ID: Christoph Hellwig wants to unexport get_empty_filp(), which is an ugly internal interface. Change the modular user in ib_uverbs_alloc_event_file() to use the better alloc_file() interface; this makes the code cleaner too. Signed-off-by: Roland Dreier --- Planning on merging this for 2.6.26... drivers/infiniband/core/uverbs_main.c | 19 +++++++------------ 1 files changed, 7 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 63335da..f49f946 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -517,23 +517,18 @@ struct file *ib_uverbs_alloc_event_file(struct ib_uverbs_file *uverbs_file, goto err; } - filp = get_empty_filp(); - if (!filp) { - ret = -ENFILE; - goto err_fd; - } - /* * fops_get() can't fail here, because we're coming from a * system call on a uverbs file, which will already have a * module reference. */ - filp->f_op = fops_get(&uverbs_event_fops); - filp->f_path.mnt = mntget(uverbs_event_mnt); - filp->f_path.dentry = dget(uverbs_event_mnt->mnt_root); - filp->f_mapping = filp->f_path.dentry->d_inode->i_mapping; - filp->f_flags = O_RDONLY; - filp->f_mode = FMODE_READ; + filp = alloc_file(uverbs_event_mnt, dget(uverbs_event_mnt->mnt_root), + FMODE_READ, fops_get(&uverbs_event_fops)); + if (!filp) { + ret = -ENFILE; + goto err_fd; + } + filp->private_data = ev_file; return filp; -- 1.5.4.3 From rdreier at cisco.com Wed Mar 12 12:51:36 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 12:51:36 -0700 Subject: [ofa-general] [PATCH/RFC] RDMA/nes: Remove redundant NULL check in nes_unregister_ofa_device() Message-ID: nes_unregister_ofa_device() dereferences the nesibdev pointer before testing if it's NULL. Also, the test is doubly redundant because the only caller of nes_unregister_ofa_device() is nes_destroy_ofa_device(), which already tests if nesibdev is NULL. Remove the unnecessary test. This was spotted by the Coverity checker (CID 2190). Signed-off-by: Roland Dreier --- Planning on merging this for 2.6.26 also... drivers/infiniband/hw/nes/nes_verbs.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c index a651e9d..90fa06e 100644 --- a/drivers/infiniband/hw/nes/nes_verbs.c +++ b/drivers/infiniband/hw/nes/nes_verbs.c @@ -3900,9 +3900,6 @@ void nes_unregister_ofa_device(struct nes_ib_device *nesibdev) struct nes_vnic *nesvnic = nesibdev->nesvnic; int i; - if (nesibdev == NULL) - return; - for (i = 0; i < ARRAY_SIZE(nes_class_attributes); ++i) { class_device_remove_file(&nesibdev->ibdev.class_dev, nes_class_attributes[i]); } -- 1.5.4.3 From weiny2 at llnl.gov Wed Mar 12 12:54:48 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 12 Mar 2008 12:54:48 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and 1.2.1 specification In-Reply-To: <1205345222.28072.61.camel@hrosenstock-ws.xsigo.com> References: <20080312102333.49779c75.weiny2@llnl.gov> <1205345222.28072.61.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080312125448.27887255.weiny2@llnl.gov> Hey Hal, On Wed, 12 Mar 2008 11:07:02 -0700 Hal Rosenstock wrote: > On Wed, 2008-03-12 at 10:23 -0700, Ira Weiny wrote: > > While making changes to the DataDetails for trap 144 I noticed that trap 256 and 259 were wrong. > > > > This patch should fix them acording to both the 1.2 and 1.2.1 spec. > > > > IRa > > > > > > >From 9ad1430729151fab371b98fce82e28b33c49f036 Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Mon, 10 Mar 2008 13:09:45 -0700 > > Subject: [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and > > 1.2.1 specification > > > > Signed-off-by: Ira K. Weiny > > --- > > opensm/include/iba/ib_types.h | 12 +++++++----- > > 1 files changed, 7 insertions(+), 5 deletions(-) > > > > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > > index a026ac7..f80d0d5 100644 > > --- a/opensm/include/iba/ib_types.h > > +++ b/opensm/include/iba/ib_types.h > > @@ -7160,13 +7160,13 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > > struct _ntc_256 { // total: 54 > > ib_net16_t pad1; // 2 > > ib_net16_t lid; // 2 > > - ib_net16_t pad2; // 2 > > + ib_net16_t dr_slid; // 2 > > uint8_t method; // 1 > > - uint8_t pad3; // 1 > > + uint8_t pad2; // 1 > > ib_net16_t attr_id; // 2 > > ib_net32_t attr_mod; // 4 > > ib_net64_t mkey; // 8 > > - uint8_t dr_slid; // 1 > > + uint8_t pad3; // 1 > > uint8_t dr_trunc_hop; // 1 > > uint8_t dr_rtn_path[30]; // 30 > > } PACK_SUFFIX ntc_256; > > @@ -7189,9 +7189,11 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > > ib_net16_t data_valid; // 2 > > ib_net16_t lid1; // 2 > > ib_net16_t lid2; // 2 > > - ib_net32_t key; // 4 > > + ib_net16_t key; // 4 > > Isn't key still 32 bits ? In 1.2.1, I see on page 825 Table 140 "Notice DataDetails for Trap 259" Field Length (bits) Description PKEY 16 P_Key In 1.2, the table is on page 817 Table 139. I assumed it was just a copy/paste error in the code? > > > uint8_t sl; // 1 > > - ib_net32_t qp1; // 4 > > + uint8_t qp1_msb; // 1 > > + ib_net16_t qp1_lsb; // 2 > > + uint8_t pad; // 1 > > uint8_t qp2_msb; // 1 > > ib_net16_t qp2_lsb; // 2 > > I think splitting up QPN like this would make use harder. > We could get rid of that. But qp2 is split so I figured there was precedence to use msb/lsb. Also I like the fact it is more explicit which bits are the qp. I thought some might use the 32bit value including the pad accidentally. It's your call, Ira From sleddedcyz26 at activemarker.com Wed Mar 12 12:56:16 2008 From: sleddedcyz26 at activemarker.com (Edmond Tidwell) Date: Thu, 13 Mar 2008 04:56:16 +0900 Subject: [ofa-general] The tests that took place over a 6 month period showed that Message-ID: <01c884c6$91871800$b273c77a@sleddedcyz26> A breakthrough in herbal Science has created a pill that has been designed specifically for penis enlargement. The tests that took place over a 6 month period showed that out of the 5,000 Males from around the world who participated, the average gain after 5 months of taking VPXL pills was 3.02 Inches! Amazing, PERMANENT RESULTS that will last. http://kopckudqsf28.blogspot.com From treasuresdmp0 at wnyconcrete.com Wed Mar 12 13:14:00 2008 From: treasuresdmp0 at wnyconcrete.com (Robin Dodd) Date: Wed, 12 Mar 2008 21:14:00 +0100 Subject: [ofa-general] the average gain after 5 months of taking VPXL pills was 3.02 Inches! Message-ID: <01c88485$fdc8a100$230d7a57@treasuresdmp0> A breakthrough in herbal Science has created a pill that has been designed specifically for penis enlargement. The tests that took place over a 6 month period showed that out of the 5,000 Males from around the world who participated, the average gain after 5 months of taking VPXL pills was 3.02 Inches! Amazing, PERMANENT RESULTS that will last. http://ogcbdcngn47.blogspot.com From xma at us.ibm.com Wed Mar 12 13:17:52 2008 From: xma at us.ibm.com (Shirley Ma) Date: Wed, 12 Mar 2008 13:17:52 -0700 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: Message-ID: Hello Roland, >The following is what I have queued up to ask Linus to pull soon. Are >there any other patches that we want to get into 2.6.25? I am planning to resubmit 4K MTU patch soon. I will target 2.6.26 instead if it's too late. Thanks Shirley -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Wed Mar 12 13:21:24 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 12 Mar 2008 13:21:24 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and 1.2.1 specification In-Reply-To: <20080312125448.27887255.weiny2@llnl.gov> References: <20080312102333.49779c75.weiny2@llnl.gov> <1205345222.28072.61.camel@hrosenstock-ws.xsigo.com> <20080312125448.27887255.weiny2@llnl.gov> Message-ID: <1205353284.28072.106.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-12 at 12:54 -0700, Ira Weiny wrote: > Hey Hal, > > On Wed, 12 Mar 2008 11:07:02 -0700 > Hal Rosenstock wrote: > > > On Wed, 2008-03-12 at 10:23 -0700, Ira Weiny wrote: > > > While making changes to the DataDetails for trap 144 I noticed that trap 256 and 259 were wrong. > > > > > > This patch should fix them acording to both the 1.2 and 1.2.1 spec. > > > > > > IRa > > > > > > > > > >From 9ad1430729151fab371b98fce82e28b33c49f036 Mon Sep 17 00:00:00 2001 > > > From: Ira K. Weiny > > > Date: Mon, 10 Mar 2008 13:09:45 -0700 > > > Subject: [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and > > > 1.2.1 specification > > > > > > Signed-off-by: Ira K. Weiny > > > --- > > > opensm/include/iba/ib_types.h | 12 +++++++----- > > > 1 files changed, 7 insertions(+), 5 deletions(-) > > > > > > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > > > index a026ac7..f80d0d5 100644 > > > --- a/opensm/include/iba/ib_types.h > > > +++ b/opensm/include/iba/ib_types.h > > > @@ -7160,13 +7160,13 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > > > struct _ntc_256 { // total: 54 > > > ib_net16_t pad1; // 2 > > > ib_net16_t lid; // 2 > > > - ib_net16_t pad2; // 2 > > > + ib_net16_t dr_slid; // 2 > > > uint8_t method; // 1 > > > - uint8_t pad3; // 1 > > > + uint8_t pad2; // 1 > > > ib_net16_t attr_id; // 2 > > > ib_net32_t attr_mod; // 4 > > > ib_net64_t mkey; // 8 > > > - uint8_t dr_slid; // 1 > > > + uint8_t pad3; // 1 > > > uint8_t dr_trunc_hop; // 1 > > > uint8_t dr_rtn_path[30]; // 30 > > > } PACK_SUFFIX ntc_256; > > > @@ -7189,9 +7189,11 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > > > ib_net16_t data_valid; // 2 > > > ib_net16_t lid1; // 2 > > > ib_net16_t lid2; // 2 > > > - ib_net32_t key; // 4 > > > + ib_net16_t key; // 4 > > > > Isn't key still 32 bits ? > > In 1.2.1, I see on page 825 Table 140 "Notice DataDetails for Trap 259" > > Field Length (bits) Description > PKEY 16 P_Key > > In 1.2, the table is on page 817 Table 139. It can be QKey also (trap 258). PKey is encapsulated in the 32 bits as indicated in the description. > I assumed it was just a copy/paste error in the code? > > > > > uint8_t sl; // 1 > > > - ib_net32_t qp1; // 4 > > > + uint8_t qp1_msb; // 1 > > > + ib_net16_t qp1_lsb; // 2 > > > + uint8_t pad; // 1 > > > uint8_t qp2_msb; // 1 > > > ib_net16_t qp2_lsb; // 2 > > > > I think splitting up QPN like this would make use harder. > > > > We could get rid of that. But qp2 is split so I figured there was precedence > to use msb/lsb. Also I like the fact it is more explicit which bits are the > qp. I thought some might use the 32bit value including the pad accidentally. Maybe there's precedence but it might be bad predecence. I think it makes it harder to do any endian conversions. It does clarify the bits but that can be done another way (e.g. comments). > It's your call, Not mine anymore :-) -- Hal > Ira > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From gefesmilezoj at esmile.jp Wed Mar 12 13:28:05 2008 From: gefesmilezoj at esmile.jp (Antonia Groves) Date: Wed, 12 Mar 2008 22:28:05 +0200 Subject: [ofa-general] Software Message-ID: <988029025.57592078216017@esmile.jp> Looking for the better valuein software rebates? This time you'll obtain the chance to have the software environment you dream of from very long ago. And the best thing is, all softs are extremely cheap. Make sure by yourself & take the softwares for lowest prices ever seen! http://ocafoatuhd28.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Wed Mar 12 13:48:22 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 12 Mar 2008 13:48:22 -0700 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: (Shirley Ma's message of "Wed, 12 Mar 2008 13:17:52 -0700") References: Message-ID: > I am planning to resubmit 4K MTU patch soon. I will target 2.6.26 instead > if it's too late. Yes, way way too late for 2.6.25, given that -rc5 is already out. In fact 2.6.26 stuff needs to appear withing the next few weeks. From xma at us.ibm.com Wed Mar 12 14:00:59 2008 From: xma at us.ibm.com (Shirley Ma) Date: Wed, 12 Mar 2008 14:00:59 -0700 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: Message-ID: > > I am planning to resubmit 4K MTU patch soon. I will target 2.6.26 instead > > if it's too late. > > Yes, way way too late for 2.6.25, given that -rc5 is already out. > In fact 2.6.26 stuff needs to appear withing the next few weeks. I have been busy on other releases and support. I will submit the 4K mtu patch asap. I am also thinking to write a RFC patch for multiple interrupt vectors in IPoIB. If 2.6.26 stuff is expecting in the next few weeks, then I might not be able to finish it. Thanks Shirley -------------- next part -------------- An HTML attachment was scrubbed... URL: From weiny2 at llnl.gov Wed Mar 12 14:10:15 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 12 Mar 2008 14:10:15 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and 1.2.1 specification In-Reply-To: <1205353284.28072.106.camel@hrosenstock-ws.xsigo.com> References: <20080312102333.49779c75.weiny2@llnl.gov> <1205345222.28072.61.camel@hrosenstock-ws.xsigo.com> <20080312125448.27887255.weiny2@llnl.gov> <1205353284.28072.106.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080312141015.2b63fd9e.weiny2@llnl.gov> On Wed, 12 Mar 2008 13:21:24 -0700 Hal Rosenstock wrote: > On Wed, 2008-03-12 at 12:54 -0700, Ira Weiny wrote: > > Hey Hal, > > > > On Wed, 12 Mar 2008 11:07:02 -0700 > > Hal Rosenstock wrote: > > > > > On Wed, 2008-03-12 at 10:23 -0700, Ira Weiny wrote: > > > > While making changes to the DataDetails for trap 144 I noticed that trap 256 and 259 were wrong. > > > > > > > > This patch should fix them acording to both the 1.2 and 1.2.1 spec. > > > > > > > > IRa > > > > > > > > > > > > >From 9ad1430729151fab371b98fce82e28b33c49f036 Mon Sep 17 00:00:00 2001 > > > > From: Ira K. Weiny > > > > Date: Mon, 10 Mar 2008 13:09:45 -0700 > > > > Subject: [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and > > > > 1.2.1 specification > > > > > > > > Signed-off-by: Ira K. Weiny > > > > --- > > > > opensm/include/iba/ib_types.h | 12 +++++++----- > > > > 1 files changed, 7 insertions(+), 5 deletions(-) > > > > > > > > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > > > > index a026ac7..f80d0d5 100644 > > > > --- a/opensm/include/iba/ib_types.h > > > > +++ b/opensm/include/iba/ib_types.h > > > > @@ -7160,13 +7160,13 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > > > > struct _ntc_256 { // total: 54 > > > > ib_net16_t pad1; // 2 > > > > ib_net16_t lid; // 2 > > > > - ib_net16_t pad2; // 2 > > > > + ib_net16_t dr_slid; // 2 > > > > uint8_t method; // 1 > > > > - uint8_t pad3; // 1 > > > > + uint8_t pad2; // 1 > > > > ib_net16_t attr_id; // 2 > > > > ib_net32_t attr_mod; // 4 > > > > ib_net64_t mkey; // 8 > > > > - uint8_t dr_slid; // 1 > > > > + uint8_t pad3; // 1 > > > > uint8_t dr_trunc_hop; // 1 > > > > uint8_t dr_rtn_path[30]; // 30 > > > > } PACK_SUFFIX ntc_256; > > > > @@ -7189,9 +7189,11 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated > > > > ib_net16_t data_valid; // 2 > > > > ib_net16_t lid1; // 2 > > > > ib_net16_t lid2; // 2 > > > > - ib_net32_t key; // 4 > > > > + ib_net16_t key; // 4 > > > > > > Isn't key still 32 bits ? > > > > In 1.2.1, I see on page 825 Table 140 "Notice DataDetails for Trap 259" > > > > Field Length (bits) Description > > PKEY 16 P_Key > > > > In 1.2, the table is on page 817 Table 139. > > It can be QKey also (trap 258). PKey is encapsulated in the 32 bits as > indicated in the description. But trap 257 and 258 have a separate structure: struct _ntc_257_258 // violation of p/q_key // 49 { ib_net16_t pad1; // 2 ib_net16_t lid1; // 2 ib_net16_t lid2; // 2 ib_net32_t key; // 2 uint8_t sl; // 1 ib_net32_t qp1; // 4 ib_net32_t qp2; // 4 ib_gid_t gid1; // 16 ib_gid_t gid2; // 16 } PACK_SUFFIX ntc_257_258; Upon closer inspection I have revised the patch to update the comment section as well. > > > I assumed it was just a copy/paste error in the code? > > > > > > > > uint8_t sl; // 1 > > > > - ib_net32_t qp1; // 4 > > > > + uint8_t qp1_msb; // 1 > > > > + ib_net16_t qp1_lsb; // 2 > > > > + uint8_t pad; // 1 > > > > uint8_t qp2_msb; // 1 > > > > ib_net16_t qp2_lsb; // 2 > > > > > > I think splitting up QPN like this would make use harder. > > > > > > > We could get rid of that. But qp2 is split so I figured there was precedence > > to use msb/lsb. Also I like the fact it is more explicit which bits are the > > qp. I thought some might use the 32bit value including the pad accidentally. > > Maybe there's precedence but it might be bad predecence. I think it > makes it harder to do any endian conversions. It does clarify the bits > but that can be done another way (e.g. comments). > > > It's your call, > > Not mine anymore :-) > Ok, I have changed this in the new patch as well. Tell me what you think. Ira >From 0476b0d102ada643db9abf6271aae8a540a417d7 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Mon, 10 Mar 2008 13:09:45 -0700 Subject: [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and 1.2.1 specification Signed-off-by: Ira K. Weiny --- opensm/include/iba/ib_types.h | 22 +++++++++++++--------- 1 files changed, 13 insertions(+), 9 deletions(-) diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 649ef1c..e160200 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -7158,13 +7158,13 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated struct _ntc_256 { // total: 54 ib_net16_t pad1; // 2 ib_net16_t lid; // 2 - ib_net16_t pad2; // 2 + ib_net16_t dr_slid; // 2 uint8_t method; // 1 - uint8_t pad3; // 1 + uint8_t pad2; // 1 ib_net16_t attr_id; // 2 ib_net32_t attr_mod; // 4 ib_net64_t mkey; // 8 - uint8_t dr_slid; // 1 + uint8_t pad3; // 1 uint8_t dr_trunc_hop; // 1 uint8_t dr_rtn_path[30]; // 30 } PACK_SUFFIX ntc_256; @@ -7182,16 +7182,14 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated ib_gid_t gid2; // 16 } PACK_SUFFIX ntc_257_258; - struct _ntc_259 // p/q_key violation with sw info 53 + struct _ntc_259 // pkey violation from switch 51 { ib_net16_t data_valid; // 2 ib_net16_t lid1; // 2 ib_net16_t lid2; // 2 - ib_net32_t key; // 4 - uint8_t sl; // 1 - ib_net32_t qp1; // 4 - uint8_t qp2_msb; // 1 - ib_net16_t qp2_lsb; // 2 + ib_net16_t pkey; // 2 + ib_net32_t sl_qp1; // 4b sl, 4b pad, 24b qp1 + ib_net32_t qp2; // 8b pad, 24b qp2 ib_gid_t gid1; // 16 ib_gid_t gid2; // 16 ib_net16_t sw_lid; // 2 @@ -7205,6 +7203,12 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated } PACK_SUFFIX ib_mad_notice_attr_t; #include +/** + * Trap 259 masks + */ +#define TRAP_259_MASK_SL (CL_HTON32(0xF0000000)) +#define TRAP_259_MASK_QP (CL_HTON32(0x00FFFFFF)) + /****f* IBA Base: Types/ib_notice_is_generic * NAME * ib_notice_is_generic -- 1.5.1 From vuhuong at mellanox.com Wed Mar 12 15:11:40 2008 From: vuhuong at mellanox.com (Vu Pham) Date: Wed, 12 Mar 2008 15:11:40 -0700 Subject: [ofa-general] SRP target sporadic behaviour with Solaris, VMware In-Reply-To: <47D70FC3.2030302@pocock.com.au> References: <47D70FC3.2030302@pocock.com.au> Message-ID: <47D8551C.6080602@mellanox.com> > > I've recently set up the SRP target module on Linux (2.6.22). > How did you set up your target (scsi pass-thru, blockio, fileio, nullio)? > > On VMware: > - I had to reboot my new VMware ESX server a few times before it found > my 500GB target. What is your fabric configuration (switch, which SM, which HCA, fw version)? Do you see any *weird* srp message in vmware /var/log/vmkernel? > - VMWare completely rejects a target if it doesn't have a partition > table - I ran parted on Linux and then VMWare was OK It does not make sense. Once again I need to see how you setup and expose your luns on srp target. > - Also, the messages in VMWare gave me the impression it would clobber > the whole volume, rather than just a single partition - so to avoid the > possibility of losing my other partitions, I made a special target > representing the intended partition rather than the entire volume. Now > I have a VMware partition table nested within a partition. > - VMware only seems to show one target at a time - I had created a few > test targets, but I could only see one of them. Is this what other > people see? ibsrpdm on the other Linux hosts shows all the targets. > > Any suggestions? > If you have several srp targets running blockio or fileio modes and export the same block device names (ex: all srp targets export /dev/cciss/c1d0 device ie. echo "open vdisk0 /dev/cciss/c1d0 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk) then they end up to have same lun UID. Vmware will *correctly* see single lun with multipaths to it instead of multiple luns of multiple targets. To correct this you need to load scst_vdisk with different scsi_vdisk_ID for different srp targets exporting the same device names in the fabric srp target 1: modprobe scsi_vdisk scst_vdisk_ID=1 srp target 2: modprobe scsi_vdisk scst_vdisk_ID=2 If you already create datastor on this lun you need to destroy it; otherwise, vmware won't see the datastor (because of different scst_vdisk_ID will generate different lun UID and vmware's datastor relies mainly on lun UID) -vu From vuhuong at mellanox.com Wed Mar 12 15:18:00 2008 From: vuhuong at mellanox.com (Vu Pham) Date: Wed, 12 Mar 2008 15:18:00 -0700 Subject: [ofa-general] SRP target sporadic behaviour with Solaris, VMware In-Reply-To: References: <47D70FC3.2030302@pocock.com.au> Message-ID: <47D85698.3030500@mellanox.com> Bart Van Assche wrote: > On Wed, Mar 12, 2008 at 12:03 AM, Daniel Pocock wrote: >> I've recently set up the SRP target module on Linux (2.6.22). >> >> Trying to access the target from various initiators (Fedora, Debian, >> Solaris 10, VMWare ESX 3.5) gives mixed results. >> >> The Linux clients, despite having limited configuration tools, worked >> immediately. >> >> I've opened a thread on the Sun forums to discuss the Solaris 10 issue: >> >> http://forum.java.sun.com/thread.jspa?threadID=5273631 >> >> On VMware: >> - I had to reboot my new VMware ESX server a few times before it found >> my 500GB target. >> - VMWare completely rejects a target if it doesn't have a partition >> table - I ran parted on Linux and then VMWare was OK >> - Also, the messages in VMWare gave me the impression it would clobber >> the whole volume, rather than just a single partition - so to avoid the >> possibility of losing my other partitions, I made a special target >> representing the intended partition rather than the entire volume. Now >> I have a VMware partition table nested within a partition. >> - VMware only seems to show one target at a time - I had created a few >> test targets, but I could only see one of them. Is this what other >> people see? ibsrpdm on the other Linux hosts shows all the targets. > > My experience with SRP is as follows (with Linux 2.6.24 + SCST + SRPT > as target): > * Linux SRP initiator: works perfectly. > * OpenSolaris SRP initiator: I could not get Sun's SRP initiator > working on OpenSolaris. I even asked a Solaris expert to help me, but > he couldn't get the SRP initiator working either. > * VMware ESX 3.5 + Mellanox InfiniBand drivers (released in January > 2008): until now I only have tested a setup with a single target. When > doing a lot of I/O over the SRP connection, after about 10 minutes the > virtual machine running on VMware starts logging communication errors. > I reported this yesterday to Mellanox support, and Mellanox is > currently working on this issue. Note: I had to upgrade the InfiniBand > switch firmware before the ESX server was able to find the SRP target. > Which mode of virtual disk did you use (rmd, rdmp, vmfs, raw)? Could you provide Mellanox FAE both VM /var/log/messages and esx's /var/log/vmkernel? I'll look over them -vu From vuhuong at mellanox.com Wed Mar 12 15:20:40 2008 From: vuhuong at mellanox.com (Vu Pham) Date: Wed, 12 Mar 2008 15:20:40 -0700 Subject: [ofa-general] SRP target/LVM HA configuration In-Reply-To: References: Message-ID: <47D85738.6060507@mellanox.com> I would second to Stanley - client hosts fail-over/load-balancing would be straightforward For linux host you have several tools: sw raid, lvm or dm-multipath -vu > I had looked at this configuration as well and decided to use the volume > management at the clients to mirror the data. Windows LDM mirrored > across 2 SRPT servers and Linux md RAID 1 mirrored. > > This provides transparent failover and the SRP client/host will rebuild > the slices that went offline. > >> -----Original Message----- >> From: general-bounces at lists.openfabrics.org >> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of >> Daniel Pocock >> Sent: Tuesday, March 11, 2008 4:26 PM >> To: general at lists.openfabrics.org >> Subject: [ofa-general] SRP target/LVM HA configuration >> >> >> >> >> >> >> I'm contemplating a HA configuration based on SRP and LVM (or >> maybe EVMS). >> >> There are many good resources based on NFS and drbd (see >> http://www.linux-ha.org/HaNFS) but it would be more flexible to work >> with block level (e.g SRP) rather than file level (NFS). Obviously, >> SRP/RDMA offers a major performance benefit compared with drbd (which >> uses IP). >> >> Basically, I envisage the primary server having access to the >> secondary >> (passive) server's disk using SRP, and putting both the local >> (primary) >> disk and SRP (secondary) disk into RAID1. The RAID1 set >> would contain a >> volume group and multiple volumes - which would, in turn, be >> SRP targets >> (for VMware to use) or possibly NFS shares. >> >> This leads me to a few issues: >> >> - Read operations - would it be better for the primary to >> read from both >> disks, or just it's own disk? Using drbd, the secondary disk is not >> read unless the primary is down. However, given the >> performance of SRP, >> I suspect that reading from both the local and SRP disk would give a >> boost to performance. >> >> - Does it make sense to use md or LVM to combine a local disk >> and an SRP >> disk into RAID1 (or potentially RAID5)? Are there technical >> challenges >> there, given that one target is slightly faster than the other? >> >> - Fail-over - when the secondary detects that the primary is >> down, can >> it dynamically take the place of the failed SRP target? Will the >> end-user initiators (e.g. VMWare, see diagram below) be confused when >> the changeover occurs? Is there the possibility of data >> inconsistency >> if some write operations had been acknowledged by the primary but not >> propagated to the secondary's disk at the moment when the >> failure occurred? >> >> - Recovery - when the old primary comes back online as a >> secondary, it >> will need to resync it's disk - is a partial resync possible, >> or is full >> rebuild mandatory? >> >> >> Diagram: >> >> >> Disk--Primary Server-------------------SRP Initiator (e.g. VMware ESX) >> | +------NFS client >> | . >> SRP . >> (RAID1 of primary's . >> disk and secondary's . >> disk) . (fail-over path to storage >> | . when primary is down) >> Disk--Secondary Server. . . . . . >> >> >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From daniel at pocock.com.au Wed Mar 12 15:27:06 2008 From: daniel at pocock.com.au (Daniel Pocock) Date: Wed, 12 Mar 2008 22:27:06 +0000 Subject: [ofa-general] SRP target/LVM HA configuration In-Reply-To: <47D85738.6060507@mellanox.com> References: <47D85738.6060507@mellanox.com> Message-ID: <47D858BA.1040504@pocock.com.au> Vu Pham wrote: > I would second to Stanley - client hosts fail-over/load-balancing > would be straightforward > > For linux host you have several tools: sw raid, lvm or dm-multipath Thanks for this feedback - I can do that with some clients. Will VMware allow me to mirror two SRP targets? From chu11 at llnl.gov Wed Mar 12 15:37:23 2008 From: chu11 at llnl.gov (Al Chu) Date: Wed, 12 Mar 2008 15:37:23 -0700 Subject: [ofa-general] [PATCH] Opensm: switchbalance console option In-Reply-To: <1205341036.17718.87.camel@cardanus.llnl.gov> References: <1205341036.17718.87.camel@cardanus.llnl.gov> Message-ID: <1205361443.17718.121.camel@cardanus.llnl.gov> Hey Sasha, Forgot to run this through osm_indent. Here's the cleaned up patch. Al On Wed, 2008-03-12 at 09:57 -0700, Al Chu wrote: > Hey Sasha, > > Here's a patch that does the previously discussed switchbalance console > option. Algorithmically it does pretty much the exact same thing as the > check_lft_balance script, but everything is faster of course b/c opensm > already knows everything. > > Al > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-add-switchbalance-command-to-console.patch Type: text/x-patch Size: 5753 bytes Desc: not available URL: From vuhuong at mellanox.com Wed Mar 12 16:05:55 2008 From: vuhuong at mellanox.com (Vu Pham) Date: Wed, 12 Mar 2008 16:05:55 -0700 Subject: [ofa-general] SRP target/LVM HA configuration In-Reply-To: <47D858BA.1040504@pocock.com.au> References: <47D85738.6060507@mellanox.com> <47D858BA.1040504@pocock.com.au> Message-ID: <47D861D3.8020403@mellanox.com> Daniel Pocock wrote: > > > Vu Pham wrote: >> I would second to Stanley - client hosts fail-over/load-balancing >> would be straightforward >> >> For linux host you have several tools: sw raid, lvm or dm-multipath > > Thanks for this feedback - I can do that with some clients. Will VMware > allow me to mirror two SRP targets? Tough question ;-) As far as I know Vmware does not have mirroring virtual disks, striping VM data across physical disks From victimss9 at timetreasures.com Wed Mar 12 19:08:01 2008 From: victimss9 at timetreasures.com (Margarito Melvin) Date: Wed, 12 Mar 2008 21:08:01 -0500 Subject: [ofa-general] Office 2007 Enterprise OEM version Message-ID: <01c88485$279a4e80$5675e9c9@victimss9> Microsoft Office Enterprise 2007 includes: • Access 2007 • Communicator 2007 • Excel 2007 • Groove 2007 • InfoPath 2007 • OneNote 2007 • Outlook 2007 • PowerPoint 2007 • Publisher 2007 • Word 2007 http://sydneybennett050964.blogspot.com System Requirements • Intel® Pentium® or AMD® 500 MHz processor • Microsoft Windows® XP Professional or Home Edition with Service Pack 2, Windows Server® 2003 with SP1 , Microsoft Windows Vista. • 256 Mb of RAM • 2GB of available hard-disk space. • 1024x768 or higher resolution monitor • Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From transter at gmail.com Wed Mar 12 23:31:11 2008 From: transter at gmail.com (Lan Tran) Date: Wed, 12 Mar 2008 23:31:11 -0700 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value Message-ID: Hello, I was wondering if anyone has seen a similar issue or have any ideas on why this issue may have shown up? Thanks in advance for any potential pointers! I have a Master SM and Standby SM running on two separate nodes, both connected to an 8-port Flextronic IB switch. Initially, both SM's came up fine (and negotiated mastership OK). But then the Standby SM node was rebooted; when the Standby SM came up again, it was outputing the following error in OpenSM logs: Mar 12 17:05:57 837338 [AAAB71A0] -> OpenSM Rev:openib-3.0.13 Mar 12 17:05:57 837371 [AAAB71A0] -> OpenSM Rev:openib-3.0.13 Mar 12 17:05:57 838467 [AAAB71A0] -> osm_vendor_bind: Binding to port 0x50450134010002 Mar 12 17:05:57 841495 [AAAB71A0] -> osm_vendor_bind: Binding to port 0x50450134010002 Mar 12 17:05:57 843032 [43204940] -> osm_si_rcv_process: ERR 3610: Bad LinearFDBTop value = 0xC000 on switch 0xb8cffff004879 Forcing correction to 0x0 Mar 12 17:05:58 049623 [41401940] -> Entering STANDBY state Restarting the Standby SM doesn't help and neither does resetting the IB switch. In this state, no LIDS were getting assigned. It seems functional SM behaviour (and valid Linear forwarding table) was only recovered by killing the Master SM, which caused SM migration. It's not easily reproducible unfortunately. The SM code seems to indicate this is a bug workaround, but not sure if this means that it's HW switch issue, or how it would have occurred in the first place? /* Hack for bad value in Mellanox switch */ if( cl_ntoh16( p_si->lin_top ) > IB_LID_UCAST_END_HO ) { osm_log( p_rcv->p_log, OSM_LOG_ERROR, "osm_si_rcv_process: ERR 3610: " "\n\t\t\t\tBad LinearFDBTop value = 0x%X " "on switch 0x%" PRIx64 "\n\t\t\t\tForcing correction to 0x%X\n", cl_ntoh16( p_si->lin_top ), cl_ntoh64( osm_node_get_node_guid( p_node ) ), 0 ); p_si->lin_top = 0; } Thanks again! Lan From jackm at dev.mellanox.co.il Wed Mar 12 23:30:02 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 13 Mar 2008 08:30:02 +0200 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: References: Message-ID: <200803130830.03422.jackm@dev.mellanox.co.il> On Wednesday 12 March 2008 20:28, Roland Dreier wrote: > > I know Jack sent a few patches. I'm planning on holding off on the > driver version number changes until 2.6.26, since we've lived for a > long time with the current info and I don't see any urgency in the > change. I sent some comments about the warning message supression > patches and didn't see a response, so I'm holding off on those as > well. > I just reviewed things. I submitted a 5-patch series in October. Two were for removing the extra CQE (which we needed to revert -- Ouch!). However, the other 3 were for passing device max values from kernel to userspace, and for limiting max allowable wqes per QP to avoid create-qp failures resulting from need for headroom: http://lists.openfabrics.org/pipermail/general/2007-October/042351.html http://lists.openfabrics.org/pipermail/general/2007-October/042352.html http://lists.openfabrics.org/pipermail/general/2007-October/042355.html These patches are in OFED 1.3. If you wish, I can rework them for your current tree. - Jack From jackm at dev.mellanox.co.il Wed Mar 12 23:38:07 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 13 Mar 2008 08:38:07 +0200 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: References: Message-ID: <200803130838.07544.jackm@dev.mellanox.co.il> On Wednesday 12 March 2008 20:28, Roland Dreier wrote: > I know Jack sent a few patches.  I'm planning on holding off on the > driver version number changes until 2.6.26, since we've lived for a > long time with the current info and I don't see any urgency in the > change. > I sent some comments about the warning message supression > patches and didn't see a response, so I'm holding off on those as > well. We're still reviewing and analyzing the message suppression issue, so please hold off on that until we respond (your point was well taken). Also, what about the XRC patches? - Jack From ewdafwa at bldn.com Wed Mar 12 23:48:06 2008 From: ewdafwa at bldn.com (Albert Soto) Date: Thu, 13 Mar 2008 14:48:06 +0800 Subject: [ofa-general] Want to be a hero in bed? Message-ID: <01c88519$3f231700$a061d53c@ewdafwa> Are U Tired with erectile dysfunction? Enhance your sexual life now! Want to be ready for sex in few minutes? Reproductive and ED problems solution http://geocities.com/basilvance66 We are verified by VISA. Confidential purchase. From sashak at voltaire.com Thu Mar 13 00:06:17 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 13 Mar 2008 07:06:17 +0000 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: References: Message-ID: <20080313070617.GU10479@sashak.voltaire.com> Hi, On 23:31 Wed 12 Mar , Lan Tran wrote: > > I was wondering if anyone has seen a similar issue or have any ideas > on why this issue may have shown up? I'm not familiar with this issue, but according to OpenSM code it looks like switch bug workaround. Don't see how it should affect the normal workflow. > I have a Master SM and Standby SM running on two separate nodes, both > connected to an 8-port Flextronic IB switch. Initially, both SM's came > up fine (and negotiated mastership OK). But then the Standby SM node > was rebooted; when the Standby SM came up again, it was outputing the > following error in OpenSM logs: > > Mar 12 17:05:57 837338 [AAAB71A0] -> OpenSM Rev:openib-3.0.13 > Mar 12 17:05:57 837371 [AAAB71A0] -> OpenSM Rev:openib-3.0.13 > Mar 12 17:05:57 838467 [AAAB71A0] -> osm_vendor_bind: Binding to port > 0x50450134010002 > Mar 12 17:05:57 841495 [AAAB71A0] -> osm_vendor_bind: Binding to port > 0x50450134010002 > Mar 12 17:05:57 843032 [43204940] -> osm_si_rcv_process: ERR 3610: > Bad LinearFDBTop value = 0xC000 on > switch 0xb8cffff004879 > Forcing correction to 0x0 > Mar 12 17:05:58 049623 [41401940] -> Entering STANDBY state This shows that switch reports bad value of SwitchInfo.LinearFDBTop (you can re-verify with 'smpquery switchinfo'). > Restarting the Standby SM doesn't help and neither does resetting the > IB switch. In this state, no LIDS were getting assigned. Standby SM should not do this. LIDs are assigned by master SM only. Run 'ibnetdiscover' or 'smpquery portinfo' on the local port to be sure that node gets LIDs after reboot. Sasha From bart.vanassche at gmail.com Thu Mar 13 00:10:13 2008 From: bart.vanassche at gmail.com (Bart Van Assche) Date: Thu, 13 Mar 2008 08:10:13 +0100 Subject: [ofa-general] SRP target sporadic behaviour with Solaris, VMware In-Reply-To: <47D85698.3030500@mellanox.com> References: <47D70FC3.2030302@pocock.com.au> <47D85698.3030500@mellanox.com> Message-ID: On Wed, Mar 12, 2008 at 11:18 PM, Vu Pham wrote: > Which mode of virtual disk did you use (rmd, rdmp, vmfs, raw)? It was a raw disk. > Could you provide Mellanox FAE both VM /var/log/messages and > esx's /var/log/vmkernel? > > I'll look over them Thanks. I'll send you the logs in a separate e-mail. Bart. From dotanb at dev.mellanox.co.il Thu Mar 13 01:05:28 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 13 Mar 2008 10:05:28 +0200 Subject: [ofa-general] proper way to recover from poll CQ failed error In-Reply-To: <47D81B71.8090902@tradeworx.com> References: <47D81B71.8090902@tradeworx.com> Message-ID: <47D8E048.7050600@dev.mellanox.co.il> Hi. The fact that ibv_poll_cq failed indicates that something bad happened. Usually this failure should create any problem and only the process that had the problem is being effected from this. I personally think that the ib_* performance tools are better to check the performance of your subnet. I will be happy if you'll answer the following questions: Is this error is consistent? Can you please send me the output of the ibv_devinfo of your machines? Did you have any error message in the /var/log/messages when you saw this error? thanks Dotan Murray Smigel wrote: > Hi, > I am running OFED-3.0 using ConnectX adapters in a two machine direct > connect mode. > Most of the various pingpong tests seem ok, but when I run > ibv_srq_pingpong -s 500 -n 1000 > > I get poll "CQ failed -2" when I start up the client side. Smaller > values of -s worked fine. > Once this happens, no other pingpong tests seem to work. > I have then unloaded all the ib_* mlmx_* and iw_* modules, reloaded > them and things still > fail. I have to reboot the machines to get things back. > > 1) Is there a cleaner way to recover from this situation? > 2) Is the initial failure an indication that something else is wrong? > 3) Is the -s 1 latency I see with ibv_rc_pingpong of ~7 microseconds > reasonable? > > Thanks, > murray smigel > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From ogerlitz at voltaire.com Thu Mar 13 02:20:46 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 13 Mar 2008 11:20:46 +0200 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: <200803130830.03422.jackm@dev.mellanox.co.il> References: <200803130830.03422.jackm@dev.mellanox.co.il> Message-ID: <47D8F1EE.9020406@voltaire.com> Jack Morgenstein wrote: > I just reviewed things. I submitted a 5-patch series in October. Two were > for removing the extra CQE (which we needed to revert -- Ouch!). > > However, the other 3 were for passing device max values from kernel > to userspace, and for limiting max allowable wqes per QP to avoid > create-qp failures resulting from need for headroom: > > http://lists.openfabrics.org/pipermail/general/2007-October/042351.html > http://lists.openfabrics.org/pipermail/general/2007-October/042352.html > http://lists.openfabrics.org/pipermail/general/2007-October/042355.html > > These patches are in OFED 1.3. > Hi Jack, I have now looked on the patches, and I wonder: can't they be implemented at the core layer such that they apply to all HW drivers? Or. From lareuinstitutecyg at euinstitute.org Thu Mar 13 02:40:58 2008 From: lareuinstitutecyg at euinstitute.org (Eliseo Young) Date: Thu, 13 Mar 2008 10:40:58 +0100 Subject: [ofa-general] Software Message-ID: <497590816.44777114637186@euinstitute.org> Searching for the best value in discount software? This time you'll obtain the opportunity to have the software environment you want from very long ago. And the better thing for you is, all softwares are of utmost low cost. Examine by yourself and get the softwares for lowest prices ever seen! http://oqekxtunxu32.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Thu Mar 13 03:00:45 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 13 Mar 2008 12:00:45 +0200 Subject: [ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: <1205329275.26105.28.camel@mtls03> References: <1205329275.26105.28.camel@mtls03> Message-ID: <47D8FB4D.1000405@voltaire.com> Eli Cohen wrote: > I noticed that too and I am not sure what the solution should be. From > one side, for CM mode we clear NETIF_F_SG but then we use an MTU of > 65520 bytes and it might not be so trivial for the networking stack to > provide SKB with a linear data so large. I am not following you. With the connected mode not advertising SG support to the stack and exposing 64K MTU, the stack is forced to provide upto 64K sized linear SKBs, this is how it goes from the CM's day one, no? Or. From ogerlitz at voltaire.com Thu Mar 13 03:04:33 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 13 Mar 2008 12:04:33 +0200 Subject: [ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: References: <1205329275.26105.28.camel@mtls03> Message-ID: <47D8FC31.9050105@voltaire.com> Roland Dreier wrote: > We could do this for the main (datagram mode) tx_ring, but then we eat > an extra pointer indirection for every send. I suspect that's worse > than the TLB miss we take by using vmalloc(). > Roland, Just curious why you think extra pointer indirection for each send is worse than TLB misses taken by using vmalloc(), are the latter being excepted very un often? Also assuming keeping the dma addresses at the TX descriptor is a practice taken also by Ethernet drivers who do have big send queues, maybe we should look there to see how this problem was solved? Or. From eli at dev.mellanox.co.il Thu Mar 13 03:07:57 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 13 Mar 2008 12:07:57 +0200 Subject: [ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: <47D8FB4D.1000405@voltaire.com> References: <1205329275.26105.28.camel@mtls03> <47D8FB4D.1000405@voltaire.com> Message-ID: <1205402877.25950.6.camel@mtls03> On Thu, 2008-03-13 at 12:00 +0200, Or Gerlitz wrote: > I am not following you. With the connected mode not advertising SG > support to the stack and exposing 64K MTU, the stack is forced to > provide upto 64K sized linear SKBs, this is how it goes from the CM's > day one, no? > Correct. I was just thinking ahead if we ever want to advertise NETIF_F_SG for CM to make it easier for the networking stack to allocated such buffers. From eli at dev.mellanox.co.il Thu Mar 13 03:15:38 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 13 Mar 2008 12:15:38 +0200 Subject: [ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: <47D8FC31.9050105@voltaire.com> References: <1205329275.26105.28.camel@mtls03> <47D8FC31.9050105@voltaire.com> Message-ID: <1205403338.25950.14.camel@mtls03> On Thu, 2008-03-13 at 12:04 +0200, Or Gerlitz wrote: > Roland, > > Just curious why you think extra pointer indirection for each send is > worse than TLB misses taken by using vmalloc(), are the latter being > excepted very un often? Also assuming keeping the dma addresses at the > TX descriptor is a practice taken also by Ethernet drivers who do have > big send queues, maybe we should look there to see how this problem was > solved? > > Or. > I think that a TLB miss will happen only once for each process while a cache miss on the pointer might happen more often. The question is whether it is significant or not. I think the biggest drawback of vmalloc is the scarce space in 32 bit systems - other than that it is preferred over kmalloc for each entry. From tziporet at dev.mellanox.co.il Thu Mar 13 03:17:45 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 13 Mar 2008 12:17:45 +0200 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: <200803130838.07544.jackm@dev.mellanox.co.il> References: <200803130838.07544.jackm@dev.mellanox.co.il> Message-ID: <47D8FF49.5030209@mellanox.co.il> Jack Morgenstein wrote: > > Also, what about the XRC patches? > > Roland, Please note that XRC is being defined now in IBTA. You can contact Cisco representative in IBTA to get the initial proposal. I hope that now when XRC become a standard you will be able to include it in kernel 2.6.26 Tziporet From hrosenstock at xsigo.com Thu Mar 13 04:35:39 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 13 Mar 2008 04:35:39 -0700 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: <20080313070617.GU10479@sashak.voltaire.com> References: <20080313070617.GU10479@sashak.voltaire.com> Message-ID: <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-13 at 07:06 +0000, Sasha Khapyorsky wrote: > > Mar 12 17:05:57 843032 [43204940] -> osm_si_rcv_process: ERR 3610: > > Bad LinearFDBTop value = 0xC000 on > > switch 0xb8cffff004879 > > Forcing correction to 0x0 > > Mar 12 17:05:58 049623 [41401940] -> Entering STANDBY state > > This shows that switch reports bad value of SwitchInfo.LinearFDBTop Yes, but standby shouldn't program subnet, so correction shouldn't be done by standby. -- Hal From murray at tradeworx.com Thu Mar 13 05:28:36 2008 From: murray at tradeworx.com (Murray Smigel) Date: Thu, 13 Mar 2008 08:28:36 -0400 Subject: [ofa-general] proper way to recover from poll CQ failed error In-Reply-To: <47D8E048.7050600@dev.mellanox.co.il> References: <47D81B71.8090902@tradeworx.com> <47D8E048.7050600@dev.mellanox.co.il> Message-ID: <47D91DF4.9030402@tradeworx.com> Dotan Barak wrote: > Hi. > > The fact that ibv_poll_cq failed indicates that something bad happened. > Usually this failure should create any problem and only the process > that had the problem is being > effected from this. > > I personally think that the ib_* performance tools are better to check > the performance of your subnet. ** Which programs are these? > > I will be happy if you'll answer the following questions: > Is this error is consistent? ** Yes. From this morning... murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 1 -n 1000 ... 2000 bytes in 0.00 seconds = 3.56 Mbit/sec 1000 iters in 0.00 seconds = 4.49 usec/iter murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 100 -n 1000 ... 200000 bytes in 0.00 seconds = 357.46 Mbit/sec 1000 iters in 0.00 seconds = 4.48 usec/iter murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 200 -n 1000 ... 400000 bytes in 0.00 seconds = 711.74 Mbit/sec 1000 iters in 0.00 seconds = 4.50 usec/iter murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 500 -n 1000 ... 1000000 bytes in 0.00 seconds = 1752.08 Mbit/sec 1000 iters in 0.00 seconds = 4.57 usec/iter murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 1000 -n 1000 ... 2000000 bytes in 0.01 seconds = 2973.98 Mbit/sec 1000 iters in 0.01 seconds = 5.38 usec/iter murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 2000 -n 1000 ... 4000000 bytes in 0.01 seconds = 5226.20 Mbit/sec 1000 iters in 0.01 seconds = 6.12 usec/iter murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 5000 -n 1000 ... poll CQ failed -2 Now, this used to work: murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 1 -n 1000 ... poll CQ failed -2 As did this: murray at nasnu2:/usr/local/bin$ ./ibv_rc_pingpong -s 1 -n 1000 local address: LID 0x0003, QPN 0xa9004a, PSN 0x540960 remote address: LID 0x0004, QPN 0xaf004a, PSN 0x787c46 poll CQ failed -2 > Can you please send me the output of the ibv_devinfo of your machines? murray at nasnu2:/usr/local/bin$ ./ibv_devinfo hca_id: mlx4_0 fw_ver: 2.3.000 node_guid: 0002:c903:0000:c51c sys_image_guid: 0002:c903:0000:c51f vendor_id: 0x02c9 vendor_part_id: 25418 hw_ver: 0xA0 board_id: MT_04A0120002 phys_port_cnt: 2 port: 1 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 3 port_lid: 3 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 0 port_lid: 0 port_lmc: 0x00 murray at nasnu3:~$ cd /usr/local/bin murray at nasnu3:/usr/local/bin$ ./ibv_devinfo hca_id: mlx4_0 fw_ver: 2.3.000 node_guid: 0002:c903:0000:c474 sys_image_guid: 0002:c903:0000:c477 vendor_id: 0x02c9 vendor_part_id: 25418 hw_ver: 0xA0 board_id: MT_04A0120002 phys_port_cnt: 2 port: 1 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 3 port_lid: 4 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 0 port_lid: 0 port_lmc: 0x00 > Did you have any error message in the /var/log/messages when you saw > this error? nasnu2:/usr/local/bin# tail /var/log/messages Mar 13 05:30:05 nasnu2 -- MARK -- Mar 13 05:50:05 nasnu2 -- MARK -- Mar 13 06:10:05 nasnu2 -- MARK -- Mar 13 06:30:05 nasnu2 -- MARK -- Mar 13 06:50:05 nasnu2 -- MARK -- Mar 13 07:10:05 nasnu2 -- MARK -- Mar 13 07:30:05 nasnu2 -- MARK -- Mar 13 07:36:09 nasnu2 syslogd 1.4.1#18: restart. Mar 13 07:50:05 nasnu2 -- MARK -- Mar 13 08:10:05 nasnu2 -- MARK -- nasnu2:/usr/local/bin# tail /var/log/opensm.log Mar 13 08:16:46 024587 [43806960] 0x02 -> SUBNET UP Mar 13 08:16:56 024615 [43806960] 0x02 -> SUBNET UP Mar 13 08:17:06 024596 [43806960] 0x02 -> SUBNET UP Mar 13 08:17:16 024549 [43806960] 0x02 -> SUBNET UP Mar 13 08:17:26 024664 [43806960] 0x02 -> SUBNET UP Mar 13 08:17:36 024626 [43806960] 0x02 -> SUBNET UP Mar 13 08:17:46 024627 [43806960] 0x02 -> SUBNET UP Mar 13 08:17:56 024611 [43806960] 0x02 -> SUBNET UP Mar 13 08:18:06 024607 [43806960] 0x02 -> SUBNET UP Thanks for your help, murray smigel > > thanks > Dotan > > Murray Smigel wrote: > >> Hi, >> I am running OFED-3.0 using ConnectX adapters in a two machine direct >> connect mode. >> Most of the various pingpong tests seem ok, but when I run >> ibv_srq_pingpong -s 500 -n 1000 >> >> I get poll "CQ failed -2" when I start up the client side. Smaller >> values of -s worked fine. >> Once this happens, no other pingpong tests seem to work. >> I have then unloaded all the ib_* mlmx_* and iw_* modules, reloaded >> them and things still >> fail. I have to reboot the machines to get things back. >> >> 1) Is there a cleaner way to recover from this situation? >> 2) Is the initial failure an indication that something else is wrong? >> 3) Is the -s 1 latency I see with ibv_rc_pingpong of ~7 microseconds >> reasonable? >> >> Thanks, >> murray smigel >> >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> > From sashak at voltaire.com Thu Mar 13 07:06:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 13 Mar 2008 14:06:08 +0000 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> References: <20080313070617.GU10479@sashak.voltaire.com> <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080313140608.GW10479@sashak.voltaire.com> On 04:35 Thu 13 Mar , Hal Rosenstock wrote: > On Thu, 2008-03-13 at 07:06 +0000, Sasha Khapyorsky wrote: > > > Mar 12 17:05:57 843032 [43204940] -> osm_si_rcv_process: ERR 3610: > > > Bad LinearFDBTop value = 0xC000 on > > > switch 0xb8cffff004879 > > > Forcing correction to 0x0 > > > Mar 12 17:05:58 049623 [41401940] -> Entering STANDBY state > > > > This shows that switch reports bad value of SwitchInfo.LinearFDBTop > > Yes, but standby shouldn't program subnet, so correction shouldn't be > done by standby. It doesn't modify LinearFDBTop on switch, just local variable value during discovery - in case of the master OpenSM will later update the value on switch too. Sasha From dotanb at dev.mellanox.co.il Thu Mar 13 07:02:26 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 13 Mar 2008 16:02:26 +0200 Subject: [ofa-general] proper way to recover from poll CQ failed error In-Reply-To: <47D91DF4.9030402@tradeworx.com> References: <47D81B71.8090902@tradeworx.com> <47D8E048.7050600@dev.mellanox.co.il> <47D91DF4.9030402@tradeworx.com> Message-ID: <47D933F2.3070307@dev.mellanox.co.il> Can you execute ibv_devinfo after you get this error? (I'm trying to reproduce this error in our labs without any success so far). can you please send me the output of "uname -a" + "cat /proc/cpuinfo" Dotan Murray Smigel wrote: > Dotan Barak wrote: > >> Hi. >> >> The fact that ibv_poll_cq failed indicates that something bad happened. >> Usually this failure should create any problem and only the process >> that had the problem is being >> effected from this. >> >> I personally think that the ib_* performance tools are better to >> check the performance of your subnet. > > ** > Which programs are these? > >> >> I will be happy if you'll answer the following questions: >> Is this error is consistent? > > ** > Yes. From this morning... > murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 1 -n 1000 > ... > 2000 bytes in 0.00 seconds = 3.56 Mbit/sec > 1000 iters in 0.00 seconds = 4.49 usec/iter > > murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 100 -n 1000 > ... > 200000 bytes in 0.00 seconds = 357.46 Mbit/sec > 1000 iters in 0.00 seconds = 4.48 usec/iter > > murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 200 -n 1000 > ... > 400000 bytes in 0.00 seconds = 711.74 Mbit/sec > 1000 iters in 0.00 seconds = 4.50 usec/iter > > murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 500 -n 1000 > ... > 1000000 bytes in 0.00 seconds = 1752.08 Mbit/sec > 1000 iters in 0.00 seconds = 4.57 usec/iter > > murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 1000 -n 1000 > ... > 2000000 bytes in 0.01 seconds = 2973.98 Mbit/sec > 1000 iters in 0.01 seconds = 5.38 usec/iter > > murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 2000 -n 1000 > ... > 4000000 bytes in 0.01 seconds = 5226.20 Mbit/sec > 1000 iters in 0.01 seconds = 6.12 usec/iter > > murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 5000 -n 1000 > ... > poll CQ failed -2 > > Now, this used to work: > murray at nasnu2:/usr/local/bin$ ./ibv_srq_pingpong -s 1 -n 1000 > ... > poll CQ failed -2 > > As did this: > murray at nasnu2:/usr/local/bin$ ./ibv_rc_pingpong -s 1 -n 1000 > local address: LID 0x0003, QPN 0xa9004a, PSN 0x540960 > remote address: LID 0x0004, QPN 0xaf004a, PSN 0x787c46 > poll CQ failed -2 > >> Can you please send me the output of the ibv_devinfo of your machines? > > murray at nasnu2:/usr/local/bin$ ./ibv_devinfo > hca_id: mlx4_0 > fw_ver: 2.3.000 > node_guid: 0002:c903:0000:c51c > sys_image_guid: 0002:c903:0000:c51f > vendor_id: 0x02c9 > vendor_part_id: 25418 > hw_ver: 0xA0 > board_id: MT_04A0120002 > phys_port_cnt: 2 > port: 1 > state: PORT_ACTIVE (4) > max_mtu: 2048 (4) > active_mtu: 2048 (4) > sm_lid: 3 > port_lid: 3 > port_lmc: 0x00 > > port: 2 > state: PORT_DOWN (1) > max_mtu: 2048 (4) > active_mtu: 2048 (4) > sm_lid: 0 > port_lid: 0 > port_lmc: 0x00 > > > murray at nasnu3:~$ cd /usr/local/bin > murray at nasnu3:/usr/local/bin$ ./ibv_devinfo > hca_id: mlx4_0 > fw_ver: 2.3.000 > node_guid: 0002:c903:0000:c474 > sys_image_guid: 0002:c903:0000:c477 > vendor_id: 0x02c9 > vendor_part_id: 25418 > hw_ver: 0xA0 > board_id: MT_04A0120002 > phys_port_cnt: 2 > port: 1 > state: PORT_ACTIVE (4) > max_mtu: 2048 (4) > active_mtu: 2048 (4) > sm_lid: 3 > port_lid: 4 > port_lmc: 0x00 > > port: 2 > state: PORT_DOWN (1) > max_mtu: 2048 (4) > active_mtu: 2048 (4) > sm_lid: 0 > port_lid: 0 > port_lmc: 0x00 > > >> Did you have any error message in the /var/log/messages when you saw >> this error? > > > > nasnu2:/usr/local/bin# tail /var/log/messages > Mar 13 05:30:05 nasnu2 -- MARK -- > Mar 13 05:50:05 nasnu2 -- MARK -- > Mar 13 06:10:05 nasnu2 -- MARK -- > Mar 13 06:30:05 nasnu2 -- MARK -- > Mar 13 06:50:05 nasnu2 -- MARK -- > Mar 13 07:10:05 nasnu2 -- MARK -- > Mar 13 07:30:05 nasnu2 -- MARK -- > Mar 13 07:36:09 nasnu2 syslogd 1.4.1#18: restart. > Mar 13 07:50:05 nasnu2 -- MARK -- > Mar 13 08:10:05 nasnu2 -- MARK -- > > nasnu2:/usr/local/bin# tail /var/log/opensm.log > Mar 13 08:16:46 024587 [43806960] 0x02 -> SUBNET UP > Mar 13 08:16:56 024615 [43806960] 0x02 -> SUBNET UP > Mar 13 08:17:06 024596 [43806960] 0x02 -> SUBNET UP > Mar 13 08:17:16 024549 [43806960] 0x02 -> SUBNET UP > Mar 13 08:17:26 024664 [43806960] 0x02 -> SUBNET UP > Mar 13 08:17:36 024626 [43806960] 0x02 -> SUBNET UP > Mar 13 08:17:46 024627 [43806960] 0x02 -> SUBNET UP > Mar 13 08:17:56 024611 [43806960] 0x02 -> SUBNET UP > Mar 13 08:18:06 024607 [43806960] 0x02 -> SUBNET UP > > Thanks for your help, > murray smigel > > >> >> thanks >> Dotan >> >> Murray Smigel wrote: >> >>> Hi, >>> I am running OFED-3.0 using ConnectX adapters in a two machine >>> direct connect mode. >>> Most of the various pingpong tests seem ok, but when I run >>> ibv_srq_pingpong -s 500 -n 1000 >>> >>> I get poll "CQ failed -2" when I start up the client side. Smaller >>> values of -s worked fine. >>> Once this happens, no other pingpong tests seem to work. >>> I have then unloaded all the ib_* mlmx_* and iw_* modules, reloaded >>> them and things still >>> fail. I have to reboot the machines to get things back. >>> >>> 1) Is there a cleaner way to recover from this situation? >>> 2) Is the initial failure an indication that something else is wrong? >>> 3) Is the -s 1 latency I see with ibv_rc_pingpong of ~7 microseconds >>> reasonable? >>> >>> Thanks, >>> murray smigel >>> >>> >>> _______________________________________________ >>> general mailing list >>> general at lists.openfabrics.org >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>> >>> To unsubscribe, please visit >>> http://openib.org/mailman/listinfo/openib-general >>> >> > > From hrosenstock at xsigo.com Thu Mar 13 07:10:30 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 13 Mar 2008 07:10:30 -0700 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: <20080313140608.GW10479@sashak.voltaire.com> References: <20080313070617.GU10479@sashak.voltaire.com> <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> <20080313140608.GW10479@sashak.voltaire.com> Message-ID: <1205417430.28072.190.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-13 at 14:06 +0000, Sasha Khapyorsky wrote: > On 04:35 Thu 13 Mar , Hal Rosenstock wrote: > > On Thu, 2008-03-13 at 07:06 +0000, Sasha Khapyorsky wrote: > > > > Mar 12 17:05:57 843032 [43204940] -> osm_si_rcv_process: ERR 3610: > > > > Bad LinearFDBTop value = 0xC000 on > > > > switch 0xb8cffff004879 > > > > Forcing correction to 0x0 > > > > Mar 12 17:05:58 049623 [41401940] -> Entering STANDBY state > > > > > > This shows that switch reports bad value of SwitchInfo.LinearFDBTop > > > > Yes, but standby shouldn't program subnet, so correction shouldn't be > > done by standby. > > It doesn't modify LinearFDBTop on switch, just local variable value > during discovery - in case of the master OpenSM will later update the > value on switch too. Isn't message which states "Forcing correction to 0x0" misleading on standby and should be conditionalized for master only ? -- Hal > > Sasha From murray at tradeworx.com Thu Mar 13 07:36:38 2008 From: murray at tradeworx.com (Murray Smigel) Date: Thu, 13 Mar 2008 10:36:38 -0400 Subject: [ofa-general] proper way to recover from poll CQ failed error In-Reply-To: <47D933F2.3070307@dev.mellanox.co.il> References: <47D81B71.8090902@tradeworx.com> <47D8E048.7050600@dev.mellanox.co.il> <47D91DF4.9030402@tradeworx.com> <47D933F2.3070307@dev.mellanox.co.il> Message-ID: <47D93BF6.2060003@tradeworx.com> Dotan Barak wrote: > Can you execute ibv_devinfo after you get this error? murray at nasnu2:/usr/local/bin$ ./ibv_devinfo hca_id: mlx4_0 fw_ver: 2.3.000 node_guid: 0002:c903:0000:c51c sys_image_guid: 0002:c903:0000:c51f vendor_id: 0x02c9 vendor_part_id: 25418 hw_ver: 0xA0 board_id: MT_04A0120002 phys_port_cnt: 2 port: 1 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 3 port_lid: 3 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 0 port_lid: 0 port_lmc: 0x00 > (I'm trying to reproduce this error in our labs without any success so > far). > > can you please send me the output of "uname -a" + "cat /proc/cpuinfo" murray at nasnu2:~/tmp$ uname -a Linux nasnu2 2.6.24infiniband #2 SMP Tue Mar 11 14:55:26 EDT 2008 x86_64 GNU/Linux murray at nasnu2:~/tmp$ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Xeon(R) CPU 5130 @ 2.00GHz stepping : 11 cpu MHz : 1994.999 cache size : 4096 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm bogomips : 3992.98 clflush size : 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Xeon(R) CPU 5130 @ 2.00GHz stepping : 11 cpu MHz : 1994.999 cache size : 4096 KB physical id : 3 siblings : 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm bogomips : 3990.10 clflush size : 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Xeon(R) CPU 5130 @ 2.00GHz stepping : 11 cpu MHz : 1994.999 cache size : 4096 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm bogomips : 3990.09 clflush size : 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Xeon(R) CPU 5130 @ 2.00GHz stepping : 11 cpu MHz : 1994.999 cache size : 4096 KB physical id : 3 siblings : 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm bogomips : 3990.10 clflush size : 64 cache_alignment : 64 address sizes : 38 bits physical, 48 bits virtual power management: Thanks, murray smigel From avbdulhcju at bostonretail.com Thu Mar 13 08:11:13 2008 From: avbdulhcju at bostonretail.com (Rocky Washington) Date: Fri, 14 Mar 2008 00:11:13 +0900 Subject: [ofa-general] Help your loved one stop smoking Message-ID: <01c88567$e9c22680$1ea4f87b@avbdulhcju> Quit smoking today At least read it! It is YOUR life, it is important. Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: ls.zip Type: application/zip Size: 560 bytes Desc: not available URL: From chu11 at llnl.gov Thu Mar 13 10:14:03 2008 From: chu11 at llnl.gov (Al Chu) Date: Thu, 13 Mar 2008 10:14:03 -0700 Subject: [ofa-general] [PATCH] Opensm: portbalance console option Message-ID: <1205428443.17718.135.camel@cardanus.llnl.gov> Hey Sasha, This follows up my switchbalance console option patch. This console command is similar to the switchbalance one, but it determines if individual ports have balanced routing on a switch, i.e. are they forwarded to remote locations "evenly" and forwarded out of ports "evenly". Al -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-add-portbalance-command-to-console.patch Type: text/x-patch Size: 6902 bytes Desc: not available URL: From rdreier at cisco.com Thu Mar 13 10:37:39 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 13 Mar 2008 10:37:39 -0700 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: <200803130830.03422.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Thu, 13 Mar 2008 08:30:02 +0200") References: <200803130830.03422.jackm@dev.mellanox.co.il> Message-ID: > http://lists.openfabrics.org/pipermail/general/2007-October/042351.html > http://lists.openfabrics.org/pipermail/general/2007-October/042352.html libmlx4 (userspace) patches, right? > http://lists.openfabrics.org/pipermail/general/2007-October/042355.html It's not clear from this message exactly what this is fixing. Is this causing any problems in practice? - R. From rdreier at cisco.com Thu Mar 13 10:37:55 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 13 Mar 2008 10:37:55 -0700 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: <200803130838.07544.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Thu, 13 Mar 2008 08:38:07 +0200") References: <200803130838.07544.jackm@dev.mellanox.co.il> Message-ID: > Also, what about the XRC patches? Clearly not for 2.6.25... From rdreier at cisco.com Thu Mar 13 10:45:51 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 13 Mar 2008 10:45:51 -0700 Subject: [ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: <47D8FC31.9050105@voltaire.com> (Or Gerlitz's message of "Thu, 13 Mar 2008 12:04:33 +0200") References: <1205329275.26105.28.camel@mtls03> <47D8FC31.9050105@voltaire.com> Message-ID: > Just curious why you think extra pointer indirection for each send is > worse than TLB misses taken by using vmalloc(), are the latter being > excepted very un often? Also assuming keeping the dma addresses at the > TX descriptor is a practice taken also by Ethernet drivers who do have > big send queues, maybe we should look there to see how this problem > was solved? It's just a gut feeling. Maybe it's wrong on sparc64 or powerpc where the MMU isn't as good as the x86's. As far as other ethernet drivers, using vmalloc seems fairly common: e1000/e1000_main.c: txdr->buffer_info = vmalloc(size); e1000/e1000_main.c: rxdr->buffer_info = vmalloc(size); bnx2.c: bp->rx_buf_ring = vmalloc(sizeof(struct sw_bd) * RX_DESC_CNT * [bnx2x does the same thing but hides it in a macro] ixgbe/ixgbe_main.c: txdr->tx_buffer_info = vmalloc(size); ixgbe/ixgbe_main.c: rxdr->rx_buffer_info = vmalloc(size); e1000e/netdev.c: tx_ring->buffer_info = vmalloc(size); e1000e/netdev.c: rx_ring->buffer_info = vmalloc(size); netxen/netxen_nic_main.c: cmd_buf_arr = (struct netxen_cmd_buffer *)vmalloc(TX_RINGSIZE); etc From chu11 at llnl.gov Thu Mar 13 10:59:11 2008 From: chu11 at llnl.gov (Al Chu) Date: Thu, 13 Mar 2008 10:59:11 -0700 Subject: [ofa-general] [PATCH] Opensm: portbalance console option In-Reply-To: <1205428443.17718.135.camel@cardanus.llnl.gov> References: <1205428443.17718.135.camel@cardanus.llnl.gov> Message-ID: <1205431151.17718.143.camel@cardanus.llnl.gov> Spoke to Ira, and he says he was a bit confused on what this console command did. In the end, we felt that naming it "lidbalance" was maybe a better name for the command. It's useful under LMC > 0 circumstances to detect when multiple lids are not routed evenly (fixed via my 'multi lid routing balancing' patch a week ago). Al On Thu, 2008-03-13 at 10:14 -0700, Al Chu wrote: > Hey Sasha, > > This follows up my switchbalance console option patch. This console > command is similar to the switchbalance one, but it determines if > individual ports have balanced routing on a switch, i.e. are they > forwarded to remote locations "evenly" and forwarded out of ports > "evenly". > > Al > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-add-lidbalance-command-to-console.patch Type: text/x-patch Size: 6901 bytes Desc: not available URL: From vcldywqmcnkl at bohnanddawson.com Thu Mar 13 11:30:49 2008 From: vcldywqmcnkl at bohnanddawson.com (Chi Burton) Date: Fri, 14 Mar 2008 00:00:49 +0530 Subject: [ofa-general] Live longer life without cigarettes Message-ID: <01c88566$75d34e80$898ac174@vcldywqmcnkl> Live longer life without cigarettes At least read it! It is YOUR life, it is important. Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: ls.zip Type: application/zip Size: 560 bytes Desc: not available URL: From rdreier at cisco.com Thu Mar 13 13:15:11 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 13 Mar 2008 13:15:11 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get some more small post-2.6.25-rc5 fixes, mostly to the ipath driver but some to IP-over-IB also: Or Gerlitz (1): IPoIB: Don't drop multicast sends when they can be queued Patrick Marchand Latifi (4): IB/ipath: Fix potentially wrong RNR retry counter returned in ipath_query_qp() IB/ipath: Fix RC QP initialization IB/ipath: Fix error completion put on send CQ instead of recv CQ IB/ipath: Reset the retry counter for RDMA_READ_RESPONSE_MIDDLE packets Ralph Campbell (1): IB/ipath: Fix IB compliance problems with link state vs physical state Roland Dreier (2): IPoIB/cm: Set tx_wr.num_sge in connected mode post_send() IPoIB: Allocate priv->tx_ring with vmalloc() drivers/infiniband/hw/ipath/ipath_common.h | 2 +- drivers/infiniband/hw/ipath/ipath_driver.c | 28 ++++++++++------------- drivers/infiniband/hw/ipath/ipath_kernel.h | 1 + drivers/infiniband/hw/ipath/ipath_mad.c | 7 ++--- drivers/infiniband/hw/ipath/ipath_qp.c | 13 ++++++----- drivers/infiniband/hw/ipath/ipath_rc.c | 4 +++ drivers/infiniband/hw/ipath/ipath_registers.h | 2 +- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 9 +++++-- drivers/infiniband/ulp/ipoib/ipoib_main.c | 9 ++++--- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 2 +- 10 files changed, 41 insertions(+), 36 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_common.h b/drivers/infiniband/hw/ipath/ipath_common.h index 4146210..591901a 100644 --- a/drivers/infiniband/hw/ipath/ipath_common.h +++ b/drivers/infiniband/hw/ipath/ipath_common.h @@ -75,7 +75,7 @@ #define IPATH_IB_LINKDOWN 0 #define IPATH_IB_LINKARM 1 #define IPATH_IB_LINKACTIVE 2 -#define IPATH_IB_LINKINIT 3 +#define IPATH_IB_LINKDOWN_ONLY 3 #define IPATH_IB_LINKDOWN_SLEEP 4 #define IPATH_IB_LINKDOWN_DISABLE 5 #define IPATH_IB_LINK_LOOPBACK 6 /* enable local loopback */ diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index d5ff6ca..ca4d0ac 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -851,8 +851,7 @@ void ipath_disarm_piobufs(struct ipath_devdata *dd, unsigned first, * -ETIMEDOUT state can have multiple states set, for any of several * transitions. */ -static int ipath_wait_linkstate(struct ipath_devdata *dd, u32 state, - int msecs) +int ipath_wait_linkstate(struct ipath_devdata *dd, u32 state, int msecs) { dd->ipath_state_wanted = state; wait_event_interruptible_timeout(ipath_state_wait, @@ -1656,8 +1655,8 @@ void ipath_cancel_sends(struct ipath_devdata *dd, int restore_sendctrl) static void ipath_set_ib_lstate(struct ipath_devdata *dd, int which) { static const char *what[4] = { - [0] = "DOWN", - [INFINIPATH_IBCC_LINKCMD_INIT] = "INIT", + [0] = "NOP", + [INFINIPATH_IBCC_LINKCMD_DOWN] = "DOWN", [INFINIPATH_IBCC_LINKCMD_ARMED] = "ARMED", [INFINIPATH_IBCC_LINKCMD_ACTIVE] = "ACTIVE" }; @@ -1672,9 +1671,9 @@ static void ipath_set_ib_lstate(struct ipath_devdata *dd, int which) (dd, dd->ipath_kregs->kr_ibcstatus) >> INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT) & INFINIPATH_IBCS_LINKTRAININGSTATE_MASK]); - /* flush all queued sends when going to DOWN or INIT, to be sure that + /* flush all queued sends when going to DOWN to be sure that * they don't block MAD packets */ - if (!linkcmd || linkcmd == INFINIPATH_IBCC_LINKCMD_INIT) + if (linkcmd == INFINIPATH_IBCC_LINKCMD_DOWN) ipath_cancel_sends(dd, 1); ipath_write_kreg(dd, dd->ipath_kregs->kr_ibcctrl, @@ -1687,6 +1686,13 @@ int ipath_set_linkstate(struct ipath_devdata *dd, u8 newstate) int ret; switch (newstate) { + case IPATH_IB_LINKDOWN_ONLY: + ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_DOWN << + INFINIPATH_IBCC_LINKCMD_SHIFT); + /* don't wait */ + ret = 0; + goto bail; + case IPATH_IB_LINKDOWN: ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKINITCMD_POLL << INFINIPATH_IBCC_LINKINITCMD_SHIFT); @@ -1709,16 +1715,6 @@ int ipath_set_linkstate(struct ipath_devdata *dd, u8 newstate) ret = 0; goto bail; - case IPATH_IB_LINKINIT: - if (dd->ipath_flags & IPATH_LINKINIT) { - ret = 0; - goto bail; - } - ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_INIT << - INFINIPATH_IBCC_LINKCMD_SHIFT); - lstate = IPATH_LINKINIT; - break; - case IPATH_IB_LINKARM: if (dd->ipath_flags & IPATH_LINKARMED) { ret = 0; diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 4cc0f95..ecf3f7f 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -767,6 +767,7 @@ void ipath_kreceive(struct ipath_portdata *); int ipath_setrcvhdrsize(struct ipath_devdata *, unsigned); int ipath_reset_device(int); void ipath_get_faststats(unsigned long); +int ipath_wait_linkstate(struct ipath_devdata *, u32, int); int ipath_set_linkstate(struct ipath_devdata *, u8); int ipath_set_mtu(struct ipath_devdata *, u16); int ipath_set_lid(struct ipath_devdata *, u32, u8); diff --git a/drivers/infiniband/hw/ipath/ipath_mad.c b/drivers/infiniband/hw/ipath/ipath_mad.c index d98d5f1..b34b91d 100644 --- a/drivers/infiniband/hw/ipath/ipath_mad.c +++ b/drivers/infiniband/hw/ipath/ipath_mad.c @@ -555,10 +555,7 @@ static int recv_subn_set_portinfo(struct ib_smp *smp, /* FALLTHROUGH */ case IB_PORT_DOWN: if (lstate == 0) - if (get_linkdowndefaultstate(dd)) - lstate = IPATH_IB_LINKDOWN_SLEEP; - else - lstate = IPATH_IB_LINKDOWN; + lstate = IPATH_IB_LINKDOWN_ONLY; else if (lstate == 1) lstate = IPATH_IB_LINKDOWN_SLEEP; else if (lstate == 2) @@ -568,6 +565,8 @@ static int recv_subn_set_portinfo(struct ib_smp *smp, else goto err; ipath_set_linkstate(dd, lstate); + ipath_wait_linkstate(dd, IPATH_LINKINIT | IPATH_LINKARMED | + IPATH_LINKACTIVE, 1000); break; case IB_PORT_ARMED: ipath_set_linkstate(dd, IPATH_IB_LINKARM); diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 80dc623..087ed31 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -329,8 +329,9 @@ struct ipath_qp *ipath_lookup_qpn(struct ipath_qp_table *qpt, u32 qpn) /** * ipath_reset_qp - initialize the QP state to the reset state * @qp: the QP to reset + * @type: the QP type */ -static void ipath_reset_qp(struct ipath_qp *qp) +static void ipath_reset_qp(struct ipath_qp *qp, enum ib_qp_type type) { qp->remote_qpn = 0; qp->qkey = 0; @@ -342,7 +343,7 @@ static void ipath_reset_qp(struct ipath_qp *qp) qp->s_psn = 0; qp->r_psn = 0; qp->r_msn = 0; - if (qp->ibqp.qp_type == IB_QPT_RC) { + if (type == IB_QPT_RC) { qp->s_state = IB_OPCODE_RC_SEND_LAST; qp->r_state = IB_OPCODE_RC_SEND_LAST; } else { @@ -414,7 +415,7 @@ int ipath_error_qp(struct ipath_qp *qp, enum ib_wc_status err) wc.wr_id = qp->r_wr_id; wc.opcode = IB_WC_RECV; wc.status = err; - ipath_cq_enter(to_icq(qp->ibqp.send_cq), &wc, 1); + ipath_cq_enter(to_icq(qp->ibqp.recv_cq), &wc, 1); } wc.status = IB_WC_WR_FLUSH_ERR; @@ -534,7 +535,7 @@ int ipath_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, switch (new_state) { case IB_QPS_RESET: - ipath_reset_qp(qp); + ipath_reset_qp(qp, ibqp->qp_type); break; case IB_QPS_ERR: @@ -647,7 +648,7 @@ int ipath_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, attr->port_num = 1; attr->timeout = qp->timeout; attr->retry_cnt = qp->s_retry_cnt; - attr->rnr_retry = qp->s_rnr_retry; + attr->rnr_retry = qp->s_rnr_retry_cnt; attr->alt_port_num = 0; attr->alt_timeout = 0; @@ -839,7 +840,7 @@ struct ib_qp *ipath_create_qp(struct ib_pd *ibpd, goto bail_qp; } qp->ip = NULL; - ipath_reset_qp(qp); + ipath_reset_qp(qp, init_attr->qp_type); break; default: diff --git a/drivers/infiniband/hw/ipath/ipath_rc.c b/drivers/infiniband/hw/ipath/ipath_rc.c index 459e46e..40f3e37 100644 --- a/drivers/infiniband/hw/ipath/ipath_rc.c +++ b/drivers/infiniband/hw/ipath/ipath_rc.c @@ -1196,6 +1196,10 @@ static inline void ipath_rc_rcv_resp(struct ipath_ibdev *dev, list_move_tail(&qp->timerwait, &dev->pending[dev->pending_index]); spin_unlock(&dev->pending_lock); + + if (opcode == OP(RDMA_READ_RESPONSE_MIDDLE)) + qp->s_retry = qp->s_retry_cnt; + /* * Update the RDMA receive state but do the copy w/o * holding the locks and blocking interrupts. diff --git a/drivers/infiniband/hw/ipath/ipath_registers.h b/drivers/infiniband/hw/ipath/ipath_registers.h index 6d2a17f..92ad73a 100644 --- a/drivers/infiniband/hw/ipath/ipath_registers.h +++ b/drivers/infiniband/hw/ipath/ipath_registers.h @@ -185,7 +185,7 @@ #define INFINIPATH_IBCC_LINKINITCMD_SLEEP 3 #define INFINIPATH_IBCC_LINKINITCMD_SHIFT 16 #define INFINIPATH_IBCC_LINKCMD_MASK 0x3ULL -#define INFINIPATH_IBCC_LINKCMD_INIT 1 /* move to 0x11 */ +#define INFINIPATH_IBCC_LINKCMD_DOWN 1 /* move to 0x11 */ #define INFINIPATH_IBCC_LINKCMD_ARMED 2 /* move to 0x21 */ #define INFINIPATH_IBCC_LINKCMD_ACTIVE 3 /* move to 0x31 */ #define INFINIPATH_IBCC_LINKCMD_SHIFT 18 diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 52b1beb..2490b2d 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -38,6 +38,7 @@ #include #include #include +#include #include "ipoib.h" @@ -637,6 +638,7 @@ static inline int post_send(struct ipoib_dev_priv *priv, priv->tx_sge[0].addr = addr; priv->tx_sge[0].length = len; + priv->tx_wr.num_sge = 1; priv->tx_wr.wr_id = wr_id | IPOIB_OP_CM; return ib_post_send(tx->qp, &priv->tx_wr, &bad_wr); @@ -1030,13 +1032,13 @@ static int ipoib_cm_tx_init(struct ipoib_cm_tx *p, u32 qpn, struct ipoib_dev_priv *priv = netdev_priv(p->dev); int ret; - p->tx_ring = kzalloc(ipoib_sendq_size * sizeof *p->tx_ring, - GFP_KERNEL); + p->tx_ring = vmalloc(ipoib_sendq_size * sizeof *p->tx_ring); if (!p->tx_ring) { ipoib_warn(priv, "failed to allocate tx ring\n"); ret = -ENOMEM; goto err_tx; } + memset(p->tx_ring, 0, ipoib_sendq_size * sizeof *p->tx_ring); p->qp = ipoib_cm_create_tx_qp(p->dev, p); if (IS_ERR(p->qp)) { @@ -1077,6 +1079,7 @@ err_id: ib_destroy_qp(p->qp); err_qp: p->qp = NULL; + vfree(p->tx_ring); err_tx: return ret; } @@ -1127,7 +1130,7 @@ timeout: if (p->qp) ib_destroy_qp(p->qp); - kfree(p->tx_ring); + vfree(p->tx_ring); kfree(p); } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index f96477a..5728204 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -41,6 +41,7 @@ #include #include #include +#include #include /* For ARPHRD_xxx */ @@ -887,13 +888,13 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port) goto out; } - priv->tx_ring = kzalloc(ipoib_sendq_size * sizeof *priv->tx_ring, - GFP_KERNEL); + priv->tx_ring = vmalloc(ipoib_sendq_size * sizeof *priv->tx_ring); if (!priv->tx_ring) { printk(KERN_WARNING "%s: failed to allocate TX ring (%d entries)\n", ca->name, ipoib_sendq_size); goto out_rx_ring_cleanup; } + memset(priv->tx_ring, 0, ipoib_sendq_size * sizeof *priv->tx_ring); /* priv->tx_head, tx_tail & tx_outstanding are already 0 */ @@ -903,7 +904,7 @@ int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port) return 0; out_tx_ring_cleanup: - kfree(priv->tx_ring); + vfree(priv->tx_ring); out_rx_ring_cleanup: kfree(priv->rx_ring); @@ -928,7 +929,7 @@ void ipoib_dev_cleanup(struct net_device *dev) ipoib_ib_dev_cleanup(dev); kfree(priv->rx_ring); - kfree(priv->tx_ring); + vfree(priv->tx_ring); priv->rx_ring = NULL; priv->tx_ring = NULL; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 2628339..31a53c5 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -650,7 +650,7 @@ void ipoib_mcast_send(struct net_device *dev, void *mgid, struct sk_buff *skb) */ spin_lock(&priv->lock); - if (!test_bit(IPOIB_MCAST_STARTED, &priv->flags) || + if (!test_bit(IPOIB_FLAG_OPER_UP, &priv->flags) || !priv->broadcast || !test_bit(IPOIB_MCAST_FLAG_ATTACHED, &priv->broadcast->flags)) { ++dev->stats.tx_dropped; From bigirl.iy at fibertel.com.ar Thu Mar 13 13:11:08 2008 From: bigirl.iy at fibertel.com.ar (Bi-Mdel) Date: Thu, 13 Mar 2008 21:11:08 +0100 Subject: [ofa-general] Was ist los mit Dir? Message-ID: Hallo! Habe Dir schonmal geantwortet auf deine Mail, aber es kam keine Reaktion mehr von Dir. Hast Du kein Interesse mehr an einem geiles Sexdate mit mir? Ich suche noch immer geile erotische Abenteuer. Melde Dich �ber meine Homepage bei mir: http://www.bigirl-2008.tk Auf meiner privaten Homepage findest Du auch Fotos von mir, ich hoffe das ich Dir gefalle. Oder hast Du dich vielleicht deshalb nicht gemeldet? Gefalle ich Dir nicht? W�rde mich sehr freuen wenn Du dich meldest. Auf meine Homepage findest Du auch ne Telefonnummer ;-) Ruf ruhig mal an... Freue mich auf Dich! Ganz liebe Gr�sse Bi-M�del From maxloragno at mailcompserv.com Thu Mar 13 13:33:59 2008 From: maxloragno at mailcompserv.com (University Degree) Date: Thu, 13 Mar 2008 21:33:59 +0100 Subject: [ofa-general] Change Life, Get University Degree Message-ID: <779019094.30909462805174@mailcompserv.com> 1-410-230-1849 University Degree These are real, genuine degrees that include Bachelors, Masters, MBA and Doctorate Degrees. Confidentiality Assured 1-410-230-1849 24 hours a day, 7 days a week including Sundays and Holidays -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Thu Mar 13 13:47:32 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 13 Mar 2008 20:47:32 +0000 Subject: [ofa-general] [PATCH] Opensm: switchbalance console option In-Reply-To: <1205361443.17718.121.camel@cardanus.llnl.gov> References: <1205341036.17718.87.camel@cardanus.llnl.gov> <1205361443.17718.121.camel@cardanus.llnl.gov> Message-ID: <20080313204732.GA10479@sashak.voltaire.com> On 15:37 Wed 12 Mar , Al Chu wrote: > Hey Sasha, > > Forgot to run this through osm_indent. Here's the cleaned up patch. > > Al > > On Wed, 2008-03-12 at 09:57 -0700, Al Chu wrote: > > Hey Sasha, > > > > Here's a patch that does the previously discussed switchbalance console > > option. Algorithmically it does pretty much the exact same thing as the > > check_lft_balance script, but everything is faster of course b/c opensm > > already knows everything. > > > > Al > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > -- > Albert Chu > chu11 at llnl.gov > 925-422-5311 > Computer Scientist > High Performance Systems Division > Lawrence Livermore National Laboratory > From ffdba2635aeea1d1be58c8966f8f6137bb048dea Mon Sep 17 00:00:00 2001 > From: Albert L. Chu > Date: Wed, 12 Mar 2008 15:34:49 -0700 > Subject: [PATCH] add switchbalance command to console > > > Signed-off-by: Albert L. Chu Applied. Thanks. Small question is below. [snip...] > > +static void switchbalance_check(osm_opensm_t * p_osm, > + osm_switch_t * p_sw, FILE * out, int verbose) > +{ > + uint8_t port_num; > + uint8_t num_ports; > + const cl_qmap_t *p_port_tbl; > + osm_port_t *p_port; > + osm_physp_t *p_physp; > + osm_physp_t *p_rem_physp; > + osm_node_t *p_rem_node; > + uint32_t count[255]; /* max ports is a uint8_t */ > + uint8_t output_ports[255]; > + uint8_t output_ports_count = 0; > + uint32_t min_count = 0xFFFFFFFF; > + uint32_t max_count = 0; > + unsigned int i; > + > + memset(count, '\0', sizeof(uint32_t) * 255); > + > + /* Count port usage */ > + p_port_tbl = &p_osm->subn.port_guid_tbl; > + for (p_port = (osm_port_t *) cl_qmap_head(p_port_tbl); > + p_port != (osm_port_t *) cl_qmap_end(p_port_tbl); > + p_port = (osm_port_t *) cl_qmap_next(&p_port->map_item)) { > + uint16_t min_lid_ho; > + uint16_t max_lid_ho; > + uint16_t lid_ho; > + > + /* Don't count switches in port usage */ > + if (osm_node_get_type(p_port->p_node) == IB_NODE_TYPE_SWITCH) > + continue; > + > + osm_port_get_lid_range_ho(p_port, &min_lid_ho, &max_lid_ho); > + > + if (min_lid_ho == 0 || max_lid_ho == 0) > + continue; > + > + for (lid_ho = min_lid_ho; lid_ho <= max_lid_ho; lid_ho++) { > + port_num = osm_fwd_tbl_get(&(p_sw->fwd_tbl), lid_ho); > + if (port_num == OSM_NO_PATH) > + continue; > + > + count[port_num]++; > + } > + } > + > + num_ports = p_sw->num_ports; > + for (port_num = 1; port_num < num_ports; port_num++) { > + p_physp = osm_node_get_physp_ptr(p_sw->p_node, port_num); > + > + /* if port is down/unhealthy, don't consider it in > + * min/max calculations > + */ > + if (!p_physp || !osm_physp_is_healthy(p_physp) > + || !osm_physp_get_remote(p_physp)) > + continue; > + > + p_rem_physp = osm_physp_get_remote(p_physp); > + p_rem_node = osm_physp_get_node_ptr(p_rem_physp); > + > + /* If we are directly connected to a CA, its not really > + * up for balancing consideration. > + */ > + if (osm_node_get_type(p_rem_node) == IB_NODE_TYPE_CA) > + continue; Should this be if (osm_node_get_type(p_rem_node) != IB_NODE_TYPE_SWITCH) ? So routers will be not counted too? Sasha From sashak at voltaire.com Thu Mar 13 13:48:36 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 13 Mar 2008 20:48:36 +0000 Subject: [ofa-general] Re: [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080313204836.GB10479@sashak.voltaire.com> On 04:41 Tue 11 Mar , Hal Rosenstock wrote: > infiniband-diags: Fix install of IBswcountlimits.pm script > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Thu Mar 13 13:51:31 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 13 Mar 2008 20:51:31 +0000 Subject: [ofa-general] [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <1205236404.6469.1052.camel@hrosenstock-ws.xsigo.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> <1205236404.6469.1052.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080313205131.GC10479@sashak.voltaire.com> On 04:53 Tue 11 Mar , Hal Rosenstock wrote: > On Tue, 2008-03-11 at 04:41 -0700, Hal Rosenstock wrote: > > infiniband-diags: Fix install of IBswcountlimits.pm script > > Please install for both master and ofed_1_3. I'm not sure about 1.3. At least I would suggest to wait. In my install-sh '-c' does nothing, but it is possible that this option is deprecated in some another libtool version. Sasha From chu11 at llnl.gov Thu Mar 13 14:23:26 2008 From: chu11 at llnl.gov (Al Chu) Date: Thu, 13 Mar 2008 14:23:26 -0700 Subject: [ofa-general] [PATCH] Opensm: switchbalance console option In-Reply-To: <20080313204732.GA10479@sashak.voltaire.com> References: <1205341036.17718.87.camel@cardanus.llnl.gov> <1205361443.17718.121.camel@cardanus.llnl.gov> <20080313204732.GA10479@sashak.voltaire.com> Message-ID: <1205443406.17718.155.camel@cardanus.llnl.gov> Hey Sasha, On Thu, 2008-03-13 at 20:47 +0000, Sasha Khapyorsky wrote: > On 15:37 Wed 12 Mar , Al Chu wrote: > > Hey Sasha, > > > > Forgot to run this through osm_indent. Here's the cleaned up patch. > > > > Al > > > > On Wed, 2008-03-12 at 09:57 -0700, Al Chu wrote: > > > Hey Sasha, > > > > > > Here's a patch that does the previously discussed switchbalance console > > > option. Algorithmically it does pretty much the exact same thing as the > > > check_lft_balance script, but everything is faster of course b/c opensm > > > already knows everything. > > > > > > Al > > > > > > _______________________________________________ > > > general mailing list > > > general at lists.openfabrics.org > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > -- > > Albert Chu > > chu11 at llnl.gov > > 925-422-5311 > > Computer Scientist > > High Performance Systems Division > > Lawrence Livermore National Laboratory > > > From ffdba2635aeea1d1be58c8966f8f6137bb048dea Mon Sep 17 00:00:00 2001 > > From: Albert L. Chu > > Date: Wed, 12 Mar 2008 15:34:49 -0700 > > Subject: [PATCH] add switchbalance command to console > > > > > > Signed-off-by: Albert L. Chu > > Applied. Thanks. Small question is below. > > [snip...] > > > > +static void switchbalance_check(osm_opensm_t * p_osm, > > + osm_switch_t * p_sw, FILE * out, int verbose) > > +{ > > + uint8_t port_num; > > + uint8_t num_ports; > > + const cl_qmap_t *p_port_tbl; > > + osm_port_t *p_port; > > + osm_physp_t *p_physp; > > + osm_physp_t *p_rem_physp; > > + osm_node_t *p_rem_node; > > + uint32_t count[255]; /* max ports is a uint8_t */ > > + uint8_t output_ports[255]; > > + uint8_t output_ports_count = 0; > > + uint32_t min_count = 0xFFFFFFFF; > > + uint32_t max_count = 0; > > + unsigned int i; > > + > > + memset(count, '\0', sizeof(uint32_t) * 255); > > + > > + /* Count port usage */ > > + p_port_tbl = &p_osm->subn.port_guid_tbl; > > + for (p_port = (osm_port_t *) cl_qmap_head(p_port_tbl); > > + p_port != (osm_port_t *) cl_qmap_end(p_port_tbl); > > + p_port = (osm_port_t *) cl_qmap_next(&p_port->map_item)) { > > + uint16_t min_lid_ho; > > + uint16_t max_lid_ho; > > + uint16_t lid_ho; > > + > > + /* Don't count switches in port usage */ > > + if (osm_node_get_type(p_port->p_node) == IB_NODE_TYPE_SWITCH) > > + continue; > > + > > + osm_port_get_lid_range_ho(p_port, &min_lid_ho, &max_lid_ho); > > + > > + if (min_lid_ho == 0 || max_lid_ho == 0) > > + continue; > > + > > + for (lid_ho = min_lid_ho; lid_ho <= max_lid_ho; lid_ho++) { > > + port_num = osm_fwd_tbl_get(&(p_sw->fwd_tbl), lid_ho); > > + if (port_num == OSM_NO_PATH) > > + continue; > > + > > + count[port_num]++; > > + } > > + } > > + > > + num_ports = p_sw->num_ports; > > + for (port_num = 1; port_num < num_ports; port_num++) { > > + p_physp = osm_node_get_physp_ptr(p_sw->p_node, port_num); > > + > > + /* if port is down/unhealthy, don't consider it in > > + * min/max calculations > > + */ > > + if (!p_physp || !osm_physp_is_healthy(p_physp) > > + || !osm_physp_get_remote(p_physp)) > > + continue; > > + > > + p_rem_physp = osm_physp_get_remote(p_physp); > > + p_rem_node = osm_physp_get_node_ptr(p_rem_physp); > > + > > + /* If we are directly connected to a CA, its not really > > + * up for balancing consideration. > > + */ > > + if (osm_node_get_type(p_rem_node) == IB_NODE_TYPE_CA) > > + continue; > > Should this be > > if (osm_node_get_type(p_rem_node) != IB_NODE_TYPE_SWITCH) > > ? So routers will be not counted too? I think you're right. I should adjust for this in my 'lidbalance' script and 'check_lft_balance.pl' tool. I'll post some new patches. Al > Sasha -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory From sashak at voltaire.com Thu Mar 13 14:36:27 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 13 Mar 2008 21:36:27 +0000 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: <1205417430.28072.190.camel@hrosenstock-ws.xsigo.com> References: <20080313070617.GU10479@sashak.voltaire.com> <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> <20080313140608.GW10479@sashak.voltaire.com> <1205417430.28072.190.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080313213627.GD10479@sashak.voltaire.com> On 07:10 Thu 13 Mar , Hal Rosenstock wrote: > > Isn't message which states "Forcing correction to 0x0" misleading It is in sense of OpenSM internal SwitchInfo representation. > on > standby and should be conditionalized for master only ? The message? Why? It should fix this value anyway. Sasha From hrosenstock at xsigo.com Thu Mar 13 14:31:19 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 13 Mar 2008 14:31:19 -0700 Subject: [ofa-general] [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <20080313205131.GC10479@sashak.voltaire.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> <1205236404.6469.1052.camel@hrosenstock-ws.xsigo.com> <20080313205131.GC10479@sashak.voltaire.com> Message-ID: <1205443879.28072.257.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-13 at 20:51 +0000, Sasha Khapyorsky wrote: > On 04:53 Tue 11 Mar , Hal Rosenstock wrote: > > On Tue, 2008-03-11 at 04:41 -0700, Hal Rosenstock wrote: > > > infiniband-diags: Fix install of IBswcountlimits.pm script > > > > Please install for both master and ofed_1_3. > > I'm not sure about 1.3. At least I would suggest to wait. for what ? > In my install-sh '-c' does nothing, Are you sure ? We've seen others which say this but it makes a difference but those could've been transitional. -- Hal > but it is possible that this option is > deprecated in some another libtool version. > Sasha From hrosenstock at xsigo.com Thu Mar 13 14:35:05 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 13 Mar 2008 14:35:05 -0700 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: <20080313213627.GD10479@sashak.voltaire.com> References: <20080313070617.GU10479@sashak.voltaire.com> <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> <20080313140608.GW10479@sashak.voltaire.com> <1205417430.28072.190.camel@hrosenstock-ws.xsigo.com> <20080313213627.GD10479@sashak.voltaire.com> Message-ID: <1205444105.28072.260.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-13 at 21:36 +0000, Sasha Khapyorsky wrote: > On 07:10 Thu 13 Mar , Hal Rosenstock wrote: > > > > Isn't message which states "Forcing correction to 0x0" misleading > > It is in sense of OpenSM internal SwitchInfo representation. The message is unclear and makes it seem like it is updating SwitchInfo. > > on > > standby and should be conditionalized for master only ? > > The message? Why? It should fix this value anyway. on the standby ? For later, if it becomes master ? -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From chu11 at llnl.gov Thu Mar 13 14:53:57 2008 From: chu11 at llnl.gov (Al Chu) Date: Thu, 13 Mar 2008 14:53:57 -0700 Subject: [ofa-general] [PATCH] Opensm: switchbalance console option In-Reply-To: <1205443406.17718.155.camel@cardanus.llnl.gov> References: <1205341036.17718.87.camel@cardanus.llnl.gov> <1205361443.17718.121.camel@cardanus.llnl.gov> <20080313204732.GA10479@sashak.voltaire.com> <1205443406.17718.155.camel@cardanus.llnl.gov> Message-ID: <1205445237.17718.158.camel@cardanus.llnl.gov> Hey Sasha, Here's the patches to support routers in the switchbalance console command and the check_lft_balance.pl script. Thanks, Al On Thu, 2008-03-13 at 14:23 -0700, Al Chu wrote: > Hey Sasha, > > On Thu, 2008-03-13 at 20:47 +0000, Sasha Khapyorsky wrote: > > On 15:37 Wed 12 Mar , Al Chu wrote: > > > Hey Sasha, > > > > > > Forgot to run this through osm_indent. Here's the cleaned up patch. > > > > > > Al > > > > > > On Wed, 2008-03-12 at 09:57 -0700, Al Chu wrote: > > > > Hey Sasha, > > > > > > > > Here's a patch that does the previously discussed switchbalance console > > > > option. Algorithmically it does pretty much the exact same thing as the > > > > check_lft_balance script, but everything is faster of course b/c opensm > > > > already knows everything. > > > > > > > > Al > > > > > > > > _______________________________________________ > > > > general mailing list > > > > general at lists.openfabrics.org > > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > -- > > > Albert Chu > > > chu11 at llnl.gov > > > 925-422-5311 > > > Computer Scientist > > > High Performance Systems Division > > > Lawrence Livermore National Laboratory > > > > > From ffdba2635aeea1d1be58c8966f8f6137bb048dea Mon Sep 17 00:00:00 2001 > > > From: Albert L. Chu > > > Date: Wed, 12 Mar 2008 15:34:49 -0700 > > > Subject: [PATCH] add switchbalance command to console > > > > > > > > > Signed-off-by: Albert L. Chu > > > > Applied. Thanks. Small question is below. > > > > [snip...] > > > > > > +static void switchbalance_check(osm_opensm_t * p_osm, > > > + osm_switch_t * p_sw, FILE * out, int verbose) > > > +{ > > > + uint8_t port_num; > > > + uint8_t num_ports; > > > + const cl_qmap_t *p_port_tbl; > > > + osm_port_t *p_port; > > > + osm_physp_t *p_physp; > > > + osm_physp_t *p_rem_physp; > > > + osm_node_t *p_rem_node; > > > + uint32_t count[255]; /* max ports is a uint8_t */ > > > + uint8_t output_ports[255]; > > > + uint8_t output_ports_count = 0; > > > + uint32_t min_count = 0xFFFFFFFF; > > > + uint32_t max_count = 0; > > > + unsigned int i; > > > + > > > + memset(count, '\0', sizeof(uint32_t) * 255); > > > + > > > + /* Count port usage */ > > > + p_port_tbl = &p_osm->subn.port_guid_tbl; > > > + for (p_port = (osm_port_t *) cl_qmap_head(p_port_tbl); > > > + p_port != (osm_port_t *) cl_qmap_end(p_port_tbl); > > > + p_port = (osm_port_t *) cl_qmap_next(&p_port->map_item)) { > > > + uint16_t min_lid_ho; > > > + uint16_t max_lid_ho; > > > + uint16_t lid_ho; > > > + > > > + /* Don't count switches in port usage */ > > > + if (osm_node_get_type(p_port->p_node) == IB_NODE_TYPE_SWITCH) > > > + continue; > > > + > > > + osm_port_get_lid_range_ho(p_port, &min_lid_ho, &max_lid_ho); > > > + > > > + if (min_lid_ho == 0 || max_lid_ho == 0) > > > + continue; > > > + > > > + for (lid_ho = min_lid_ho; lid_ho <= max_lid_ho; lid_ho++) { > > > + port_num = osm_fwd_tbl_get(&(p_sw->fwd_tbl), lid_ho); > > > + if (port_num == OSM_NO_PATH) > > > + continue; > > > + > > > + count[port_num]++; > > > + } > > > + } > > > + > > > + num_ports = p_sw->num_ports; > > > + for (port_num = 1; port_num < num_ports; port_num++) { > > > + p_physp = osm_node_get_physp_ptr(p_sw->p_node, port_num); > > > + > > > + /* if port is down/unhealthy, don't consider it in > > > + * min/max calculations > > > + */ > > > + if (!p_physp || !osm_physp_is_healthy(p_physp) > > > + || !osm_physp_get_remote(p_physp)) > > > + continue; > > > + > > > + p_rem_physp = osm_physp_get_remote(p_physp); > > > + p_rem_node = osm_physp_get_node_ptr(p_rem_physp); > > > + > > > + /* If we are directly connected to a CA, its not really > > > + * up for balancing consideration. > > > + */ > > > + if (osm_node_get_type(p_rem_node) == IB_NODE_TYPE_CA) > > > + continue; > > > > Should this be > > > > if (osm_node_get_type(p_rem_node) != IB_NODE_TYPE_SWITCH) > > > > ? So routers will be not counted too? > > I think you're right. I should adjust for this in my 'lidbalance' > script and 'check_lft_balance.pl' tool. > > I'll post some new patches. > > Al > > > Sasha > -- > Albert Chu > chu11 at llnl.gov > 925-422-5311 > Computer Scientist > High Performance Systems Division > Lawrence Livermore National Laboratory > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-handle-routers-in-switchbalance-console-command.patch Type: text/x-patch Size: 1067 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-add-router-support-to-check_lft_balance.pl.patch Type: text/x-patch Size: 1106 bytes Desc: not available URL: From chu11 at llnl.gov Thu Mar 13 14:55:30 2008 From: chu11 at llnl.gov (Al Chu) Date: Thu, 13 Mar 2008 14:55:30 -0700 Subject: [ofa-general] [PATCH] Opensm: portbalance console option In-Reply-To: <1205431151.17718.143.camel@cardanus.llnl.gov> References: <1205428443.17718.135.camel@cardanus.llnl.gov> <1205431151.17718.143.camel@cardanus.llnl.gov> Message-ID: <1205445330.17718.161.camel@cardanus.llnl.gov> Hey Sasha, As discussed in the 'switchbalance console option' thread, this patch has the option also handle routers now. Thanks, Al On Thu, 2008-03-13 at 10:59 -0700, Al Chu wrote: > Spoke to Ira, and he says he was a bit confused on what this console > command did. In the end, we felt that naming it "lidbalance" was maybe > a better name for the command. It's useful under LMC > 0 circumstances > to detect when multiple lids are not routed evenly (fixed via my 'multi > lid routing balancing' patch a week ago). > > Al > > On Thu, 2008-03-13 at 10:14 -0700, Al Chu wrote: > > Hey Sasha, > > > > This follows up my switchbalance console option patch. This console > > command is similar to the switchbalance one, but it determines if > > individual ports have balanced routing on a switch, i.e. are they > > forwarded to remote locations "evenly" and forwarded out of ports > > "evenly". > > > > Al > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Albert Chu chu11 at llnl.gov 925-422-5311 Computer Scientist High Performance Systems Division Lawrence Livermore National Laboratory -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-add-lidbalance-command-to-console.patch Type: text/x-patch Size: 7306 bytes Desc: not available URL: From hdvchwsihyve at blueskyradio.com Thu Mar 13 15:20:41 2008 From: hdvchwsihyve at blueskyradio.com (Alva Hartman) Date: Thu, 13 Mar 2008 23:20:41 +0100 Subject: [ofa-general] Save today 60% Off ALL Designer Footwear such as Gucci Prada Chanel Message-ID: <01c88560$da8baa80$5dd30853@hdvchwsihyve> Hey Have you heard? The 2008 Collections are in. Designer Shoes, Heels, Boots, the Year's Craziest sale. Forget department store prices, Find Gucci Prada Chanel Dior, D&G, Dsquared Versace and More, Men & Women HALF-OFF on ALL International Shipping at NO Cost on all orders! http://www.17starshoe.com What are you waiting for? This is how we Save! An Amazing Offer! From sashak at voltaire.com Thu Mar 13 16:31:03 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 13 Mar 2008 23:31:03 +0000 Subject: [ofa-general] [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <1205443879.28072.257.camel@hrosenstock-ws.xsigo.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> <1205236404.6469.1052.camel@hrosenstock-ws.xsigo.com> <20080313205131.GC10479@sashak.voltaire.com> <1205443879.28072.257.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080313233103.GE10479@sashak.voltaire.com> On 14:31 Thu 13 Mar , Hal Rosenstock wrote: > > > > I'm not sure about 1.3. At least I would suggest to wait. > > for what ? For potential problems with other libtool versions. > > In my install-sh '-c' does nothing, > > Are you sure ? Yes. This is: while test $# -ne 0; do case $1 in -c) shift continue;; Sasha From sashak at voltaire.com Thu Mar 13 16:34:29 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 13 Mar 2008 23:34:29 +0000 Subject: [ofa-general] [PATCH] Opensm: switchbalance console option In-Reply-To: <1205445237.17718.158.camel@cardanus.llnl.gov> References: <1205341036.17718.87.camel@cardanus.llnl.gov> <1205361443.17718.121.camel@cardanus.llnl.gov> <20080313204732.GA10479@sashak.voltaire.com> <1205443406.17718.155.camel@cardanus.llnl.gov> <1205445237.17718.158.camel@cardanus.llnl.gov> Message-ID: <20080313233429.GF10479@sashak.voltaire.com> On 14:53 Thu 13 Mar , Al Chu wrote: > Hey Sasha, > > Here's the patches to support routers in the switchbalance console > command and the check_lft_balance.pl script. Applied. Thanks. Sasha From favors at denverage.com Thu Mar 13 16:25:06 2008 From: favors at denverage.com (Klaus Davis) Date: Thu, 13 Mar 2008 19:25:06 -0400 Subject: [ofa-general] Office2OO8 beta by Microsof+ 73, Pc or mac at 86%"discount Message-ID: <000401c88560$ce075200$0100007f@bemhpmq> zend studio - 49 virtual pc 7.0 for mac - 49 stuffit deluxe 11 for mac - 29 sas jmp statistical discovery 7 - 129 zend studio - 49 adobe font folio 11 - 189 readiris pro 11.5 for mac - 39 office professional xp - 49 " goto "allsoft4less. com" in Web Browser Remove " before you goto in Web Browser " Venezuelans Assess Calm After Colombia Tensions As Iran prepares to elect a new parliament on Friday, hundreds of reform candidates have been disqualified by a non-elected council. Iran's leaders say this is democracy. Critics in Iran say it's hardly democratic. From mmw at bollinger.com.au Thu Mar 13 16:33:43 2008 From: mmw at bollinger.com.au (Violet Mendez) Date: Fri, 14 Mar 2008 01:33:43 +0200 Subject: [ofa-general] Live longer life without cigarettes Message-ID: <01c88573$70303580$5469d551@mmw> Show your close ones you care, quit smoking today At least read it! It is YOUR life, it is important. Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: ls.zip Type: application/zip Size: 560 bytes Desc: not available URL: From sashak at voltaire.com Thu Mar 13 17:03:44 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 00:03:44 +0000 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and 1.2.1 specification In-Reply-To: <20080312141015.2b63fd9e.weiny2@llnl.gov> References: <20080312102333.49779c75.weiny2@llnl.gov> <1205345222.28072.61.camel@hrosenstock-ws.xsigo.com> <20080312125448.27887255.weiny2@llnl.gov> <1205353284.28072.106.camel@hrosenstock-ws.xsigo.com> <20080312141015.2b63fd9e.weiny2@llnl.gov> Message-ID: <20080314000344.GG10479@sashak.voltaire.com> On 14:10 Wed 12 Mar , Ira Weiny wrote: > Subject: [PATCH] opensm/include/iba/ib_types.h: fix DataDetails definitions based on 1.2 and > 1.2.1 specification > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From fisexecutivezob at executive.gr Thu Mar 13 16:59:18 2008 From: fisexecutivezob at executive.gr (Arthur Hairston) Date: Fri, 14 Mar 2008 00:59:18 +0100 Subject: [ofa-general] Legal software sales Message-ID: <493197952.33306237617377@executive.gr> Our goal is to render low price PC and Macintosh legal software and computer solutions for anyone. Whether you're a corporate customer, a holder of small business, or shopping for your own home personal computer, we guess that we'll help you. ENJOY OF OUR SOFTWARE! http://averyalexander886356.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Thu Mar 13 17:12:18 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 00:12:18 +0000 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080314001218.GH10479@sashak.voltaire.com> On 11:13 Wed 12 Mar , Hal Rosenstock wrote: > > > > struct _ntc_144 { > > ib_net16_t pad1; > > - ib_net16_t lid; // lid where capability mask changed > > - ib_net16_t pad2; > > - ib_net32_t new_cap_mask; // new capability mask > > + ib_net16_t lid; // lid where change occured > > + uint8_t pad2; // reserved > > + uint8_t local_changes; // 7b reserved 1b local changes > > + ib_net32_t new_cap_mask; // new capability mask > > + ib_net16_t change_flgs; // 13b reserved 3b change flags > > Should this be padded out as in the 1.2.1 spec ? It is stated in the 1.2.1 that an upper bits in change_flgs are reserved for the same purpose - OtherLocalChanges mask. So it looks ok for me to have it as one field and not redo later. Sasha From sashak at voltaire.com Thu Mar 13 17:17:06 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 00:17:06 +0000 Subject: [ofa-general] Re: [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <20080312102335.5eecc8d0.weiny2@llnl.gov> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> Message-ID: <20080314001705.GI10479@sashak.voltaire.com> On 10:23 Wed 12 Mar , Ira Weiny wrote: > > From c5c6b2f6bd3e889a4eb6fa462507fcbf25a433e9 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Mon, 10 Mar 2008 11:19:29 -0700 > Subject: [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From cavfsyqdeoa at blpmobilepaint.com Thu Mar 13 17:10:22 2008 From: cavfsyqdeoa at blpmobilepaint.com (Shawna Bass) Date: Thu, 13 Mar 2008 20:10:22 -0400 Subject: [ofa-general] Be leaner and slimmer by next week! Message-ID: <01c88546$444a8b00$78652cbe@cavfsyqdeoa> Burn pounds off with it You must know this secret! Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: as.zip Type: application/zip Size: 578 bytes Desc: not available URL: From sashak at voltaire.com Thu Mar 13 17:24:06 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 00:24:06 +0000 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: <1205444105.28072.260.camel@hrosenstock-ws.xsigo.com> References: <20080313070617.GU10479@sashak.voltaire.com> <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> <20080313140608.GW10479@sashak.voltaire.com> <1205417430.28072.190.camel@hrosenstock-ws.xsigo.com> <20080313213627.GD10479@sashak.voltaire.com> <1205444105.28072.260.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080314002406.GJ10479@sashak.voltaire.com> On 14:35 Thu 13 Mar , Hal Rosenstock wrote: > On Thu, 2008-03-13 at 21:36 +0000, Sasha Khapyorsky wrote: > > On 07:10 Thu 13 Mar , Hal Rosenstock wrote: > > > > > > Isn't message which states "Forcing correction to 0x0" misleading > > > > It is in sense of OpenSM internal SwitchInfo representation. > > The message is unclear and makes it seem like it is updating SwitchInfo. Which message would be better? Would care about the patch? > > > on > > > standby and should be conditionalized for master only ? > > > > The message? Why? It should fix this value anyway. > > on the standby ? For later, if it becomes master ? Yes (although SwitchInfo should be refetched on new heavy sweep), and in general there is no reason to keep invalid data in OpenSM, so I think the warning is valid. The text itself can mislead... Sasha From sashak at voltaire.com Thu Mar 13 19:55:45 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 02:55:45 +0000 Subject: [ofa-general] [PATCH] Opensm: portbalance console option In-Reply-To: <1205445330.17718.161.camel@cardanus.llnl.gov> References: <1205428443.17718.135.camel@cardanus.llnl.gov> <1205431151.17718.143.camel@cardanus.llnl.gov> <1205445330.17718.161.camel@cardanus.llnl.gov> Message-ID: <20080314025545.GN30723@sashak.voltaire.com> On 14:55 Thu 13 Mar , Al Chu wrote: > Subject: [PATCH] add lidbalance command to console > > > Signed-off-by: Albert L. Chu Applied. Thanks. Sasha From dwpoweredparagliderm at poweredparaglider.com Thu Mar 13 19:52:49 2008 From: dwpoweredparagliderm at poweredparaglider.com (Jody Romano) Date: Fri, 14 Mar 2008 10:52:49 +0800 Subject: [ofa-general] Medications that you need. Message-ID: <01c885c1$8b299680$ae8c133c@dwpoweredparagliderm> Buy Must Have medications at Canada based pharmacy. No prescription at all! Same quality! Save your money, buy pills immediately! http://geocities.com/edmond_holcomb We provide confidential and secure purchase! From dillonbill64 at searchhound.com Thu Mar 13 19:45:33 2008 From: dillonbill64 at searchhound.com (lazar rutland) Date: Fri, 14 Mar 2008 02:45:33 +0000 Subject: [ofa-general] penny Message-ID: <000901c8858c$0387299e$4e281293@xrnehgj> ! bangladeshautomorphic but adrienne From kraft at vfc.com Fri Mar 14 02:34:00 2008 From: kraft at vfc.com (Mattie David) Date: Fri, 14 Mar 2008 10:34:00 +0100 Subject: [ofa-general] Die Software Kann Billig Sein. Ihr PC Ist Froh Message-ID: <650659846.44828281757589@vfc.com> Die echte und vollige Produkte der Software fur wenig Geld? Das ist wirklich. Sie momentan zu bekommen? Ja ist die Antwort. Einfach bezahlen und auslasten. Au?erdem sind die Programmen auf allen europaischen Sprachen uberlassen und fur Windows und Macintosh vorherbestimmt. http://geocities.com/beachelisabeth/Mit der Hilfe der professionellen Konsultation des Anwenderdienstes ist Die Aufstellung des Programms kein Problem fur Ihnen. Antwort ist garantiert. Die Ruckzahlung ist moglich. So konnen Sie die vollkommen funktionierende Software leicht kaufen -------------- next part -------------- An HTML attachment was scrubbed... URL: From Thomas.Talpey at netapp.com Fri Mar 14 04:53:17 2008 From: Thomas.Talpey at netapp.com (Talpey, Thomas) Date: Fri, 14 Mar 2008 07:53:17 -0400 Subject: [ofa-general] AMP/RDMA performance expectation In-Reply-To: <47D63679.8070900@ncic.ac.cn> References: <47D63679.8070900@ncic.ac.cn> Message-ID: At 03:36 AM 3/11/2008, yangdong wrote: > >Hello! I implemented a message passing subsystem -- AMP for our cluster >file system--DCFS3, >I want AMP to run over Infiniband, I designed it in kernel(verbs). >when client writing file, my protocol is implemented by RDMA WRITE, and >reading file by RDMA >READ, but perf of AMP is less than my expectation. Single thread (1t) >read is 215MB/s, 4t is 340MB/s. >And I see that the read file perf of NFS/RDMA is close to 600MB/s (1t). NFS/RDMA is not single threaded. The client has several ways to keep multiple I/Os active, among them use of a kernel rpc state machine, enlisting application threads performing synch i/o, use of kernel helpers (rpciod's), the filesystem buffer cache, and the RPC slot table. Also, its protocol supports credits and RDMA scatter/gather, and is tuned to optimize server transfers. BTW, NFS/RDMA achieves much more than 600MB/s. On DDR IB, for example, Sandia's testing achieved over double that. Just some ideas, since you give so little information about your implementation it's hard to conclude anything. Tom. >I made use of FMR and All physical reg ..and scatter/gather, but why the >perf of AMP is lower? > > > >-- >Dong Yang >Institute of Computing Technology, Chinese Academy of Sciences >Address: National Research Center for Intelligent Computing Systems >(NCIC), P.O. Box 2704, Beijing 100080, P.R. China >Phone: +86-10-62601005 > >_______________________________________________ >general mailing list >general at lists.openfabrics.org >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From nwaitt at waittcorp.com Fri Mar 14 05:14:58 2008 From: nwaitt at waittcorp.com (Jean Tackett) Date: Fri, 14 Mar 2008 19:14:58 +0700 Subject: [ofa-general] Great Selection of Downloadable Software Message-ID: <013782540.79294437529004@waittcorp.com> Our main aim is to render PC and Mac lawful software and computer solutions of low price for anyone. Whether you are a corporate buyer, a possessor of small business, or shopping for your home personal computer, we suppose that we can assist you.http://geocities.com/raymundo_tran/* Adobe Acrobat 8.0 Professional: $69.95 * Windows XP Professional With SP2 Full Version: $59.95 * Office System Professional 2003 (5 Cds): $59.95 * Adobe Photoshop CS2 with ImageReady CS2: $79.95 * AutoCAD 2008: $129.95 http://geocities.com/raymundo_tran/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jack at arkedu.k12.ar.us Fri Mar 14 05:48:11 2008 From: Jack at arkedu.k12.ar.us (Jack Miller) Date: Fri, 14 Mar 2008 14:48:11 +0200 Subject: [ofa-general] I wanted to even the score with a cheating partner. Message-ID: <01bd01c885d1$a9847b30$0a000006@Jack> I just love Adam's new size; it makes me come a lot more easily when we have sex http://junieryi.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sudhirsudhir at sndindia.com Fri Mar 14 06:25:01 2008 From: sudhirsudhir at sndindia.com (Joel Joyner) Date: Fri, 14 Mar 2008 14:25:01 +0100 Subject: [ofa-general] Billig und ausgezeichnete Software Message-ID: <910681610.69257663765841@sndindia.com> Die echte und vollige Produkte der Software fur wenig Geld? Das ist wirklich. Sie momentan zu bekommen? Ja ist die Antwort. Einfach bezahlen und auslasten. Au?erdem sind die Programmen auf allen europaischen Sprachen uberlassen und fur Windows und Macintosh vorherbestimmt. http://geocities.com/galloway.mattie/Haben Sie Haben Sie Schwierigkeiten bei der Aufstellung des Programms? Sie bekommen die Hilfe der professionellen Konsultation des Anwenderdienstes. Haben Sie Fragen? Wir antworten schnell. Die Ruckzahlung ist moglich. Sie kaufen, die Software funktionieren, ausgezeichnet -------------- next part -------------- An HTML attachment was scrubbed... URL: From tecfergisanxow at fergisan.com Fri Mar 14 06:31:30 2008 From: tecfergisanxow at fergisan.com (Bradly Bynum) Date: Fri, 14 Mar 2008 15:31:30 +0200 Subject: [ofa-general] Legal software sales Message-ID: <796656709.86299098160217@fergisan.com> Our main aim is to assure PC and Mac legal soft and computer solutions of low cost for any budget. Whether you're a corporate buyer, an owner of small-scale enterprise, or shopping for your own home personal computer, we believe we can assist you. CHECK WHAT WE GOT TO PROPOSE! http://destinygarcia809534.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Fri Mar 14 07:13:58 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 14 Mar 2008 07:13:58 -0700 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: <20080314002406.GJ10479@sashak.voltaire.com> References: <20080313070617.GU10479@sashak.voltaire.com> <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> <20080313140608.GW10479@sashak.voltaire.com> <1205417430.28072.190.camel@hrosenstock-ws.xsigo.com> <20080313213627.GD10479@sashak.voltaire.com> <1205444105.28072.260.camel@hrosenstock-ws.xsigo.com> <20080314002406.GJ10479@sashak.voltaire.com> Message-ID: <1205504038.28072.300.camel@hrosenstock-ws.xsigo.com> On Fri, 2008-03-14 at 00:24 +0000, Sasha Khapyorsky wrote: > On 14:35 Thu 13 Mar , Hal Rosenstock wrote: > > On Thu, 2008-03-13 at 21:36 +0000, Sasha Khapyorsky wrote: > > > On 07:10 Thu 13 Mar , Hal Rosenstock wrote: > > > > > > > > Isn't message which states "Forcing correction to 0x0" misleading > > > > > > It is in sense of OpenSM internal SwitchInfo representation. > > > > The message is unclear and makes it seem like it is updating SwitchInfo. > > Which message would be better? Would care about the patch? How about the following ? opensm/osm_sw_info_rcv.c: Clarify LinearFDBTop correction log message Signed-off-by: Hal Rosenstock diff --git a/opensm/opensm/osm_sw_info_rcv.c b/opensm/opensm/osm_sw_info_rcv.c index d056c1d..46229dc 100644 --- a/opensm/opensm/osm_sw_info_rcv.c +++ b/opensm/opensm/osm_sw_info_rcv.c @@ -517,7 +517,7 @@ void osm_si_rcv_process(IN void *context, IN void *data) OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 3610: " "\n\t\t\t\tBad LinearFDBTop value = 0x%X " "on switch 0x%" PRIx64 - "\n\t\t\t\tForcing correction to 0x%X\n", + "\n\t\t\t\tForcing internal correction to 0x%X\n", cl_ntoh16(p_si->lin_top), cl_ntoh64(osm_node_get_node_guid(p_node)), 0); > > > > > on > > > > standby and should be conditionalized for master only ? > > > > > > The message? Why? It should fix this value anyway. > > > > on the standby ? For later, if it becomes master ? > > Yes (although SwitchInfo should be refetched on new heavy sweep), and > in general there is no reason to keep invalid data in OpenSM, so I think > the warning is valid. The text itself can mislead... > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Fri Mar 14 07:14:05 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 14 Mar 2008 07:14:05 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <20080314001218.GH10479@sashak.voltaire.com> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> Message-ID: <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> On Fri, 2008-03-14 at 00:12 +0000, Sasha Khapyorsky wrote: > On 11:13 Wed 12 Mar , Hal Rosenstock wrote: > > > > > > struct _ntc_144 { > > > ib_net16_t pad1; > > > - ib_net16_t lid; // lid where capability mask changed > > > - ib_net16_t pad2; > > > - ib_net32_t new_cap_mask; // new capability mask > > > + ib_net16_t lid; // lid where change occured > > > + uint8_t pad2; // reserved > > > + uint8_t local_changes; // 7b reserved 1b local changes > > > + ib_net32_t new_cap_mask; // new capability mask > > > + ib_net16_t change_flgs; // 13b reserved 3b change flags > > > > Should this be padded out as in the 1.2.1 spec ? > > It is stated in the 1.2.1 that an upper bits in change_flgs are reserved > for the same purpose - OtherLocalChanges mask. So it looks ok for me to > have it as one field and not redo later. Have what as one field ? change_flgs ? I was referring to a pad3 at the end of the structure as in the spec. -- Hal > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Fri Mar 14 07:14:10 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 14 Mar 2008 07:14:10 -0700 Subject: [ofa-general] [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <20080313233103.GE10479@sashak.voltaire.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> <1205236404.6469.1052.camel@hrosenstock-ws.xsigo.com> <20080313205131.GC10479@sashak.voltaire.com> <1205443879.28072.257.camel@hrosenstock-ws.xsigo.com> <20080313233103.GE10479@sashak.voltaire.com> Message-ID: <1205504050.28072.303.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-13 at 23:31 +0000, Sasha Khapyorsky wrote: > On 14:31 Thu 13 Mar , Hal Rosenstock wrote: > > > > > > I'm not sure about 1.3. At least I would suggest to wait. > > > > for what ? > > For potential problems with other libtool versions. What possible problems ? It's a noop. > > > In my install-sh '-c' does nothing, > > > > Are you sure ? > > Yes. This is: > > while test $# -ne 0; do > case $1 in > -c) shift > continue;; On mine, it is: -c) instcmd=$cpprog shift continue;; so mine is transitional where it's weaned out of help but still supported. -- Hal > Sasha From efkeq at bonecreeper.com Fri Mar 14 07:41:11 2008 From: efkeq at bonecreeper.com (Pedro Ricks) Date: Fri, 14 Mar 2008 22:41:11 +0800 Subject: [ofa-general] Help your loved one stop smoking Message-ID: <01c88624$809e9400$86d1a576@efkeq> Take control of your life again and beat nicotine At least read it! It is YOUR life, it is important. Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: ls.zip Type: application/zip Size: 560 bytes Desc: not available URL: From cooper at udcja.com Fri Mar 14 09:03:52 2008 From: cooper at udcja.com (Antone Morton) Date: Fri, 14 Mar 2008 17:03:52 +0100 Subject: [ofa-general] Rechtliche Vertrieb von Software Message-ID: <767339144.52568722660553@udcja.com> Industry standard software for less than cheapWir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren.http://geocities.com/erikrandolph73/* Windows XP Professional With SP2 Full Version: $59.95 * Office Enterprise 2007: $79.95 * Adobe Photoshop CS2 with ImageReady CS2: $79.95 http://geocities.com/erikrandolph73/ Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.mietke at informatik.tu-chemnitz.de Fri Mar 14 09:08:42 2008 From: frank.mietke at informatik.tu-chemnitz.de (Frank Mietke) Date: Fri, 14 Mar 2008 17:08:42 +0100 Subject: [ofa-general] kernel panic (sporadically) OFED-1.2 Message-ID: <20080314160842.GV3073@saul.informatik.tu-chemnitz.de> Hi, has anybody seen the kernel panic below? Any hints? We're using RHEL-4 clone with special Lustre kernel 2.6.9-55.0.9.EL_lustre.1.6.4.2smp and OFED-1.2 ib0: queue stopped 1, tx_head 102359453, tx_tail 102359399 ^Mib0: transmit timeout: latency 1010 msecs ^Mib0: queue stopped 1, tx_head 102359503, tx_tail 102359453 ^Mib0: transmit timeout: latency 1140 msecs ^Mib0: queue stopped 1, tx_head 102359542, tx_tail 102359497 ^Mib0: transmit timeout: latency 1350 msecs ^Mib0: queue stopped 1, tx_head 102359624, tx_tail 102359561 ^Mgeneral protection fault: 0000 [1] SMP ^MCPU 3 ^MModules linked in: osc(U) mgc(U) lustre(U) lov(U) lquota(U) mdc(U) ko2iblnd(U) ptlrpc(U) obdclass(U) lnet(U) lvfs(U) libcfs(U) ib_mthca(U) tg3(U) sata_svw(U) ib_sdp(U) ib_srp(U) rdma_ucm(U) rdma_cm(U) iw_cm(U) ib_local_sa(U) ib_addr(U) ib_ipoib(U) ipv6(U) ib_umad(U) ib_ucm(U) ib_cm(U) ib_sa(U) ib_mad(U) ib_uverbs(U) ib_core(U) sd_mod(U) ipmi_devintf(U) ipmi_si(U) ipmi_msghandler(U) ext3(U) jbd(U) nfs(U) lockd(U) nfs_acl(U) sunrpc(U) ohci_hcd(U) ehci_hcd(U) ^MPid: 843, comm: ib_mad1 Tainted: GF 2.6.9-55.0.9.EL_lustre.1.6.4.2smp ^MRIP: 0010:[] {:ib_mthca:mthca_ah_grh_present+4} ^MRSP: 0018:000001007cf5dcb0 EFLAGS: 00010046 ^MRAX: 0000ffff00000001 RBX: 000001012d971b80 RCX: 000001006e0526c0 ^MRDX: 000000000000002c RSI: 000001012d971a00 RDI: 000001007183a380 ^MRBP: 000001012d971a00 R08: 000001007cf2c600 R09: 000001007cf2c610 ^MR10: 0000000000000004 R11: 0000000000000246 R12: 000001007cf2c600 ^MR13: 000001006e0526c0 R14: 000001007d4ef000 R15: 000001007cf2c610 ^MFS: 0000002a95e266e0(0000) GS:ffffffff804a6880(0000) knlGS:0000000000000000 ^MCS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b ^MCR2: 0000002a9556c000 CR3: 0000000080ca0000 CR4: 00000000000006e0 ^MProcess ib_mad1 (pid: 843, threadinfo 000001007cf5c000, task 000001012cc77800) ^MStack: ffffffffa01cf91a 000000000000002c 000001007cf2c610 000001007cf2c580 ^M 000001012d8fc5e0 000001006e0526c0 000001012d971a00 000000000000002c ^M ffffffffa01d0fd8 000001012d8fc5e0 ^MCall Trace:{:ib_mthca:build_mlx_header+45} {:ib_mthca:mthca_arbel_post_send+1558} ^M {:ib_mad:ib_mad_send_done_handler+354} ^M {:ib_mad:ib_mad_completion_handler+1370} ^M {:ib_mad:ib_mad_completion_handler+0} ^M {worker_thread+419} {default_wake_function+0} ^M {__wake_up_common+67} {default_wake_function+0} ^M {keventd_create_kthread+0} {worker_thread+0} ^M {keventd_create_kthread+0} {kthread+200} ^M {child_rip+8} {keventd_create_kthread+0} ^M {kthread+0} {child_rip+0} ^M ^MCode: 0f be 40 05 c1 e8 1f c3 41 54 b8 ea ff ff ff 49 89 fc 55 48 ^MRIP {:ib_mthca:mthca_ah_grh_present+4} RSP <000001007cf5dcb0> ^M <6>NETDEV WATCHDOG: ib0: transmit timed out ^Mib0: transmit timeout: latency 1970 msecs ^Mib0: queue stopped 1, tx_head 102359700, tx_tail 102359664 ^MKernel panic - not syncing: Oops Best Regards, Frank -- Dipl.-Inf. Frank Mietke | Fakultätsrechen- und Informationszentrum Tel.: 0371 - 531 - 35538 | Fak. für Informatik Fax: 0371 - 531 8 35538 | TU-Chemnitz Key-ID: 60F59599 | frank.mietke at informatik.tu-chemnitz.de From riviran at privateforestry.com Fri Mar 14 09:09:29 2008 From: riviran at privateforestry.com (Georgine Bond) Date: Fri, 14 Mar 2008 17:09:29 +0100 Subject: [ofa-general] Enlarge the Size and Boost Your Sex Life Message-ID: <535409229.45016266401733@privateforestry.com> Bigger size, less problems in your sexual life; we know how to make it bigger.Choose a really effective and money worth product which is also the safest one - a VPXL. It has helped a lot of men worldwide to increase their erotic confidence. Order the VPXL and become our happy customer.http://geocities.com/petersen.stevie/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Fri Mar 14 09:31:46 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 16:31:46 +0000 Subject: [ofa-general] [PATCH] infiniband-diags: Fix install of IBswcountlimits.pm script In-Reply-To: <1205504050.28072.303.camel@hrosenstock-ws.xsigo.com> References: <1205235694.6469.1050.camel@hrosenstock-ws.xsigo.com> <1205236404.6469.1052.camel@hrosenstock-ws.xsigo.com> <20080313205131.GC10479@sashak.voltaire.com> <1205443879.28072.257.camel@hrosenstock-ws.xsigo.com> <20080313233103.GE10479@sashak.voltaire.com> <1205504050.28072.303.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080314163146.GN10479@sashak.voltaire.com> On 07:14 Fri 14 Mar , Hal Rosenstock wrote: > On Thu, 2008-03-13 at 23:31 +0000, Sasha Khapyorsky wrote: > > On 14:31 Thu 13 Mar , Hal Rosenstock wrote: > > > > > > > > I'm not sure about 1.3. At least I would suggest to wait. > > > > > > for what ? > > > > For potential problems with other libtool versions. > > What possible problems ? It's a noop. Some libtools could not support it at all. Sasha From toralf.foerster at gmx.de Fri Mar 14 09:23:07 2008 From: toralf.foerster at gmx.de (Toralf =?iso-8859-15?q?F=F6rster?=) Date: Fri, 14 Mar 2008 17:23:07 +0100 Subject: [ofa-general] build #422 issue for v2.6.25-rc5-298-gdba92d3 in profile.c:55: warning: 'res_name' defined but not used Message-ID: <200803141723.10284.toralf.foerster@gmx.de> Hello, the build with the attached .config gives a small warning : ... CC [M] drivers/net/mlx4/profile.o drivers/net/mlx4/profile.c:55: warning: 'res_name' defined but not used CC [M] drivers/net/mlx4/qp.o ... The build was made with : $> make mrproper && make randconfig && make Here's the config: # # Automatically generated make config: don't edit # Linux kernel version: 2.6.25-rc5 # Fri Mar 14 16:16:49 2008 # # CONFIG_64BIT is not set CONFIG_X86_32=y # CONFIG_X86_64 is not set CONFIG_X86=y # CONFIG_GENERIC_LOCKBREAK is not set CONFIG_GENERIC_TIME=y CONFIG_GENERIC_CMOS_UPDATE=y CONFIG_CLOCKSOURCE_WATCHDOG=y CONFIG_GENERIC_CLOCKEVENTS=y CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y CONFIG_LOCKDEP_SUPPORT=y CONFIG_STACKTRACE_SUPPORT=y CONFIG_HAVE_LATENCYTOP_SUPPORT=y CONFIG_SEMAPHORE_SLEEPERS=y CONFIG_FAST_CMPXCHG_LOCAL=y CONFIG_MMU=y CONFIG_ZONE_DMA=y CONFIG_GENERIC_ISA_DMA=y CONFIG_GENERIC_IOMAP=y CONFIG_GENERIC_BUG=y CONFIG_GENERIC_HWEIGHT=y # CONFIG_GENERIC_GPIO is not set CONFIG_ARCH_MAY_HAVE_PC_FDC=y CONFIG_DMI=y # CONFIG_RWSEM_GENERIC_SPINLOCK is not set CONFIG_RWSEM_XCHGADD_ALGORITHM=y # CONFIG_ARCH_HAS_ILOG2_U32 is not set # CONFIG_ARCH_HAS_ILOG2_U64 is not set CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y CONFIG_GENERIC_CALIBRATE_DELAY=y # CONFIG_GENERIC_TIME_VSYSCALL is not set CONFIG_ARCH_HAS_CPU_RELAX=y # CONFIG_HAVE_SETUP_PER_CPU_AREA is not set CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_SUSPEND_POSSIBLE=y # CONFIG_ZONE_DMA32 is not set CONFIG_ARCH_POPULATES_NODE_MAP=y # CONFIG_AUDIT_ARCH is not set CONFIG_ARCH_SUPPORTS_AOUT=y CONFIG_GENERIC_HARDIRQS=y CONFIG_GENERIC_IRQ_PROBE=y CONFIG_GENERIC_PENDING_IRQ=y CONFIG_X86_SMP=y CONFIG_X86_32_SMP=y CONFIG_X86_HT=y CONFIG_X86_BIOS_REBOOT=y CONFIG_X86_TRAMPOLINE=y CONFIG_KTIME_SCALAR=y CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config" # # General setup # CONFIG_EXPERIMENTAL=y CONFIG_LOCK_KERNEL=y CONFIG_INIT_ENV_ARG_LIMIT=32 CONFIG_LOCALVERSION="" # CONFIG_LOCALVERSION_AUTO is not set CONFIG_SYSVIPC=y CONFIG_SYSVIPC_SYSCTL=y CONFIG_BSD_PROCESS_ACCT=y # CONFIG_BSD_PROCESS_ACCT_V3 is not set # CONFIG_IKCONFIG is not set CONFIG_LOG_BUF_SHIFT=14 CONFIG_CGROUPS=y CONFIG_CGROUP_DEBUG=y CONFIG_CGROUP_NS=y CONFIG_CPUSETS=y # CONFIG_GROUP_SCHED is not set CONFIG_CGROUP_CPUACCT=y # CONFIG_RESOURCE_COUNTERS is not set CONFIG_SYSFS_DEPRECATED=y CONFIG_SYSFS_DEPRECATED_V2=y CONFIG_PROC_PID_CPUSET=y # CONFIG_RELAY is not set CONFIG_NAMESPACES=y CONFIG_UTS_NS=y # CONFIG_IPC_NS is not set CONFIG_USER_NS=y CONFIG_PID_NS=y # CONFIG_BLK_DEV_INITRD is not set # CONFIG_CC_OPTIMIZE_FOR_SIZE is not set CONFIG_SYSCTL=y CONFIG_EMBEDDED=y # CONFIG_UID16 is not set CONFIG_SYSCTL_SYSCALL=y CONFIG_KALLSYMS=y CONFIG_KALLSYMS_ALL=y # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_HOTPLUG=y # CONFIG_PRINTK is not set CONFIG_BUG=y CONFIG_ELF_CORE=y # CONFIG_COMPAT_BRK is not set # CONFIG_BASE_FULL is not set CONFIG_FUTEX=y CONFIG_ANON_INODES=y CONFIG_EPOLL=y CONFIG_SIGNALFD=y CONFIG_TIMERFD=y # CONFIG_EVENTFD is not set CONFIG_SHMEM=y # CONFIG_VM_EVENT_COUNTERS is not set # CONFIG_SLUB_DEBUG is not set # CONFIG_SLAB is not set CONFIG_SLUB=y # CONFIG_SLOB is not set CONFIG_PROFILING=y CONFIG_MARKERS=y # CONFIG_OPROFILE is not set CONFIG_HAVE_OPROFILE=y CONFIG_KPROBES=y CONFIG_KRETPROBES=y CONFIG_HAVE_KPROBES=y CONFIG_HAVE_KRETPROBES=y CONFIG_PROC_PAGE_MONITOR=y CONFIG_SLABINFO=y CONFIG_RT_MUTEXES=y # CONFIG_TINY_SHMEM is not set CONFIG_BASE_SMALL=1 CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y CONFIG_MODULE_FORCE_UNLOAD=y # CONFIG_MODVERSIONS is not set # CONFIG_MODULE_SRCVERSION_ALL is not set # CONFIG_KMOD is not set CONFIG_STOP_MACHINE=y # CONFIG_BLOCK is not set CONFIG_CLASSIC_RCU=y # # Processor type and features # CONFIG_TICK_ONESHOT=y # CONFIG_NO_HZ is not set CONFIG_HIGH_RES_TIMERS=y CONFIG_GENERIC_CLOCKEVENTS_BUILD=y CONFIG_SMP=y # CONFIG_X86_PC is not set # CONFIG_X86_ELAN is not set # CONFIG_X86_VOYAGER is not set # CONFIG_X86_NUMAQ is not set # CONFIG_X86_SUMMIT is not set # CONFIG_X86_BIGSMP is not set # CONFIG_X86_VISWS is not set CONFIG_X86_GENERICARCH=y # CONFIG_X86_ES7000 is not set # CONFIG_X86_RDC321X is not set # CONFIG_X86_VSMP is not set # CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER is not set # CONFIG_PARAVIRT_GUEST is not set CONFIG_X86_CYCLONE_TIMER=y # CONFIG_M386 is not set # CONFIG_M486 is not set # CONFIG_M586 is not set # CONFIG_M586TSC is not set CONFIG_M586MMX=y # CONFIG_M686 is not set # CONFIG_MPENTIUMII is not set # CONFIG_MPENTIUMIII is not set # CONFIG_MPENTIUMM is not set # CONFIG_MPENTIUM4 is not set # CONFIG_MK6 is not set # CONFIG_MK7 is not set # CONFIG_MK8 is not set # CONFIG_MCRUSOE is not set # CONFIG_MEFFICEON is not set # CONFIG_MWINCHIPC6 is not set # CONFIG_MWINCHIP2 is not set # CONFIG_MWINCHIP3D is not set # CONFIG_MGEODEGX1 is not set # CONFIG_MGEODE_LX is not set # CONFIG_MCYRIXIII is not set # CONFIG_MVIAC3_2 is not set # CONFIG_MVIAC7 is not set # CONFIG_MPSC is not set # CONFIG_MCORE2 is not set # CONFIG_GENERIC_CPU is not set CONFIG_X86_GENERIC=y CONFIG_X86_CMPXCHG=y CONFIG_X86_L1_CACHE_SHIFT=7 CONFIG_X86_XADD=y # CONFIG_X86_PPRO_FENCE is not set CONFIG_X86_F00F_BUG=y CONFIG_X86_WP_WORKS_OK=y CONFIG_X86_INVLPG=y CONFIG_X86_BSWAP=y CONFIG_X86_POPAD_OK=y CONFIG_X86_ALIGNMENT_16=y CONFIG_X86_GOOD_APIC=y CONFIG_X86_INTEL_USERCOPY=y CONFIG_X86_TSC=y CONFIG_X86_MINIMUM_CPU_FAMILY=4 # CONFIG_HPET_TIMER is not set # CONFIG_IOMMU_HELPER is not set CONFIG_NR_CPUS=8 CONFIG_SCHED_SMT=y CONFIG_SCHED_MC=y CONFIG_PREEMPT_NONE=y # CONFIG_PREEMPT_VOLUNTARY is not set # CONFIG_PREEMPT is not set CONFIG_X86_LOCAL_APIC=y CONFIG_X86_IO_APIC=y CONFIG_X86_MCE=y # CONFIG_X86_MCE_NONFATAL is not set CONFIG_X86_MCE_P4THERMAL=y CONFIG_VM86=y # CONFIG_TOSHIBA is not set CONFIG_I8K=y # CONFIG_X86_REBOOTFIXUPS is not set CONFIG_MICROCODE=m CONFIG_MICROCODE_OLD_INTERFACE=y CONFIG_X86_MSR=m CONFIG_X86_CPUID=m CONFIG_NOHIGHMEM=y # CONFIG_HIGHMEM4G is not set # CONFIG_HIGHMEM64G is not set CONFIG_VMSPLIT_3G=y # CONFIG_VMSPLIT_3G_OPT is not set # CONFIG_VMSPLIT_2G is not set # CONFIG_VMSPLIT_2G_OPT is not set # CONFIG_VMSPLIT_1G is not set CONFIG_PAGE_OFFSET=0xC0000000 CONFIG_X86_PAE=y CONFIG_SELECT_MEMORY_MODEL=y CONFIG_FLATMEM_MANUAL=y # CONFIG_DISCONTIGMEM_MANUAL is not set # CONFIG_SPARSEMEM_MANUAL is not set CONFIG_FLATMEM=y CONFIG_FLAT_NODE_MEM_MAP=y # CONFIG_SPARSEMEM_STATIC is not set # CONFIG_SPARSEMEM_VMEMMAP_ENABLE is not set CONFIG_SPLIT_PTLOCK_CPUS=4 CONFIG_RESOURCES_64BIT=y CONFIG_ZONE_DMA_FLAG=1 CONFIG_VIRT_TO_BUS=y # CONFIG_MATH_EMULATION is not set CONFIG_MTRR=y # CONFIG_IRQBALANCE is not set # CONFIG_SECCOMP is not set # CONFIG_HZ_100 is not set CONFIG_HZ_250=y # CONFIG_HZ_300 is not set # CONFIG_HZ_1000 is not set CONFIG_HZ=250 CONFIG_SCHED_HRTICK=y CONFIG_KEXEC=y CONFIG_PHYSICAL_START=0x100000 CONFIG_RELOCATABLE=y CONFIG_PHYSICAL_ALIGN=0x100000 CONFIG_HOTPLUG_CPU=y # CONFIG_COMPAT_VDSO is not set # # Power management options # # CONFIG_PM is not set # # CPU Frequency scaling # # CONFIG_CPU_FREQ is not set # CONFIG_CPU_IDLE is not set # # Bus options (PCI etc.) # CONFIG_PCI=y # CONFIG_PCI_GOBIOS is not set # CONFIG_PCI_GOMMCONFIG is not set CONFIG_PCI_GODIRECT=y # CONFIG_PCI_GOANY is not set CONFIG_PCI_DIRECT=y CONFIG_PCI_DOMAINS=y # CONFIG_PCIEPORTBUS is not set CONFIG_ARCH_SUPPORTS_MSI=y # CONFIG_PCI_MSI is not set CONFIG_PCI_LEGACY=y # CONFIG_PCI_DEBUG is not set CONFIG_HT_IRQ=y CONFIG_ISA_DMA_API=y # CONFIG_ISA is not set # CONFIG_MCA is not set CONFIG_SCx200=y CONFIG_SCx200HR_TIMER=y CONFIG_PCCARD=y # CONFIG_PCMCIA_DEBUG is not set # CONFIG_PCMCIA is not set CONFIG_CARDBUS=y # # PC-card bridges # # CONFIG_YENTA is not set CONFIG_HOTPLUG_PCI=y # CONFIG_HOTPLUG_PCI_FAKE is not set # CONFIG_HOTPLUG_PCI_CPCI is not set CONFIG_HOTPLUG_PCI_SHPC=m # # Executable file formats / Emulations # # CONFIG_BINFMT_ELF is not set # CONFIG_BINFMT_AOUT is not set # CONFIG_BINFMT_MISC is not set # # Networking # # CONFIG_NET is not set # # Device Drivers # # # Generic Driver Options # CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" CONFIG_STANDALONE=y CONFIG_PREVENT_FIRMWARE_BUILD=y CONFIG_FW_LOADER=y CONFIG_DEBUG_DRIVER=y CONFIG_DEBUG_DEVRES=y # CONFIG_SYS_HYPERVISOR is not set # CONFIG_MTD is not set CONFIG_PARPORT=y # CONFIG_PARPORT_PC is not set # CONFIG_PARPORT_GSC is not set # CONFIG_PARPORT_AX88796 is not set CONFIG_PARPORT_1284=y CONFIG_PARPORT_NOT_PC=y CONFIG_MISC_DEVICES=y CONFIG_IBM_ASM=m CONFIG_PHANTOM=m # CONFIG_EEPROM_93CX6 is not set CONFIG_SGI_IOC4=y CONFIG_TIFM_CORE=m # CONFIG_TIFM_7XX1 is not set CONFIG_ENCLOSURE_SERVICES=m CONFIG_HAVE_IDE=y # # SCSI device support # # CONFIG_SCSI_DMA is not set # CONFIG_SCSI_NETLINK is not set CONFIG_FUSION=y CONFIG_FUSION_MAX_SGE=128 # CONFIG_FUSION_LOGGING is not set # # IEEE 1394 (FireWire) support # CONFIG_FIREWIRE=m # CONFIG_FIREWIRE_OHCI is not set CONFIG_IEEE1394=y # # Subsystem Options # # CONFIG_IEEE1394_VERBOSEDEBUG is not set # # Controllers # # # Texas Instruments PCILynx requires I2C # CONFIG_IEEE1394_OHCI1394=m # # Protocols # CONFIG_IEEE1394_VIDEO1394=m # # SBP-2 support (for storage devices) requires SCSI # # CONFIG_IEEE1394_ETH1394_ROM_ENTRY is not set CONFIG_IEEE1394_DV1394=m # CONFIG_IEEE1394_RAWIO is not set # CONFIG_I2O is not set CONFIG_MACINTOSH_DRIVERS=y CONFIG_MAC_EMUMOUSEBTN=y CONFIG_MLX4_CORE=m CONFIG_PHONE=y # CONFIG_PHONE_IXJ is not set # # Input device support # CONFIG_INPUT=y CONFIG_INPUT_FF_MEMLESS=m CONFIG_INPUT_POLLDEV=m # # Userland interfaces # CONFIG_INPUT_MOUSEDEV=y # CONFIG_INPUT_MOUSEDEV_PSAUX is not set CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024 CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768 CONFIG_INPUT_JOYDEV=y # CONFIG_INPUT_EVDEV is not set # CONFIG_INPUT_EVBUG is not set # # Input Device Drivers # # CONFIG_INPUT_KEYBOARD is not set # CONFIG_INPUT_MOUSE is not set CONFIG_INPUT_JOYSTICK=y CONFIG_JOYSTICK_ANALOG=m # CONFIG_JOYSTICK_A3D is not set CONFIG_JOYSTICK_ADI=y # CONFIG_JOYSTICK_COBRA is not set # CONFIG_JOYSTICK_GF2K is not set # CONFIG_JOYSTICK_GRIP is not set CONFIG_JOYSTICK_GRIP_MP=y # CONFIG_JOYSTICK_GUILLEMOT is not set # CONFIG_JOYSTICK_INTERACT is not set CONFIG_JOYSTICK_SIDEWINDER=y CONFIG_JOYSTICK_TMDC=y CONFIG_JOYSTICK_IFORCE=m # CONFIG_JOYSTICK_IFORCE_USB is not set CONFIG_JOYSTICK_IFORCE_232=y CONFIG_JOYSTICK_WARRIOR=y CONFIG_JOYSTICK_MAGELLAN=m # CONFIG_JOYSTICK_SPACEORB is not set CONFIG_JOYSTICK_SPACEBALL=m CONFIG_JOYSTICK_STINGER=y # CONFIG_JOYSTICK_TWIDJOY is not set CONFIG_JOYSTICK_DB9=m CONFIG_JOYSTICK_GAMECON=y CONFIG_JOYSTICK_TURBOGRAFX=m CONFIG_JOYSTICK_JOYDUMP=y CONFIG_JOYSTICK_XPAD=m CONFIG_JOYSTICK_XPAD_FF=y # CONFIG_JOYSTICK_XPAD_LEDS is not set # CONFIG_INPUT_TABLET is not set CONFIG_INPUT_TOUCHSCREEN=y CONFIG_TOUCHSCREEN_ADS7846=y # CONFIG_TOUCHSCREEN_FUJITSU is not set # CONFIG_TOUCHSCREEN_GUNZE is not set # CONFIG_TOUCHSCREEN_ELO is not set # CONFIG_TOUCHSCREEN_MTOUCH is not set CONFIG_TOUCHSCREEN_MK712=y CONFIG_TOUCHSCREEN_PENMOUNT=m # CONFIG_TOUCHSCREEN_TOUCHRIGHT is not set # CONFIG_TOUCHSCREEN_TOUCHWIN is not set CONFIG_TOUCHSCREEN_UCB1400=m CONFIG_TOUCHSCREEN_USB_COMPOSITE=m CONFIG_TOUCHSCREEN_USB_EGALAX=y CONFIG_TOUCHSCREEN_USB_PANJIT=y # CONFIG_TOUCHSCREEN_USB_3M is not set CONFIG_TOUCHSCREEN_USB_ITM=y CONFIG_TOUCHSCREEN_USB_ETURBO=y CONFIG_TOUCHSCREEN_USB_GUNZE=y # CONFIG_TOUCHSCREEN_USB_DMC_TSC10 is not set # CONFIG_TOUCHSCREEN_USB_IRTOUCH is not set CONFIG_TOUCHSCREEN_USB_IDEALTEK=y CONFIG_TOUCHSCREEN_USB_GENERAL_TOUCH=y # CONFIG_TOUCHSCREEN_USB_GOTOP is not set CONFIG_INPUT_MISC=y CONFIG_INPUT_PCSPKR=y CONFIG_INPUT_WISTRON_BTNS=m # CONFIG_INPUT_ATI_REMOTE is not set CONFIG_INPUT_ATI_REMOTE2=y CONFIG_INPUT_KEYSPAN_REMOTE=m # CONFIG_INPUT_POWERMATE is not set # CONFIG_INPUT_YEALINK is not set CONFIG_INPUT_UINPUT=y # # Hardware I/O ports # CONFIG_SERIO=y CONFIG_SERIO_I8042=y CONFIG_SERIO_SERPORT=m CONFIG_SERIO_CT82C710=y CONFIG_SERIO_PARKBD=y CONFIG_SERIO_PCIPS2=m CONFIG_SERIO_LIBPS2=m # CONFIG_SERIO_RAW is not set CONFIG_GAMEPORT=y # CONFIG_GAMEPORT_NS558 is not set # CONFIG_GAMEPORT_L4 is not set # CONFIG_GAMEPORT_EMU10K1 is not set CONFIG_GAMEPORT_FM801=y # # Character devices # # CONFIG_VT is not set CONFIG_SERIAL_NONSTANDARD=y CONFIG_COMPUTONE=m CONFIG_ROCKETPORT=m CONFIG_CYCLADES=y CONFIG_CYZ_INTR=y CONFIG_DIGIEPCA=y CONFIG_MOXA_INTELLIO=y CONFIG_MOXA_SMARTIO=y # CONFIG_ISI is not set # CONFIG_SYNCLINK is not set CONFIG_SYNCLINKMP=y CONFIG_SYNCLINK_GT=y CONFIG_N_HDLC=y CONFIG_RISCOM8=y CONFIG_SPECIALIX=y CONFIG_SPECIALIX_RTSCTS=y # CONFIG_SX is not set CONFIG_RIO=m CONFIG_RIO_OLDPCI=y # CONFIG_STALDRV is not set # CONFIG_NOZOMI is not set # # Serial drivers # CONFIG_SERIAL_8250=y # CONFIG_SERIAL_8250_CONSOLE is not set CONFIG_FIX_EARLYCON_MEM=y CONFIG_SERIAL_8250_PCI=m CONFIG_SERIAL_8250_NR_UARTS=4 CONFIG_SERIAL_8250_RUNTIME_UARTS=4 # CONFIG_SERIAL_8250_EXTENDED is not set # # Non-8250 serial port support # CONFIG_SERIAL_CORE=y CONFIG_SERIAL_JSM=m CONFIG_UNIX98_PTYS=y # CONFIG_LEGACY_PTYS is not set # CONFIG_PRINTER is not set CONFIG_PPDEV=y CONFIG_IPMI_HANDLER=y CONFIG_IPMI_PANIC_EVENT=y CONFIG_IPMI_PANIC_STRING=y CONFIG_IPMI_DEVICE_INTERFACE=m CONFIG_IPMI_SI=m CONFIG_IPMI_WATCHDOG=m # CONFIG_IPMI_POWEROFF is not set # CONFIG_HW_RANDOM is not set CONFIG_NVRAM=m CONFIG_RTC=y # CONFIG_R3964 is not set CONFIG_APPLICOM=m # CONFIG_SONYPI is not set CONFIG_MWAVE=y CONFIG_SCx200_GPIO=m CONFIG_PC8736x_GPIO=m CONFIG_NSC_GPIO=m CONFIG_CS5535_GPIO=y # CONFIG_HANGCHECK_TIMER is not set CONFIG_TCG_TPM=y # CONFIG_TCG_ATMEL is not set CONFIG_TELCLOCK=y CONFIG_DEVPORT=y # CONFIG_I2C is not set # # SPI support # CONFIG_SPI=y # CONFIG_SPI_DEBUG is not set CONFIG_SPI_MASTER=y # # SPI Master Controller Drivers # CONFIG_SPI_BITBANG=y # CONFIG_SPI_BUTTERFLY is not set CONFIG_SPI_LM70_LLP=y # # SPI Protocol Masters # # CONFIG_SPI_AT25 is not set CONFIG_SPI_SPIDEV=y CONFIG_SPI_TLE62X0=y # CONFIG_W1 is not set CONFIG_POWER_SUPPLY=m CONFIG_POWER_SUPPLY_DEBUG=y # CONFIG_PDA_POWER is not set # CONFIG_BATTERY_DS2760 is not set CONFIG_HWMON=y CONFIG_HWMON_VID=y # CONFIG_SENSORS_ABITUGURU is not set # CONFIG_SENSORS_ABITUGURU3 is not set # CONFIG_SENSORS_K8TEMP is not set CONFIG_SENSORS_I5K_AMB=m # CONFIG_SENSORS_F71805F is not set CONFIG_SENSORS_F71882FG=m CONFIG_SENSORS_CORETEMP=y # CONFIG_SENSORS_IBMPEX is not set CONFIG_SENSORS_IT87=m CONFIG_SENSORS_LM70=y CONFIG_SENSORS_PC87360=y # CONFIG_SENSORS_PC87427 is not set CONFIG_SENSORS_SIS5595=m # CONFIG_SENSORS_SMSC47M1 is not set CONFIG_SENSORS_SMSC47B397=y CONFIG_SENSORS_VIA686A=y CONFIG_SENSORS_VT1211=y # CONFIG_SENSORS_VT8231 is not set # CONFIG_SENSORS_W83627HF is not set CONFIG_SENSORS_W83627EHF=y # CONFIG_SENSORS_HDAPS is not set # CONFIG_SENSORS_APPLESMC is not set # CONFIG_HWMON_DEBUG_CHIP is not set # CONFIG_THERMAL is not set # CONFIG_WATCHDOG is not set # # Sonics Silicon Backplane # CONFIG_SSB_POSSIBLE=y # CONFIG_SSB is not set # # Multifunction device drivers # CONFIG_MFD_SM501=m # # Multimedia devices # CONFIG_VIDEO_DEV=m CONFIG_VIDEO_V4L2_COMMON=m CONFIG_VIDEO_V4L1=y CONFIG_VIDEO_V4L1_COMPAT=y CONFIG_VIDEO_V4L2=y CONFIG_VIDEO_CAPTURE_DRIVERS=y # CONFIG_VIDEO_ADV_DEBUG is not set CONFIG_VIDEO_HELPER_CHIPS_AUTO=y # CONFIG_VIDEO_VIVI is not set # CONFIG_VIDEO_BWQCAM is not set CONFIG_VIDEO_CQCAM=y CONFIG_VIDEO_W9966=m CONFIG_VIDEO_CPIA=y CONFIG_VIDEO_CPIA_PP=y # CONFIG_VIDEO_CPIA_USB is not set CONFIG_VIDEO_CPIA2=m # CONFIG_VIDEO_STRADIS is not set # CONFIG_V4L_USB_DRIVERS is not set # CONFIG_RADIO_ADAPTERS is not set # CONFIG_DAB is not set # # Graphics support # # CONFIG_AGP is not set CONFIG_DRM=m CONFIG_DRM_TDFX=m # CONFIG_DRM_R128 is not set CONFIG_DRM_RADEON=m # CONFIG_DRM_MGA is not set CONFIG_DRM_VIA=m CONFIG_DRM_SAVAGE=m # CONFIG_VGASTATE is not set # CONFIG_VIDEO_OUTPUT_CONTROL is not set # CONFIG_FB is not set CONFIG_BACKLIGHT_LCD_SUPPORT=y CONFIG_LCD_CLASS_DEVICE=m CONFIG_LCD_LTV350QV=m # CONFIG_BACKLIGHT_CLASS_DEVICE is not set # # Display device support # # CONFIG_DISPLAY_SUPPORT is not set # # Sound # CONFIG_SOUND=m # # Advanced Linux Sound Architecture # # CONFIG_SND is not set # # Open Sound System # CONFIG_SOUND_PRIME=m # CONFIG_SOUND_TRIDENT is not set # CONFIG_SOUND_MSNDCLAS is not set CONFIG_SOUND_MSNDPIN=m CONFIG_MSNDPIN_INIT_FILE="/etc/sound/pndspini.bin" CONFIG_MSNDPIN_PERM_FILE="/etc/sound/pndsperm.bin" CONFIG_SOUND_OSS=m CONFIG_SOUND_TRACEINIT=y CONFIG_SOUND_DMAP=y # CONFIG_SOUND_SSCAPE is not set # CONFIG_SOUND_VMIDI is not set CONFIG_SOUND_TRIX=m # CONFIG_SOUND_MSS is not set CONFIG_SOUND_MPU401=m CONFIG_SOUND_PAS=m CONFIG_SOUND_PSS=m CONFIG_PSS_MIXER=y CONFIG_SOUND_SB=m CONFIG_SOUND_YM3812=m # CONFIG_SOUND_UART6850 is not set CONFIG_SOUND_AEDSP16=m CONFIG_SC6600=y CONFIG_SC6600_JOY=y CONFIG_SC6600_CDROM=4 CONFIG_SC6600_CDROMBASE=0x0 # CONFIG_AEDSP16_MSS is not set # CONFIG_AEDSP16_SBPRO is not set # CONFIG_SOUND_KAHLUA is not set CONFIG_AC97_BUS=m # CONFIG_HID_SUPPORT is not set CONFIG_USB_SUPPORT=y CONFIG_USB_ARCH_HAS_HCD=y CONFIG_USB_ARCH_HAS_OHCI=y CONFIG_USB_ARCH_HAS_EHCI=y CONFIG_USB=y CONFIG_USB_DEBUG=y # CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set # # Miscellaneous USB options # # CONFIG_USB_DEVICEFS is not set CONFIG_USB_DEVICE_CLASS=y CONFIG_USB_DYNAMIC_MINORS=y # CONFIG_USB_OTG is not set # # USB Host Controller Drivers # CONFIG_USB_EHCI_HCD=y CONFIG_USB_EHCI_ROOT_HUB_TT=y CONFIG_USB_EHCI_TT_NEWSCHED=y CONFIG_USB_ISP116X_HCD=y # CONFIG_USB_OHCI_HCD is not set CONFIG_USB_UHCI_HCD=m CONFIG_USB_U132_HCD=m # CONFIG_USB_SL811_HCD is not set CONFIG_USB_R8A66597_HCD=y # # USB Device Class drivers # CONFIG_USB_ACM=m CONFIG_USB_PRINTER=y # # NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support' # # # may also be needed; see USB_STORAGE Help for more information # # CONFIG_USB_LIBUSUAL is not set # # USB Imaging devices # # CONFIG_USB_MDC800 is not set # CONFIG_USB_MON is not set # # USB port drivers # CONFIG_USB_USS720=y CONFIG_USB_SERIAL=m CONFIG_USB_EZUSB=y CONFIG_USB_SERIAL_GENERIC=y # CONFIG_USB_SERIAL_AIRCABLE is not set # CONFIG_USB_SERIAL_AIRPRIME is not set # CONFIG_USB_SERIAL_ARK3116 is not set # CONFIG_USB_SERIAL_BELKIN is not set # CONFIG_USB_SERIAL_CH341 is not set CONFIG_USB_SERIAL_WHITEHEAT=m CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m CONFIG_USB_SERIAL_CP2101=m CONFIG_USB_SERIAL_CYPRESS_M8=m CONFIG_USB_SERIAL_EMPEG=m CONFIG_USB_SERIAL_FTDI_SIO=m # CONFIG_USB_SERIAL_FUNSOFT is not set # CONFIG_USB_SERIAL_VISOR is not set CONFIG_USB_SERIAL_IPAQ=m # CONFIG_USB_SERIAL_IR is not set CONFIG_USB_SERIAL_EDGEPORT=m CONFIG_USB_SERIAL_EDGEPORT_TI=m CONFIG_USB_SERIAL_GARMIN=m # CONFIG_USB_SERIAL_IPW is not set CONFIG_USB_SERIAL_IUU=m CONFIG_USB_SERIAL_KEYSPAN_PDA=m CONFIG_USB_SERIAL_KEYSPAN=m # CONFIG_USB_SERIAL_KEYSPAN_MPR is not set CONFIG_USB_SERIAL_KEYSPAN_USA28=y CONFIG_USB_SERIAL_KEYSPAN_USA28X=y # CONFIG_USB_SERIAL_KEYSPAN_USA28XA is not set # CONFIG_USB_SERIAL_KEYSPAN_USA28XB is not set # CONFIG_USB_SERIAL_KEYSPAN_USA19 is not set CONFIG_USB_SERIAL_KEYSPAN_USA18X=y CONFIG_USB_SERIAL_KEYSPAN_USA19W=y # CONFIG_USB_SERIAL_KEYSPAN_USA19QW is not set CONFIG_USB_SERIAL_KEYSPAN_USA19QI=y CONFIG_USB_SERIAL_KEYSPAN_USA49W=y # CONFIG_USB_SERIAL_KEYSPAN_USA49WLC is not set CONFIG_USB_SERIAL_KLSI=m # CONFIG_USB_SERIAL_KOBIL_SCT is not set CONFIG_USB_SERIAL_MCT_U232=m # CONFIG_USB_SERIAL_MOS7720 is not set CONFIG_USB_SERIAL_MOS7840=m CONFIG_USB_SERIAL_NAVMAN=m # CONFIG_USB_SERIAL_PL2303 is not set CONFIG_USB_SERIAL_OTI6858=m CONFIG_USB_SERIAL_HP4X=m CONFIG_USB_SERIAL_SAFE=m # CONFIG_USB_SERIAL_SAFE_PADDED is not set # CONFIG_USB_SERIAL_SIERRAWIRELESS is not set CONFIG_USB_SERIAL_TI=m # CONFIG_USB_SERIAL_CYBERJACK is not set # CONFIG_USB_SERIAL_XIRCOM is not set CONFIG_USB_SERIAL_OPTION=m CONFIG_USB_SERIAL_OMNINET=m # CONFIG_USB_SERIAL_DEBUG is not set # # USB Miscellaneous drivers # CONFIG_USB_EMI62=y # CONFIG_USB_EMI26 is not set CONFIG_USB_ADUTUX=y CONFIG_USB_AUERSWALD=m CONFIG_USB_RIO500=m CONFIG_USB_LEGOTOWER=m CONFIG_USB_LCD=m # CONFIG_USB_BERRY_CHARGE is not set CONFIG_USB_LED=y # CONFIG_USB_CYPRESS_CY7C63 is not set CONFIG_USB_CYTHERM=m CONFIG_USB_PHIDGET=y CONFIG_USB_PHIDGETKIT=y # CONFIG_USB_PHIDGETMOTORCONTROL is not set CONFIG_USB_PHIDGETSERVO=y CONFIG_USB_IDMOUSE=y CONFIG_USB_FTDI_ELAN=m # CONFIG_USB_APPLEDISPLAY is not set CONFIG_USB_SISUSBVGA=m CONFIG_USB_LD=y # CONFIG_USB_TRANCEVIBRATOR is not set CONFIG_USB_IOWARRIOR=m # CONFIG_USB_GADGET is not set CONFIG_MMC=m CONFIG_MMC_DEBUG=y CONFIG_MMC_UNSAFE_RESUME=y # # MMC/SD Card Drivers # CONFIG_SDIO_UART=m # # MMC/SD Host Controller Drivers # # CONFIG_MMC_SDHCI is not set CONFIG_MMC_WBSD=m CONFIG_MMC_TIFM_SD=m CONFIG_MMC_SPI=m CONFIG_MEMSTICK=m CONFIG_MEMSTICK_DEBUG=y # # MemoryStick drivers # CONFIG_MEMSTICK_UNSAFE_RESUME=y # # MemoryStick Host Controller Drivers # CONFIG_MEMSTICK_TIFM_MS=m CONFIG_MEMSTICK_JMICRON_38X=m CONFIG_NEW_LEDS=y CONFIG_LEDS_CLASS=y # # LED drivers # CONFIG_LEDS_NET48XX=m # CONFIG_LEDS_WRAP is not set CONFIG_LEDS_CLEVO_MAIL=m # # LED Triggers # # CONFIG_LEDS_TRIGGERS is not set CONFIG_INFINIBAND=y # CONFIG_INFINIBAND_USER_MAD is not set CONFIG_INFINIBAND_USER_ACCESS=m CONFIG_INFINIBAND_USER_MEM=y # CONFIG_INFINIBAND_MTHCA is not set CONFIG_MLX4_INFINIBAND=m # CONFIG_EDAC is not set CONFIG_RTC_LIB=m CONFIG_RTC_CLASS=m # # Conflicting RTC option has been selected, check GEN_RTC and RTC # # # RTC interfaces # CONFIG_RTC_INTF_SYSFS=y CONFIG_RTC_INTF_PROC=y # CONFIG_RTC_INTF_DEV is not set CONFIG_RTC_DRV_TEST=m # # SPI RTC drivers # # CONFIG_RTC_DRV_MAX6902 is not set CONFIG_RTC_DRV_R9701=m CONFIG_RTC_DRV_RS5C348=m # # Platform RTC drivers # CONFIG_RTC_DRV_CMOS=m # CONFIG_RTC_DRV_DS1511 is not set # CONFIG_RTC_DRV_DS1553 is not set # CONFIG_RTC_DRV_DS1742 is not set CONFIG_RTC_DRV_STK17TA8=m CONFIG_RTC_DRV_M48T86=m # CONFIG_RTC_DRV_M48T59 is not set CONFIG_RTC_DRV_V3020=m # # on-CPU RTC drivers # # CONFIG_DMADEVICES is not set # CONFIG_AUXDISPLAY is not set # # Userspace I/O # CONFIG_UIO=m # CONFIG_UIO_CIF is not set # # Firmware Drivers # CONFIG_EDD=m # CONFIG_DELL_RBU is not set CONFIG_DCDBAS=m CONFIG_DMIID=y # # File systems # # CONFIG_DNOTIFY is not set # CONFIG_INOTIFY is not set CONFIG_QUOTA=y CONFIG_PRINT_QUOTA_WARNING=y CONFIG_QFMT_V1=m # CONFIG_QFMT_V2 is not set CONFIG_QUOTACTL=y CONFIG_AUTOFS_FS=m CONFIG_AUTOFS4_FS=m CONFIG_FUSE_FS=m # # Pseudo filesystems # CONFIG_PROC_FS=y # CONFIG_PROC_KCORE is not set # CONFIG_PROC_SYSCTL is not set CONFIG_SYSFS=y # CONFIG_TMPFS is not set # CONFIG_HUGETLBFS is not set # CONFIG_HUGETLB_PAGE is not set # CONFIG_CONFIGFS_FS is not set # # Miscellaneous filesystems # # CONFIG_NLS is not set # # Kernel hacking # CONFIG_TRACE_IRQFLAGS_SUPPORT=y CONFIG_ENABLE_WARN_DEPRECATED=y CONFIG_ENABLE_MUST_CHECK=y # CONFIG_MAGIC_SYSRQ is not set # CONFIG_UNUSED_SYMBOLS is not set CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_SHIRQ=y # CONFIG_DETECT_SOFTLOCKUP is not set # CONFIG_SCHED_DEBUG is not set CONFIG_SCHEDSTATS=y # CONFIG_TIMER_STATS is not set # CONFIG_SLUB_STATS is not set # CONFIG_DEBUG_RT_MUTEXES is not set CONFIG_RT_MUTEX_TESTER=y CONFIG_DEBUG_SPINLOCK=y CONFIG_DEBUG_MUTEXES=y CONFIG_DEBUG_LOCK_ALLOC=y CONFIG_PROVE_LOCKING=y CONFIG_LOCKDEP=y # CONFIG_LOCK_STAT is not set CONFIG_DEBUG_LOCKDEP=y CONFIG_TRACE_IRQFLAGS=y CONFIG_DEBUG_SPINLOCK_SLEEP=y CONFIG_DEBUG_LOCKING_API_SELFTESTS=y CONFIG_STACKTRACE=y # CONFIG_DEBUG_KOBJECT is not set # CONFIG_DEBUG_BUGVERBOSE is not set # CONFIG_DEBUG_INFO is not set CONFIG_DEBUG_VM=y CONFIG_DEBUG_LIST=y CONFIG_DEBUG_SG=y # CONFIG_FRAME_POINTER is not set CONFIG_RCU_TORTURE_TEST=m CONFIG_KPROBES_SANITY_TEST=y CONFIG_BACKTRACE_SELF_TEST=m # CONFIG_FAULT_INJECTION is not set # CONFIG_LATENCYTOP is not set CONFIG_PROVIDE_OHCI1394_DMA_INIT=y CONFIG_SAMPLES=y # CONFIG_SAMPLE_MARKERS is not set CONFIG_SAMPLE_KOBJECT=m # CONFIG_SAMPLE_KPROBES is not set # CONFIG_EARLY_PRINTK is not set # CONFIG_DEBUG_STACKOVERFLOW is not set CONFIG_DEBUG_STACK_USAGE=y # CONFIG_DEBUG_PAGEALLOC is not set # CONFIG_DEBUG_RODATA is not set # CONFIG_DEBUG_NX_TEST is not set # CONFIG_4KSTACKS is not set CONFIG_X86_FIND_SMP_CONFIG=y CONFIG_X86_MPPARSE=y CONFIG_DOUBLEFAULT=y CONFIG_IO_DELAY_TYPE_0X80=0 CONFIG_IO_DELAY_TYPE_0XED=1 CONFIG_IO_DELAY_TYPE_UDELAY=2 CONFIG_IO_DELAY_TYPE_NONE=3 # CONFIG_IO_DELAY_0X80 is not set # CONFIG_IO_DELAY_0XED is not set CONFIG_IO_DELAY_UDELAY=y # CONFIG_IO_DELAY_NONE is not set CONFIG_DEFAULT_IO_DELAY_TYPE=2 CONFIG_DEBUG_BOOT_PARAMS=y CONFIG_CPA_DEBUG=y # # Security options # # CONFIG_KEYS is not set # CONFIG_SECURITY is not set CONFIG_SECURITY_FILE_CAPABILITIES=y CONFIG_CRYPTO=y CONFIG_CRYPTO_ALGAPI=y CONFIG_CRYPTO_AEAD=y CONFIG_CRYPTO_BLKCIPHER=y CONFIG_CRYPTO_SEQIV=y CONFIG_CRYPTO_HASH=y CONFIG_CRYPTO_MANAGER=y CONFIG_CRYPTO_HMAC=y # CONFIG_CRYPTO_XCBC is not set # CONFIG_CRYPTO_NULL is not set CONFIG_CRYPTO_MD4=y # CONFIG_CRYPTO_MD5 is not set CONFIG_CRYPTO_SHA1=y # CONFIG_CRYPTO_SHA256 is not set CONFIG_CRYPTO_SHA512=y # CONFIG_CRYPTO_WP512 is not set CONFIG_CRYPTO_TGR192=m CONFIG_CRYPTO_GF128MUL=y CONFIG_CRYPTO_ECB=m CONFIG_CRYPTO_CBC=m # CONFIG_CRYPTO_PCBC is not set CONFIG_CRYPTO_LRW=y # CONFIG_CRYPTO_XTS is not set CONFIG_CRYPTO_CTR=y CONFIG_CRYPTO_GCM=y # CONFIG_CRYPTO_CCM is not set CONFIG_CRYPTO_CRYPTD=m CONFIG_CRYPTO_DES=m # CONFIG_CRYPTO_FCRYPT is not set # CONFIG_CRYPTO_BLOWFISH is not set # CONFIG_CRYPTO_TWOFISH is not set # CONFIG_CRYPTO_TWOFISH_586 is not set CONFIG_CRYPTO_SERPENT=m CONFIG_CRYPTO_AES=m # CONFIG_CRYPTO_AES_586 is not set CONFIG_CRYPTO_CAST5=m CONFIG_CRYPTO_CAST6=y CONFIG_CRYPTO_TEA=y # CONFIG_CRYPTO_ARC4 is not set CONFIG_CRYPTO_KHAZAD=y CONFIG_CRYPTO_ANUBIS=y CONFIG_CRYPTO_SEED=y CONFIG_CRYPTO_SALSA20=y # CONFIG_CRYPTO_SALSA20_586 is not set # CONFIG_CRYPTO_DEFLATE is not set # CONFIG_CRYPTO_MICHAEL_MIC is not set # CONFIG_CRYPTO_CRC32C is not set CONFIG_CRYPTO_CAMELLIA=y # CONFIG_CRYPTO_TEST is not set CONFIG_CRYPTO_AUTHENC=y # CONFIG_CRYPTO_LZO is not set # CONFIG_CRYPTO_HW is not set CONFIG_HAVE_KVM=y # CONFIG_VIRTUALIZATION is not set # # Library routines # CONFIG_BITREVERSE=m # CONFIG_CRC_CCITT is not set CONFIG_CRC16=y CONFIG_CRC_ITU_T=y CONFIG_CRC32=m CONFIG_CRC7=m # CONFIG_LIBCRC32C is not set CONFIG_PLIST=y CONFIG_HAS_IOMEM=y CONFIG_HAS_IOPORT=y CONFIG_HAS_DMA=y CONFIG_CHECK_SIGNATURE=y ------------------------------------------------------- -- MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3 -------------- next part -------------- Hello, the build with the attached .config failed, make ends with: ... WARNING: modpost: Found 16 section mismatch(es). To see full details build your kernel with: 'make CONFIG_DEBUG_SECTION_MISMATCH=y' GEN .version CHK include/linux/compile.h UPD include/linux/compile.h CC init/version.o LD init/built-in.o LD .tmp_vmlinux1 drivers/built-in.o: In function `qcam_ioctl': c-qcam.c:(.text+0x4aed5): undefined reference to `video_usercopy' drivers/built-in.o: In function `qcam_read': c-qcam.c:(.text+0x4b5f7): undefined reference to `video_devdata' drivers/built-in.o: In function `cq_attach': c-qcam.c:(.text+0x4bb52): undefined reference to `video_register_device' drivers/built-in.o: In function `qcam_do_ioctl': c-qcam.c:(.text+0x4bc93): undefined reference to `video_devdata' drivers/built-in.o: In function `cpia_unregister_camera': (.text+0x4c8e4): undefined reference to `video_unregister_device' drivers/built-in.o: In function `cpia_ioctl': cpia.c:(.text+0x4d8c5): undefined reference to `video_usercopy' drivers/built-in.o: In function `cpia_open': cpia.c:(.text+0x50c58): undefined reference to `video_devdata' drivers/built-in.o: In function `cpia_register_camera': (.text+0x51146): undefined reference to `video_register_device' drivers/built-in.o: In function `cqcam_cleanup': c-qcam.c:(.exit.text+0x61b): undefined reference to `video_unregister_device' drivers/built-in.o:(.rodata+0x2bc8): undefined reference to `v4l_compat_ioctl32' drivers/built-in.o:(.rodata+0x2bd0): undefined reference to `video_exclusive_open' drivers/built-in.o:(.rodata+0x2bd8): undefined reference to `video_exclusive_release' drivers/built-in.o:(.rodata+0x2d48): undefined reference to `v4l_compat_ioctl32' make: *** [.tmp_vmlinux1] Error 1 The build was made with : $> make mrproper && make randconfig && make Here's the config: # # Automatically generated make config: don't edit # Linux kernel version: 2.6.25-rc5 # Fri Mar 14 16:16:49 2008 # # CONFIG_64BIT is not set CONFIG_X86_32=y # CONFIG_X86_64 is not set CONFIG_X86=y # CONFIG_GENERIC_LOCKBREAK is not set CONFIG_GENERIC_TIME=y CONFIG_GENERIC_CMOS_UPDATE=y CONFIG_CLOCKSOURCE_WATCHDOG=y CONFIG_GENERIC_CLOCKEVENTS=y CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y CONFIG_LOCKDEP_SUPPORT=y CONFIG_STACKTRACE_SUPPORT=y CONFIG_HAVE_LATENCYTOP_SUPPORT=y CONFIG_SEMAPHORE_SLEEPERS=y CONFIG_FAST_CMPXCHG_LOCAL=y CONFIG_MMU=y CONFIG_ZONE_DMA=y CONFIG_GENERIC_ISA_DMA=y CONFIG_GENERIC_IOMAP=y CONFIG_GENERIC_BUG=y CONFIG_GENERIC_HWEIGHT=y # CONFIG_GENERIC_GPIO is not set CONFIG_ARCH_MAY_HAVE_PC_FDC=y CONFIG_DMI=y # CONFIG_RWSEM_GENERIC_SPINLOCK is not set CONFIG_RWSEM_XCHGADD_ALGORITHM=y # CONFIG_ARCH_HAS_ILOG2_U32 is not set # CONFIG_ARCH_HAS_ILOG2_U64 is not set CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y CONFIG_GENERIC_CALIBRATE_DELAY=y # CONFIG_GENERIC_TIME_VSYSCALL is not set CONFIG_ARCH_HAS_CPU_RELAX=y # CONFIG_HAVE_SETUP_PER_CPU_AREA is not set CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_SUSPEND_POSSIBLE=y # CONFIG_ZONE_DMA32 is not set CONFIG_ARCH_POPULATES_NODE_MAP=y # CONFIG_AUDIT_ARCH is not set CONFIG_ARCH_SUPPORTS_AOUT=y CONFIG_GENERIC_HARDIRQS=y CONFIG_GENERIC_IRQ_PROBE=y CONFIG_GENERIC_PENDING_IRQ=y CONFIG_X86_SMP=y CONFIG_X86_32_SMP=y CONFIG_X86_HT=y CONFIG_X86_BIOS_REBOOT=y CONFIG_X86_TRAMPOLINE=y CONFIG_KTIME_SCALAR=y CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config" # # General setup # CONFIG_EXPERIMENTAL=y CONFIG_LOCK_KERNEL=y CONFIG_INIT_ENV_ARG_LIMIT=32 CONFIG_LOCALVERSION="" # CONFIG_LOCALVERSION_AUTO is not set CONFIG_SYSVIPC=y CONFIG_SYSVIPC_SYSCTL=y CONFIG_BSD_PROCESS_ACCT=y # CONFIG_BSD_PROCESS_ACCT_V3 is not set # CONFIG_IKCONFIG is not set CONFIG_LOG_BUF_SHIFT=14 CONFIG_CGROUPS=y CONFIG_CGROUP_DEBUG=y CONFIG_CGROUP_NS=y CONFIG_CPUSETS=y # CONFIG_GROUP_SCHED is not set CONFIG_CGROUP_CPUACCT=y # CONFIG_RESOURCE_COUNTERS is not set CONFIG_SYSFS_DEPRECATED=y CONFIG_SYSFS_DEPRECATED_V2=y CONFIG_PROC_PID_CPUSET=y # CONFIG_RELAY is not set CONFIG_NAMESPACES=y CONFIG_UTS_NS=y # CONFIG_IPC_NS is not set CONFIG_USER_NS=y CONFIG_PID_NS=y # CONFIG_BLK_DEV_INITRD is not set # CONFIG_CC_OPTIMIZE_FOR_SIZE is not set CONFIG_SYSCTL=y CONFIG_EMBEDDED=y # CONFIG_UID16 is not set CONFIG_SYSCTL_SYSCALL=y CONFIG_KALLSYMS=y CONFIG_KALLSYMS_ALL=y # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_HOTPLUG=y # CONFIG_PRINTK is not set CONFIG_BUG=y CONFIG_ELF_CORE=y # CONFIG_COMPAT_BRK is not set # CONFIG_BASE_FULL is not set CONFIG_FUTEX=y CONFIG_ANON_INODES=y CONFIG_EPOLL=y CONFIG_SIGNALFD=y CONFIG_TIMERFD=y # CONFIG_EVENTFD is not set CONFIG_SHMEM=y # CONFIG_VM_EVENT_COUNTERS is not set # CONFIG_SLUB_DEBUG is not set # CONFIG_SLAB is not set CONFIG_SLUB=y # CONFIG_SLOB is not set CONFIG_PROFILING=y CONFIG_MARKERS=y # CONFIG_OPROFILE is not set CONFIG_HAVE_OPROFILE=y CONFIG_KPROBES=y CONFIG_KRETPROBES=y CONFIG_HAVE_KPROBES=y CONFIG_HAVE_KRETPROBES=y CONFIG_PROC_PAGE_MONITOR=y CONFIG_SLABINFO=y CONFIG_RT_MUTEXES=y # CONFIG_TINY_SHMEM is not set CONFIG_BASE_SMALL=1 CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y CONFIG_MODULE_FORCE_UNLOAD=y # CONFIG_MODVERSIONS is not set # CONFIG_MODULE_SRCVERSION_ALL is not set # CONFIG_KMOD is not set CONFIG_STOP_MACHINE=y # CONFIG_BLOCK is not set CONFIG_CLASSIC_RCU=y # # Processor type and features # CONFIG_TICK_ONESHOT=y # CONFIG_NO_HZ is not set CONFIG_HIGH_RES_TIMERS=y CONFIG_GENERIC_CLOCKEVENTS_BUILD=y CONFIG_SMP=y # CONFIG_X86_PC is not set # CONFIG_X86_ELAN is not set # CONFIG_X86_VOYAGER is not set # CONFIG_X86_NUMAQ is not set # CONFIG_X86_SUMMIT is not set # CONFIG_X86_BIGSMP is not set # CONFIG_X86_VISWS is not set CONFIG_X86_GENERICARCH=y # CONFIG_X86_ES7000 is not set # CONFIG_X86_RDC321X is not set # CONFIG_X86_VSMP is not set # CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER is not set # CONFIG_PARAVIRT_GUEST is not set CONFIG_X86_CYCLONE_TIMER=y # CONFIG_M386 is not set # CONFIG_M486 is not set # CONFIG_M586 is not set # CONFIG_M586TSC is not set CONFIG_M586MMX=y # CONFIG_M686 is not set # CONFIG_MPENTIUMII is not set # CONFIG_MPENTIUMIII is not set # CONFIG_MPENTIUMM is not set # CONFIG_MPENTIUM4 is not set # CONFIG_MK6 is not set # CONFIG_MK7 is not set # CONFIG_MK8 is not set # CONFIG_MCRUSOE is not set # CONFIG_MEFFICEON is not set # CONFIG_MWINCHIPC6 is not set # CONFIG_MWINCHIP2 is not set # CONFIG_MWINCHIP3D is not set # CONFIG_MGEODEGX1 is not set # CONFIG_MGEODE_LX is not set # CONFIG_MCYRIXIII is not set # CONFIG_MVIAC3_2 is not set # CONFIG_MVIAC7 is not set # CONFIG_MPSC is not set # CONFIG_MCORE2 is not set # CONFIG_GENERIC_CPU is not set CONFIG_X86_GENERIC=y CONFIG_X86_CMPXCHG=y CONFIG_X86_L1_CACHE_SHIFT=7 CONFIG_X86_XADD=y # CONFIG_X86_PPRO_FENCE is not set CONFIG_X86_F00F_BUG=y CONFIG_X86_WP_WORKS_OK=y CONFIG_X86_INVLPG=y CONFIG_X86_BSWAP=y CONFIG_X86_POPAD_OK=y CONFIG_X86_ALIGNMENT_16=y CONFIG_X86_GOOD_APIC=y CONFIG_X86_INTEL_USERCOPY=y CONFIG_X86_TSC=y CONFIG_X86_MINIMUM_CPU_FAMILY=4 # CONFIG_HPET_TIMER is not set # CONFIG_IOMMU_HELPER is not set CONFIG_NR_CPUS=8 CONFIG_SCHED_SMT=y CONFIG_SCHED_MC=y CONFIG_PREEMPT_NONE=y # CONFIG_PREEMPT_VOLUNTARY is not set # CONFIG_PREEMPT is not set CONFIG_X86_LOCAL_APIC=y CONFIG_X86_IO_APIC=y CONFIG_X86_MCE=y # CONFIG_X86_MCE_NONFATAL is not set CONFIG_X86_MCE_P4THERMAL=y CONFIG_VM86=y # CONFIG_TOSHIBA is not set CONFIG_I8K=y # CONFIG_X86_REBOOTFIXUPS is not set CONFIG_MICROCODE=m CONFIG_MICROCODE_OLD_INTERFACE=y CONFIG_X86_MSR=m CONFIG_X86_CPUID=m CONFIG_NOHIGHMEM=y # CONFIG_HIGHMEM4G is not set # CONFIG_HIGHMEM64G is not set CONFIG_VMSPLIT_3G=y # CONFIG_VMSPLIT_3G_OPT is not set # CONFIG_VMSPLIT_2G is not set # CONFIG_VMSPLIT_2G_OPT is not set # CONFIG_VMSPLIT_1G is not set CONFIG_PAGE_OFFSET=0xC0000000 CONFIG_X86_PAE=y CONFIG_SELECT_MEMORY_MODEL=y CONFIG_FLATMEM_MANUAL=y # CONFIG_DISCONTIGMEM_MANUAL is not set # CONFIG_SPARSEMEM_MANUAL is not set CONFIG_FLATMEM=y CONFIG_FLAT_NODE_MEM_MAP=y # CONFIG_SPARSEMEM_STATIC is not set # CONFIG_SPARSEMEM_VMEMMAP_ENABLE is not set CONFIG_SPLIT_PTLOCK_CPUS=4 CONFIG_RESOURCES_64BIT=y CONFIG_ZONE_DMA_FLAG=1 CONFIG_VIRT_TO_BUS=y # CONFIG_MATH_EMULATION is not set CONFIG_MTRR=y # CONFIG_IRQBALANCE is not set # CONFIG_SECCOMP is not set # CONFIG_HZ_100 is not set CONFIG_HZ_250=y # CONFIG_HZ_300 is not set # CONFIG_HZ_1000 is not set CONFIG_HZ=250 CONFIG_SCHED_HRTICK=y CONFIG_KEXEC=y CONFIG_PHYSICAL_START=0x100000 CONFIG_RELOCATABLE=y CONFIG_PHYSICAL_ALIGN=0x100000 CONFIG_HOTPLUG_CPU=y # CONFIG_COMPAT_VDSO is not set # # Power management options # # CONFIG_PM is not set # # CPU Frequency scaling # # CONFIG_CPU_FREQ is not set # CONFIG_CPU_IDLE is not set # # Bus options (PCI etc.) # CONFIG_PCI=y # CONFIG_PCI_GOBIOS is not set # CONFIG_PCI_GOMMCONFIG is not set CONFIG_PCI_GODIRECT=y # CONFIG_PCI_GOANY is not set CONFIG_PCI_DIRECT=y CONFIG_PCI_DOMAINS=y # CONFIG_PCIEPORTBUS is not set CONFIG_ARCH_SUPPORTS_MSI=y # CONFIG_PCI_MSI is not set CONFIG_PCI_LEGACY=y # CONFIG_PCI_DEBUG is not set CONFIG_HT_IRQ=y CONFIG_ISA_DMA_API=y # CONFIG_ISA is not set # CONFIG_MCA is not set CONFIG_SCx200=y CONFIG_SCx200HR_TIMER=y CONFIG_PCCARD=y # CONFIG_PCMCIA_DEBUG is not set # CONFIG_PCMCIA is not set CONFIG_CARDBUS=y # # PC-card bridges # # CONFIG_YENTA is not set CONFIG_HOTPLUG_PCI=y # CONFIG_HOTPLUG_PCI_FAKE is not set # CONFIG_HOTPLUG_PCI_CPCI is not set CONFIG_HOTPLUG_PCI_SHPC=m # # Executable file formats / Emulations # # CONFIG_BINFMT_ELF is not set # CONFIG_BINFMT_AOUT is not set # CONFIG_BINFMT_MISC is not set # # Networking # # CONFIG_NET is not set # # Device Drivers # # # Generic Driver Options # CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" CONFIG_STANDALONE=y CONFIG_PREVENT_FIRMWARE_BUILD=y CONFIG_FW_LOADER=y CONFIG_DEBUG_DRIVER=y CONFIG_DEBUG_DEVRES=y # CONFIG_SYS_HYPERVISOR is not set # CONFIG_MTD is not set CONFIG_PARPORT=y # CONFIG_PARPORT_PC is not set # CONFIG_PARPORT_GSC is not set # CONFIG_PARPORT_AX88796 is not set CONFIG_PARPORT_1284=y CONFIG_PARPORT_NOT_PC=y CONFIG_MISC_DEVICES=y CONFIG_IBM_ASM=m CONFIG_PHANTOM=m # CONFIG_EEPROM_93CX6 is not set CONFIG_SGI_IOC4=y CONFIG_TIFM_CORE=m # CONFIG_TIFM_7XX1 is not set CONFIG_ENCLOSURE_SERVICES=m CONFIG_HAVE_IDE=y # # SCSI device support # # CONFIG_SCSI_DMA is not set # CONFIG_SCSI_NETLINK is not set CONFIG_FUSION=y CONFIG_FUSION_MAX_SGE=128 # CONFIG_FUSION_LOGGING is not set # # IEEE 1394 (FireWire) support # CONFIG_FIREWIRE=m # CONFIG_FIREWIRE_OHCI is not set CONFIG_IEEE1394=y # # Subsystem Options # # CONFIG_IEEE1394_VERBOSEDEBUG is not set # # Controllers # # # Texas Instruments PCILynx requires I2C # CONFIG_IEEE1394_OHCI1394=m # # Protocols # CONFIG_IEEE1394_VIDEO1394=m # # SBP-2 support (for storage devices) requires SCSI # # CONFIG_IEEE1394_ETH1394_ROM_ENTRY is not set CONFIG_IEEE1394_DV1394=m # CONFIG_IEEE1394_RAWIO is not set # CONFIG_I2O is not set CONFIG_MACINTOSH_DRIVERS=y CONFIG_MAC_EMUMOUSEBTN=y CONFIG_MLX4_CORE=m CONFIG_PHONE=y # CONFIG_PHONE_IXJ is not set # # Input device support # CONFIG_INPUT=y CONFIG_INPUT_FF_MEMLESS=m CONFIG_INPUT_POLLDEV=m # # Userland interfaces # CONFIG_INPUT_MOUSEDEV=y # CONFIG_INPUT_MOUSEDEV_PSAUX is not set CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024 CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768 CONFIG_INPUT_JOYDEV=y # CONFIG_INPUT_EVDEV is not set # CONFIG_INPUT_EVBUG is not set # # Input Device Drivers # # CONFIG_INPUT_KEYBOARD is not set # CONFIG_INPUT_MOUSE is not set CONFIG_INPUT_JOYSTICK=y CONFIG_JOYSTICK_ANALOG=m # CONFIG_JOYSTICK_A3D is not set CONFIG_JOYSTICK_ADI=y # CONFIG_JOYSTICK_COBRA is not set # CONFIG_JOYSTICK_GF2K is not set # CONFIG_JOYSTICK_GRIP is not set CONFIG_JOYSTICK_GRIP_MP=y # CONFIG_JOYSTICK_GUILLEMOT is not set # CONFIG_JOYSTICK_INTERACT is not set CONFIG_JOYSTICK_SIDEWINDER=y CONFIG_JOYSTICK_TMDC=y CONFIG_JOYSTICK_IFORCE=m # CONFIG_JOYSTICK_IFORCE_USB is not set CONFIG_JOYSTICK_IFORCE_232=y CONFIG_JOYSTICK_WARRIOR=y CONFIG_JOYSTICK_MAGELLAN=m # CONFIG_JOYSTICK_SPACEORB is not set CONFIG_JOYSTICK_SPACEBALL=m CONFIG_JOYSTICK_STINGER=y # CONFIG_JOYSTICK_TWIDJOY is not set CONFIG_JOYSTICK_DB9=m CONFIG_JOYSTICK_GAMECON=y CONFIG_JOYSTICK_TURBOGRAFX=m CONFIG_JOYSTICK_JOYDUMP=y CONFIG_JOYSTICK_XPAD=m CONFIG_JOYSTICK_XPAD_FF=y # CONFIG_JOYSTICK_XPAD_LEDS is not set # CONFIG_INPUT_TABLET is not set CONFIG_INPUT_TOUCHSCREEN=y CONFIG_TOUCHSCREEN_ADS7846=y # CONFIG_TOUCHSCREEN_FUJITSU is not set # CONFIG_TOUCHSCREEN_GUNZE is not set # CONFIG_TOUCHSCREEN_ELO is not set # CONFIG_TOUCHSCREEN_MTOUCH is not set CONFIG_TOUCHSCREEN_MK712=y CONFIG_TOUCHSCREEN_PENMOUNT=m # CONFIG_TOUCHSCREEN_TOUCHRIGHT is not set # CONFIG_TOUCHSCREEN_TOUCHWIN is not set CONFIG_TOUCHSCREEN_UCB1400=m CONFIG_TOUCHSCREEN_USB_COMPOSITE=m CONFIG_TOUCHSCREEN_USB_EGALAX=y CONFIG_TOUCHSCREEN_USB_PANJIT=y # CONFIG_TOUCHSCREEN_USB_3M is not set CONFIG_TOUCHSCREEN_USB_ITM=y CONFIG_TOUCHSCREEN_USB_ETURBO=y CONFIG_TOUCHSCREEN_USB_GUNZE=y # CONFIG_TOUCHSCREEN_USB_DMC_TSC10 is not set # CONFIG_TOUCHSCREEN_USB_IRTOUCH is not set CONFIG_TOUCHSCREEN_USB_IDEALTEK=y CONFIG_TOUCHSCREEN_USB_GENERAL_TOUCH=y # CONFIG_TOUCHSCREEN_USB_GOTOP is not set CONFIG_INPUT_MISC=y CONFIG_INPUT_PCSPKR=y CONFIG_INPUT_WISTRON_BTNS=m # CONFIG_INPUT_ATI_REMOTE is not set CONFIG_INPUT_ATI_REMOTE2=y CONFIG_INPUT_KEYSPAN_REMOTE=m # CONFIG_INPUT_POWERMATE is not set # CONFIG_INPUT_YEALINK is not set CONFIG_INPUT_UINPUT=y # # Hardware I/O ports # CONFIG_SERIO=y CONFIG_SERIO_I8042=y CONFIG_SERIO_SERPORT=m CONFIG_SERIO_CT82C710=y CONFIG_SERIO_PARKBD=y CONFIG_SERIO_PCIPS2=m CONFIG_SERIO_LIBPS2=m # CONFIG_SERIO_RAW is not set CONFIG_GAMEPORT=y # CONFIG_GAMEPORT_NS558 is not set # CONFIG_GAMEPORT_L4 is not set # CONFIG_GAMEPORT_EMU10K1 is not set CONFIG_GAMEPORT_FM801=y # # Character devices # # CONFIG_VT is not set CONFIG_SERIAL_NONSTANDARD=y CONFIG_COMPUTONE=m CONFIG_ROCKETPORT=m CONFIG_CYCLADES=y CONFIG_CYZ_INTR=y CONFIG_DIGIEPCA=y CONFIG_MOXA_INTELLIO=y CONFIG_MOXA_SMARTIO=y # CONFIG_ISI is not set # CONFIG_SYNCLINK is not set CONFIG_SYNCLINKMP=y CONFIG_SYNCLINK_GT=y CONFIG_N_HDLC=y CONFIG_RISCOM8=y CONFIG_SPECIALIX=y CONFIG_SPECIALIX_RTSCTS=y # CONFIG_SX is not set CONFIG_RIO=m CONFIG_RIO_OLDPCI=y # CONFIG_STALDRV is not set # CONFIG_NOZOMI is not set # # Serial drivers # CONFIG_SERIAL_8250=y # CONFIG_SERIAL_8250_CONSOLE is not set CONFIG_FIX_EARLYCON_MEM=y CONFIG_SERIAL_8250_PCI=m CONFIG_SERIAL_8250_NR_UARTS=4 CONFIG_SERIAL_8250_RUNTIME_UARTS=4 # CONFIG_SERIAL_8250_EXTENDED is not set # # Non-8250 serial port support # CONFIG_SERIAL_CORE=y CONFIG_SERIAL_JSM=m CONFIG_UNIX98_PTYS=y # CONFIG_LEGACY_PTYS is not set # CONFIG_PRINTER is not set CONFIG_PPDEV=y CONFIG_IPMI_HANDLER=y CONFIG_IPMI_PANIC_EVENT=y CONFIG_IPMI_PANIC_STRING=y CONFIG_IPMI_DEVICE_INTERFACE=m CONFIG_IPMI_SI=m CONFIG_IPMI_WATCHDOG=m # CONFIG_IPMI_POWEROFF is not set # CONFIG_HW_RANDOM is not set CONFIG_NVRAM=m CONFIG_RTC=y # CONFIG_R3964 is not set CONFIG_APPLICOM=m # CONFIG_SONYPI is not set CONFIG_MWAVE=y CONFIG_SCx200_GPIO=m CONFIG_PC8736x_GPIO=m CONFIG_NSC_GPIO=m CONFIG_CS5535_GPIO=y # CONFIG_HANGCHECK_TIMER is not set CONFIG_TCG_TPM=y # CONFIG_TCG_ATMEL is not set CONFIG_TELCLOCK=y CONFIG_DEVPORT=y # CONFIG_I2C is not set # # SPI support # CONFIG_SPI=y # CONFIG_SPI_DEBUG is not set CONFIG_SPI_MASTER=y # # SPI Master Controller Drivers # CONFIG_SPI_BITBANG=y # CONFIG_SPI_BUTTERFLY is not set CONFIG_SPI_LM70_LLP=y # # SPI Protocol Masters # # CONFIG_SPI_AT25 is not set CONFIG_SPI_SPIDEV=y CONFIG_SPI_TLE62X0=y # CONFIG_W1 is not set CONFIG_POWER_SUPPLY=m CONFIG_POWER_SUPPLY_DEBUG=y # CONFIG_PDA_POWER is not set # CONFIG_BATTERY_DS2760 is not set CONFIG_HWMON=y CONFIG_HWMON_VID=y # CONFIG_SENSORS_ABITUGURU is not set # CONFIG_SENSORS_ABITUGURU3 is not set # CONFIG_SENSORS_K8TEMP is not set CONFIG_SENSORS_I5K_AMB=m # CONFIG_SENSORS_F71805F is not set CONFIG_SENSORS_F71882FG=m CONFIG_SENSORS_CORETEMP=y # CONFIG_SENSORS_IBMPEX is not set CONFIG_SENSORS_IT87=m CONFIG_SENSORS_LM70=y CONFIG_SENSORS_PC87360=y # CONFIG_SENSORS_PC87427 is not set CONFIG_SENSORS_SIS5595=m # CONFIG_SENSORS_SMSC47M1 is not set CONFIG_SENSORS_SMSC47B397=y CONFIG_SENSORS_VIA686A=y CONFIG_SENSORS_VT1211=y # CONFIG_SENSORS_VT8231 is not set # CONFIG_SENSORS_W83627HF is not set CONFIG_SENSORS_W83627EHF=y # CONFIG_SENSORS_HDAPS is not set # CONFIG_SENSORS_APPLESMC is not set # CONFIG_HWMON_DEBUG_CHIP is not set # CONFIG_THERMAL is not set # CONFIG_WATCHDOG is not set # # Sonics Silicon Backplane # CONFIG_SSB_POSSIBLE=y # CONFIG_SSB is not set # # Multifunction device drivers # CONFIG_MFD_SM501=m # # Multimedia devices # CONFIG_VIDEO_DEV=m CONFIG_VIDEO_V4L2_COMMON=m CONFIG_VIDEO_V4L1=y CONFIG_VIDEO_V4L1_COMPAT=y CONFIG_VIDEO_V4L2=y CONFIG_VIDEO_CAPTURE_DRIVERS=y # CONFIG_VIDEO_ADV_DEBUG is not set CONFIG_VIDEO_HELPER_CHIPS_AUTO=y # CONFIG_VIDEO_VIVI is not set # CONFIG_VIDEO_BWQCAM is not set CONFIG_VIDEO_CQCAM=y CONFIG_VIDEO_W9966=m CONFIG_VIDEO_CPIA=y CONFIG_VIDEO_CPIA_PP=y # CONFIG_VIDEO_CPIA_USB is not set CONFIG_VIDEO_CPIA2=m # CONFIG_VIDEO_STRADIS is not set # CONFIG_V4L_USB_DRIVERS is not set # CONFIG_RADIO_ADAPTERS is not set # CONFIG_DAB is not set # # Graphics support # # CONFIG_AGP is not set CONFIG_DRM=m CONFIG_DRM_TDFX=m # CONFIG_DRM_R128 is not set CONFIG_DRM_RADEON=m # CONFIG_DRM_MGA is not set CONFIG_DRM_VIA=m CONFIG_DRM_SAVAGE=m # CONFIG_VGASTATE is not set # CONFIG_VIDEO_OUTPUT_CONTROL is not set # CONFIG_FB is not set CONFIG_BACKLIGHT_LCD_SUPPORT=y CONFIG_LCD_CLASS_DEVICE=m CONFIG_LCD_LTV350QV=m # CONFIG_BACKLIGHT_CLASS_DEVICE is not set # # Display device support # # CONFIG_DISPLAY_SUPPORT is not set # # Sound # CONFIG_SOUND=m # # Advanced Linux Sound Architecture # # CONFIG_SND is not set # # Open Sound System # CONFIG_SOUND_PRIME=m # CONFIG_SOUND_TRIDENT is not set # CONFIG_SOUND_MSNDCLAS is not set CONFIG_SOUND_MSNDPIN=m CONFIG_MSNDPIN_INIT_FILE="/etc/sound/pndspini.bin" CONFIG_MSNDPIN_PERM_FILE="/etc/sound/pndsperm.bin" CONFIG_SOUND_OSS=m CONFIG_SOUND_TRACEINIT=y CONFIG_SOUND_DMAP=y # CONFIG_SOUND_SSCAPE is not set # CONFIG_SOUND_VMIDI is not set CONFIG_SOUND_TRIX=m # CONFIG_SOUND_MSS is not set CONFIG_SOUND_MPU401=m CONFIG_SOUND_PAS=m CONFIG_SOUND_PSS=m CONFIG_PSS_MIXER=y CONFIG_SOUND_SB=m CONFIG_SOUND_YM3812=m # CONFIG_SOUND_UART6850 is not set CONFIG_SOUND_AEDSP16=m CONFIG_SC6600=y CONFIG_SC6600_JOY=y CONFIG_SC6600_CDROM=4 CONFIG_SC6600_CDROMBASE=0x0 # CONFIG_AEDSP16_MSS is not set # CONFIG_AEDSP16_SBPRO is not set # CONFIG_SOUND_KAHLUA is not set CONFIG_AC97_BUS=m # CONFIG_HID_SUPPORT is not set CONFIG_USB_SUPPORT=y CONFIG_USB_ARCH_HAS_HCD=y CONFIG_USB_ARCH_HAS_OHCI=y CONFIG_USB_ARCH_HAS_EHCI=y CONFIG_USB=y CONFIG_USB_DEBUG=y # CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set # # Miscellaneous USB options # # CONFIG_USB_DEVICEFS is not set CONFIG_USB_DEVICE_CLASS=y CONFIG_USB_DYNAMIC_MINORS=y # CONFIG_USB_OTG is not set # # USB Host Controller Drivers # CONFIG_USB_EHCI_HCD=y CONFIG_USB_EHCI_ROOT_HUB_TT=y CONFIG_USB_EHCI_TT_NEWSCHED=y CONFIG_USB_ISP116X_HCD=y # CONFIG_USB_OHCI_HCD is not set CONFIG_USB_UHCI_HCD=m CONFIG_USB_U132_HCD=m # CONFIG_USB_SL811_HCD is not set CONFIG_USB_R8A66597_HCD=y # # USB Device Class drivers # CONFIG_USB_ACM=m CONFIG_USB_PRINTER=y # # NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support' # # # may also be needed; see USB_STORAGE Help for more information # # CONFIG_USB_LIBUSUAL is not set # # USB Imaging devices # # CONFIG_USB_MDC800 is not set # CONFIG_USB_MON is not set # # USB port drivers # CONFIG_USB_USS720=y CONFIG_USB_SERIAL=m CONFIG_USB_EZUSB=y CONFIG_USB_SERIAL_GENERIC=y # CONFIG_USB_SERIAL_AIRCABLE is not set # CONFIG_USB_SERIAL_AIRPRIME is not set # CONFIG_USB_SERIAL_ARK3116 is not set # CONFIG_USB_SERIAL_BELKIN is not set # CONFIG_USB_SERIAL_CH341 is not set CONFIG_USB_SERIAL_WHITEHEAT=m CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m CONFIG_USB_SERIAL_CP2101=m CONFIG_USB_SERIAL_CYPRESS_M8=m CONFIG_USB_SERIAL_EMPEG=m CONFIG_USB_SERIAL_FTDI_SIO=m # CONFIG_USB_SERIAL_FUNSOFT is not set # CONFIG_USB_SERIAL_VISOR is not set CONFIG_USB_SERIAL_IPAQ=m # CONFIG_USB_SERIAL_IR is not set CONFIG_USB_SERIAL_EDGEPORT=m CONFIG_USB_SERIAL_EDGEPORT_TI=m CONFIG_USB_SERIAL_GARMIN=m # CONFIG_USB_SERIAL_IPW is not set CONFIG_USB_SERIAL_IUU=m CONFIG_USB_SERIAL_KEYSPAN_PDA=m CONFIG_USB_SERIAL_KEYSPAN=m # CONFIG_USB_SERIAL_KEYSPAN_MPR is not set CONFIG_USB_SERIAL_KEYSPAN_USA28=y CONFIG_USB_SERIAL_KEYSPAN_USA28X=y # CONFIG_USB_SERIAL_KEYSPAN_USA28XA is not set # CONFIG_USB_SERIAL_KEYSPAN_USA28XB is not set # CONFIG_USB_SERIAL_KEYSPAN_USA19 is not set CONFIG_USB_SERIAL_KEYSPAN_USA18X=y CONFIG_USB_SERIAL_KEYSPAN_USA19W=y # CONFIG_USB_SERIAL_KEYSPAN_USA19QW is not set CONFIG_USB_SERIAL_KEYSPAN_USA19QI=y CONFIG_USB_SERIAL_KEYSPAN_USA49W=y # CONFIG_USB_SERIAL_KEYSPAN_USA49WLC is not set CONFIG_USB_SERIAL_KLSI=m # CONFIG_USB_SERIAL_KOBIL_SCT is not set CONFIG_USB_SERIAL_MCT_U232=m # CONFIG_USB_SERIAL_MOS7720 is not set CONFIG_USB_SERIAL_MOS7840=m CONFIG_USB_SERIAL_NAVMAN=m # CONFIG_USB_SERIAL_PL2303 is not set CONFIG_USB_SERIAL_OTI6858=m CONFIG_USB_SERIAL_HP4X=m CONFIG_USB_SERIAL_SAFE=m # CONFIG_USB_SERIAL_SAFE_PADDED is not set # CONFIG_USB_SERIAL_SIERRAWIRELESS is not set CONFIG_USB_SERIAL_TI=m # CONFIG_USB_SERIAL_CYBERJACK is not set # CONFIG_USB_SERIAL_XIRCOM is not set CONFIG_USB_SERIAL_OPTION=m CONFIG_USB_SERIAL_OMNINET=m # CONFIG_USB_SERIAL_DEBUG is not set # # USB Miscellaneous drivers # CONFIG_USB_EMI62=y # CONFIG_USB_EMI26 is not set CONFIG_USB_ADUTUX=y CONFIG_USB_AUERSWALD=m CONFIG_USB_RIO500=m CONFIG_USB_LEGOTOWER=m CONFIG_USB_LCD=m # CONFIG_USB_BERRY_CHARGE is not set CONFIG_USB_LED=y # CONFIG_USB_CYPRESS_CY7C63 is not set CONFIG_USB_CYTHERM=m CONFIG_USB_PHIDGET=y CONFIG_USB_PHIDGETKIT=y # CONFIG_USB_PHIDGETMOTORCONTROL is not set CONFIG_USB_PHIDGETSERVO=y CONFIG_USB_IDMOUSE=y CONFIG_USB_FTDI_ELAN=m # CONFIG_USB_APPLEDISPLAY is not set CONFIG_USB_SISUSBVGA=m CONFIG_USB_LD=y # CONFIG_USB_TRANCEVIBRATOR is not set CONFIG_USB_IOWARRIOR=m # CONFIG_USB_GADGET is not set CONFIG_MMC=m CONFIG_MMC_DEBUG=y CONFIG_MMC_UNSAFE_RESUME=y # # MMC/SD Card Drivers # CONFIG_SDIO_UART=m # # MMC/SD Host Controller Drivers # # CONFIG_MMC_SDHCI is not set CONFIG_MMC_WBSD=m CONFIG_MMC_TIFM_SD=m CONFIG_MMC_SPI=m CONFIG_MEMSTICK=m CONFIG_MEMSTICK_DEBUG=y # # MemoryStick drivers # CONFIG_MEMSTICK_UNSAFE_RESUME=y # # MemoryStick Host Controller Drivers # CONFIG_MEMSTICK_TIFM_MS=m CONFIG_MEMSTICK_JMICRON_38X=m CONFIG_NEW_LEDS=y CONFIG_LEDS_CLASS=y # # LED drivers # CONFIG_LEDS_NET48XX=m # CONFIG_LEDS_WRAP is not set CONFIG_LEDS_CLEVO_MAIL=m # # LED Triggers # # CONFIG_LEDS_TRIGGERS is not set CONFIG_INFINIBAND=y # CONFIG_INFINIBAND_USER_MAD is not set CONFIG_INFINIBAND_USER_ACCESS=m CONFIG_INFINIBAND_USER_MEM=y # CONFIG_INFINIBAND_MTHCA is not set CONFIG_MLX4_INFINIBAND=m # CONFIG_EDAC is not set CONFIG_RTC_LIB=m CONFIG_RTC_CLASS=m # # Conflicting RTC option has been selected, check GEN_RTC and RTC # # # RTC interfaces # CONFIG_RTC_INTF_SYSFS=y CONFIG_RTC_INTF_PROC=y # CONFIG_RTC_INTF_DEV is not set CONFIG_RTC_DRV_TEST=m # # SPI RTC drivers # # CONFIG_RTC_DRV_MAX6902 is not set CONFIG_RTC_DRV_R9701=m CONFIG_RTC_DRV_RS5C348=m # # Platform RTC drivers # CONFIG_RTC_DRV_CMOS=m # CONFIG_RTC_DRV_DS1511 is not set # CONFIG_RTC_DRV_DS1553 is not set # CONFIG_RTC_DRV_DS1742 is not set CONFIG_RTC_DRV_STK17TA8=m CONFIG_RTC_DRV_M48T86=m # CONFIG_RTC_DRV_M48T59 is not set CONFIG_RTC_DRV_V3020=m # # on-CPU RTC drivers # # CONFIG_DMADEVICES is not set # CONFIG_AUXDISPLAY is not set # # Userspace I/O # CONFIG_UIO=m # CONFIG_UIO_CIF is not set # # Firmware Drivers # CONFIG_EDD=m # CONFIG_DELL_RBU is not set CONFIG_DCDBAS=m CONFIG_DMIID=y # # File systems # # CONFIG_DNOTIFY is not set # CONFIG_INOTIFY is not set CONFIG_QUOTA=y CONFIG_PRINT_QUOTA_WARNING=y CONFIG_QFMT_V1=m # CONFIG_QFMT_V2 is not set CONFIG_QUOTACTL=y CONFIG_AUTOFS_FS=m CONFIG_AUTOFS4_FS=m CONFIG_FUSE_FS=m # # Pseudo filesystems # CONFIG_PROC_FS=y # CONFIG_PROC_KCORE is not set # CONFIG_PROC_SYSCTL is not set CONFIG_SYSFS=y # CONFIG_TMPFS is not set # CONFIG_HUGETLBFS is not set # CONFIG_HUGETLB_PAGE is not set # CONFIG_CONFIGFS_FS is not set # # Miscellaneous filesystems # # CONFIG_NLS is not set # # Kernel hacking # CONFIG_TRACE_IRQFLAGS_SUPPORT=y CONFIG_ENABLE_WARN_DEPRECATED=y CONFIG_ENABLE_MUST_CHECK=y # CONFIG_MAGIC_SYSRQ is not set # CONFIG_UNUSED_SYMBOLS is not set CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_SHIRQ=y # CONFIG_DETECT_SOFTLOCKUP is not set # CONFIG_SCHED_DEBUG is not set CONFIG_SCHEDSTATS=y # CONFIG_TIMER_STATS is not set # CONFIG_SLUB_STATS is not set # CONFIG_DEBUG_RT_MUTEXES is not set CONFIG_RT_MUTEX_TESTER=y CONFIG_DEBUG_SPINLOCK=y CONFIG_DEBUG_MUTEXES=y CONFIG_DEBUG_LOCK_ALLOC=y CONFIG_PROVE_LOCKING=y CONFIG_LOCKDEP=y # CONFIG_LOCK_STAT is not set CONFIG_DEBUG_LOCKDEP=y CONFIG_TRACE_IRQFLAGS=y CONFIG_DEBUG_SPINLOCK_SLEEP=y CONFIG_DEBUG_LOCKING_API_SELFTESTS=y CONFIG_STACKTRACE=y # CONFIG_DEBUG_KOBJECT is not set # CONFIG_DEBUG_BUGVERBOSE is not set # CONFIG_DEBUG_INFO is not set CONFIG_DEBUG_VM=y CONFIG_DEBUG_LIST=y CONFIG_DEBUG_SG=y # CONFIG_FRAME_POINTER is not set CONFIG_RCU_TORTURE_TEST=m CONFIG_KPROBES_SANITY_TEST=y CONFIG_BACKTRACE_SELF_TEST=m # CONFIG_FAULT_INJECTION is not set # CONFIG_LATENCYTOP is not set CONFIG_PROVIDE_OHCI1394_DMA_INIT=y CONFIG_SAMPLES=y # CONFIG_SAMPLE_MARKERS is not set CONFIG_SAMPLE_KOBJECT=m # CONFIG_SAMPLE_KPROBES is not set # CONFIG_EARLY_PRINTK is not set # CONFIG_DEBUG_STACKOVERFLOW is not set CONFIG_DEBUG_STACK_USAGE=y # CONFIG_DEBUG_PAGEALLOC is not set # CONFIG_DEBUG_RODATA is not set # CONFIG_DEBUG_NX_TEST is not set # CONFIG_4KSTACKS is not set CONFIG_X86_FIND_SMP_CONFIG=y CONFIG_X86_MPPARSE=y CONFIG_DOUBLEFAULT=y CONFIG_IO_DELAY_TYPE_0X80=0 CONFIG_IO_DELAY_TYPE_0XED=1 CONFIG_IO_DELAY_TYPE_UDELAY=2 CONFIG_IO_DELAY_TYPE_NONE=3 # CONFIG_IO_DELAY_0X80 is not set # CONFIG_IO_DELAY_0XED is not set CONFIG_IO_DELAY_UDELAY=y # CONFIG_IO_DELAY_NONE is not set CONFIG_DEFAULT_IO_DELAY_TYPE=2 CONFIG_DEBUG_BOOT_PARAMS=y CONFIG_CPA_DEBUG=y # # Security options # # CONFIG_KEYS is not set # CONFIG_SECURITY is not set CONFIG_SECURITY_FILE_CAPABILITIES=y CONFIG_CRYPTO=y CONFIG_CRYPTO_ALGAPI=y CONFIG_CRYPTO_AEAD=y CONFIG_CRYPTO_BLKCIPHER=y CONFIG_CRYPTO_SEQIV=y CONFIG_CRYPTO_HASH=y CONFIG_CRYPTO_MANAGER=y CONFIG_CRYPTO_HMAC=y # CONFIG_CRYPTO_XCBC is not set # CONFIG_CRYPTO_NULL is not set CONFIG_CRYPTO_MD4=y # CONFIG_CRYPTO_MD5 is not set CONFIG_CRYPTO_SHA1=y # CONFIG_CRYPTO_SHA256 is not set CONFIG_CRYPTO_SHA512=y # CONFIG_CRYPTO_WP512 is not set CONFIG_CRYPTO_TGR192=m CONFIG_CRYPTO_GF128MUL=y CONFIG_CRYPTO_ECB=m CONFIG_CRYPTO_CBC=m # CONFIG_CRYPTO_PCBC is not set CONFIG_CRYPTO_LRW=y # CONFIG_CRYPTO_XTS is not set CONFIG_CRYPTO_CTR=y CONFIG_CRYPTO_GCM=y # CONFIG_CRYPTO_CCM is not set CONFIG_CRYPTO_CRYPTD=m CONFIG_CRYPTO_DES=m # CONFIG_CRYPTO_FCRYPT is not set # CONFIG_CRYPTO_BLOWFISH is not set # CONFIG_CRYPTO_TWOFISH is not set # CONFIG_CRYPTO_TWOFISH_586 is not set CONFIG_CRYPTO_SERPENT=m CONFIG_CRYPTO_AES=m # CONFIG_CRYPTO_AES_586 is not set CONFIG_CRYPTO_CAST5=m CONFIG_CRYPTO_CAST6=y CONFIG_CRYPTO_TEA=y # CONFIG_CRYPTO_ARC4 is not set CONFIG_CRYPTO_KHAZAD=y CONFIG_CRYPTO_ANUBIS=y CONFIG_CRYPTO_SEED=y CONFIG_CRYPTO_SALSA20=y # CONFIG_CRYPTO_SALSA20_586 is not set # CONFIG_CRYPTO_DEFLATE is not set # CONFIG_CRYPTO_MICHAEL_MIC is not set # CONFIG_CRYPTO_CRC32C is not set CONFIG_CRYPTO_CAMELLIA=y # CONFIG_CRYPTO_TEST is not set CONFIG_CRYPTO_AUTHENC=y # CONFIG_CRYPTO_LZO is not set # CONFIG_CRYPTO_HW is not set CONFIG_HAVE_KVM=y # CONFIG_VIRTUALIZATION is not set # # Library routines # CONFIG_BITREVERSE=m # CONFIG_CRC_CCITT is not set CONFIG_CRC16=y CONFIG_CRC_ITU_T=y CONFIG_CRC32=m CONFIG_CRC7=m # CONFIG_LIBCRC32C is not set CONFIG_PLIST=y CONFIG_HAS_IOMEM=y CONFIG_HAS_IOPORT=y CONFIG_HAS_DMA=y CONFIG_CHECK_SIGNATURE=y -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From sashak at voltaire.com Fri Mar 14 09:36:14 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 16:36:14 +0000 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080314163614.GO10479@sashak.voltaire.com> On 07:14 Fri 14 Mar , Hal Rosenstock wrote: > > Have what as one field ? change_flgs ? Yes. > I was referring to a pad3 at the end of the structure as in the spec. Ah, ok - this is another thing. Spec says: "Shall be ignored on read. Content is unspecified." Sasha From sashak at voltaire.com Fri Mar 14 09:39:22 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 16:39:22 +0000 Subject: [ofa-general] Standby OpenSM reporting "Bad LinearFDBTop" value In-Reply-To: <1205504038.28072.300.camel@hrosenstock-ws.xsigo.com> References: <20080313070617.GU10479@sashak.voltaire.com> <1205408139.28072.164.camel@hrosenstock-ws.xsigo.com> <20080313140608.GW10479@sashak.voltaire.com> <1205417430.28072.190.camel@hrosenstock-ws.xsigo.com> <20080313213627.GD10479@sashak.voltaire.com> <1205444105.28072.260.camel@hrosenstock-ws.xsigo.com> <20080314002406.GJ10479@sashak.voltaire.com> <1205504038.28072.300.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080314163922.GP10479@sashak.voltaire.com> On 07:13 Fri 14 Mar , Hal Rosenstock wrote: > > opensm/osm_sw_info_rcv.c: Clarify LinearFDBTop correction log message > > Signed-off-by: Hal Rosenstock Looks fine. Applied. Thanks. Sasha From hrosenstock at xsigo.com Fri Mar 14 09:27:23 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 14 Mar 2008 09:27:23 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <20080314163614.GO10479@sashak.voltaire.com> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> <20080314163614.GO10479@sashak.voltaire.com> Message-ID: <1205512043.28072.350.camel@hrosenstock-ws.xsigo.com> On Fri, 2008-03-14 at 16:36 +0000, Sasha Khapyorsky wrote: > On 07:14 Fri 14 Mar , Hal Rosenstock wrote: > > > > Have what as one field ? change_flgs ? > > Yes. > > > I was referring to a pad3 at the end of the structure as in the spec. > > Ah, ok - this is another thing. Spec says: "Shall be ignored on read. > Content is unspecified." Yes but the field is present none the less (to pad out the structure to the full MAD size so we don't get into issues there again). -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sashak at voltaire.com Fri Mar 14 09:45:58 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 16:45:58 +0000 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <1205512043.28072.350.camel@hrosenstock-ws.xsigo.com> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> <20080314163614.GO10479@sashak.voltaire.com> <1205512043.28072.350.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080314164558.GQ10479@sashak.voltaire.com> On 09:27 Fri 14 Mar , Hal Rosenstock wrote: > > Yes but the field is present none the less (to pad out the structure to > the full MAD size so we don't get into issues there again). ntc_144 is union field anyway, and there are no trailing padding in other ntc_xxxx. Sasha From weiny2 at llnl.gov Fri Mar 14 09:40:27 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 14 Mar 2008 09:40:27 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080314094027.2d6c0023.weiny2@llnl.gov> On Fri, 14 Mar 2008 07:14:05 -0700 Hal Rosenstock wrote: > On Fri, 2008-03-14 at 00:12 +0000, Sasha Khapyorsky wrote: > > On 11:13 Wed 12 Mar , Hal Rosenstock wrote: > > > > > > > > struct _ntc_144 { > > > > ib_net16_t pad1; > > > > - ib_net16_t lid; // lid where capability mask changed > > > > - ib_net16_t pad2; > > > > - ib_net32_t new_cap_mask; // new capability mask > > > > + ib_net16_t lid; // lid where change occured > > > > + uint8_t pad2; // reserved > > > > + uint8_t local_changes; // 7b reserved 1b local changes > > > > + ib_net32_t new_cap_mask; // new capability mask > > > > + ib_net16_t change_flgs; // 13b reserved 3b change flags > > > > > > Should this be padded out as in the 1.2.1 spec ? > > > > It is stated in the 1.2.1 that an upper bits in change_flgs are reserved > > for the same purpose - OtherLocalChanges mask. So it looks ok for me to > > have it as one field and not redo later. > > Have what as one field ? change_flgs ? > > I was referring to a pad3 at the end of the structure as in the spec. > I don't think we need this as the union has a "raw_data" of size 54 bytes at the beginning. However, here is a patch which pads out all the structures according to the spec. Ira >From 9c3deb0df42e31a3807ee32a0ea7c20b8303b016 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Fri, 14 Mar 2008 09:38:36 -0700 Subject: [PATCH] opensm/include/iba/ib_types.h: pad structs in ib_mad_notice_attr_t.data_details Signed-off-by: Ira K. Weiny --- opensm/include/iba/ib_types.h | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 51695b5..edc1732 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -7129,16 +7129,19 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated struct _ntc_64_67 { uint8_t res[6]; ib_gid_t gid; // the Node or Multicast Group that came in/out + uint8_t pad[32]; } PACK_SUFFIX ntc_64_67; struct _ntc_128 { ib_net16_t sw_lid; // the sw lid of which link state changed + uint8_t pad[52]; } PACK_SUFFIX ntc_128; struct _ntc_129_131 { ib_net16_t pad; ib_net16_t lid; // lid and port number of the violation uint8_t port_num; + uint8_t pad2[49]; } PACK_SUFFIX ntc_129_131; struct _ntc_144 { @@ -7148,6 +7151,7 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated uint8_t local_changes; // 7b reserved 1b local changes ib_net32_t new_cap_mask; // new capability mask ib_net16_t change_flgs; // 13b reserved 3b change flags + uint8_t pad3[42]; } PACK_SUFFIX ntc_144; struct _ntc_145 { @@ -7155,6 +7159,7 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated ib_net16_t lid; // lid where sys guid changed ib_net16_t pad2; ib_net64_t new_sys_guid; // new system image guid + uint8_t pad3[40]; } PACK_SUFFIX ntc_145; struct _ntc_256 { // total: 54 @@ -7182,6 +7187,7 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated ib_net32_t qp2; // 4 ib_gid_t gid1; // 16 ib_gid_t gid2; // 16 + uint8_t pad2[4]; } PACK_SUFFIX ntc_257_258; struct _ntc_259 // pkey violation from switch 51 @@ -7196,6 +7202,7 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated ib_gid_t gid2; // 16 ib_net16_t sw_lid; // 2 uint8_t port_no; // 1 + uint8_t pad2[3]; } PACK_SUFFIX ntc_259; } data_details; -- 1.5.1 From hrosenstock at xsigo.com Fri Mar 14 09:42:38 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 14 Mar 2008 09:42:38 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <20080314164558.GQ10479@sashak.voltaire.com> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> <20080314163614.GO10479@sashak.voltaire.com> <1205512043.28072.350.camel@hrosenstock-ws.xsigo.com> <20080314164558.GQ10479@sashak.voltaire.com> Message-ID: <1205512958.28072.357.camel@hrosenstock-ws.xsigo.com> On Fri, 2008-03-14 at 16:45 +0000, Sasha Khapyorsky wrote: > On 09:27 Fri 14 Mar , Hal Rosenstock wrote: > > > > Yes but the field is present none the less (to pad out the structure to > > the full MAD size so we don't get into issues there again). > > ntc_144 is union field anyway, and there are no trailing padding in > other ntc_xxxx. Guess I don't see how this could get us into trouble either way. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From rdreier at cisco.com Fri Mar 14 10:33:07 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 14 Mar 2008 10:33:07 -0700 Subject: [ofa-general] itt member in iscsi_data? Message-ID: iser_initiator.c has this code: itt = ntohl(hdr->itt); which triggers sparse warnings like: drivers/infiniband/ulp/iser/iser_initiator.c:419:8: warning: cast to restricted type drivers/infiniband/ulp/iser/iser_initiator.c:419:8: warning: cast from restricted type and indeed the itt field is declared as itt_t, with no particular endianness, so the sparse warnings seem like they are pointing at something that really is suspicious. It seems the only use of itt in the function that does ntohl() is to print the value our in debugging statements. What is intended here? Would it just make sense to for the debug statements to print hdr->itt out directly with no conversion? - R. From tecferrocentroservizixow at ferrocentroservizi.com Fri Mar 14 11:31:05 2008 From: tecferrocentroservizixow at ferrocentroservizi.com (Gilbert Newsome) Date: Fri, 14 Mar 2008 19:31:05 +0100 Subject: [ofa-general] Software Message-ID: <141053784.00121748345233@ferrocentroservizi.com> In search for the better prices in software rebates? Now you will obtain the chance to have the softs you dream of from long time ago. And the best matter for you is, all softs are of awfully low value. Examine by yourself & have the softwares for lowest prices! http://mayaphillips434554.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From privacy at nypost.com Fri Mar 14 14:11:17 2008 From: privacy at nypost.com (Jenifer Hawk) Date: Fri, 14 Mar 2008 23:11:17 +0200 Subject: [ofa-general] Re: Urgent!!!! Message-ID: <01c88628$b52b8080$f2557c5b@privacy> REGIONAL PAYMENT RECEIVING AGENT NEEDED DT-systems Ltd. is based in Lithuania. We specialize in exportation and importation. We export our products to North America, South America, Eastern and Western Europe and Southern Asia. We are looking for a payment representative in UK, USA and Canada. Salary is 10% of every payment you receive on our behalf. All charges such as tax and fe_es will be deducted from the balance 90%. For this job position you have to provide with your bank account information. Note: Even if you have a real job, you can be part of our business anyway as your regular and part-time job can be easily combined, your work for our company will not disturb your regular work. If you are interested in this opportuni1y please send out your contact information to our company email: zhen.gao26 at yahoo.com 1)Your Full names: 2)Your Address. 3)Postal code: 4)Home/office phone number: 5)cell phone number 6)Occupation 8)Sex: Your address should be correct and complete (including your state and country) because you will also be receiving cheques to your address. Attention ! Please write only this email : zhen.gao26 at yahoo.com Managers of our company will come in contact with you as soon as possible. Having received the information, we will give you a contract, in which the responsibility of both sides is fixed. Sincerely, HR manager Jonas Varnas From patrick.latifi at qlogic.com Fri Mar 14 14:30:15 2008 From: patrick.latifi at qlogic.com (Patrick Marchand Latifi) Date: Fri, 14 Mar 2008 14:30:15 -0700 Subject: [ofa-general] [PATCH 1/2] [uDAPL v1] fix provider-specific compiler warnings Message-ID: <20080314213015.12201.6949.stgit@b64-10.mv.qlogic.com> Initialize ds_array_start_p otherwise the compiler would claim that this variable could be used with an uninitialized value. Makes the uDAPL providers now build successfully when using make VERBS= (the -Werror flag was causing the build failure) Signed-off-by: Patrick Marchand Latifi --- dapl/openib/dapl_ib_dto.h | 8 ++++---- dapl/openib_cma/dapl_ib_dto.h | 8 ++++---- dapl/openib_scm/dapl_ib_dto.h | 8 ++++---- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/dapl/openib/dapl_ib_dto.h b/dapl/openib/dapl_ib_dto.h index 8b0e8fe..d4e40c3 100644 --- a/dapl/openib/dapl_ib_dto.h +++ b/dapl/openib/dapl_ib_dto.h @@ -67,7 +67,7 @@ dapls_ib_post_recv ( IN DAT_LMR_TRIPLET *local_iov ) { ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_recv_wr wr; struct ibv_recv_wr *bad_wr; DAT_COUNT i, total_len; @@ -116,7 +116,7 @@ dapls_ib_post_recv ( ret = ibv_post_recv(ep_ptr->qp_handle, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) @@ -147,7 +147,7 @@ dapls_ib_post_send ( remote_iov, completion_flags); ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_send_wr wr; struct ibv_send_wr *bad_wr; ib_hca_transport_t *ibt_ptr = &ep_ptr->header.owner_ia->hca_ptr->ib_trans; @@ -224,7 +224,7 @@ dapls_ib_post_send ( ret = ibv_post_send(ep_ptr->qp_handle, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) diff --git a/dapl/openib_cma/dapl_ib_dto.h b/dapl/openib_cma/dapl_ib_dto.h index 4f077de..2bdd5e0 100644 --- a/dapl/openib_cma/dapl_ib_dto.h +++ b/dapl/openib_cma/dapl_ib_dto.h @@ -67,7 +67,7 @@ dapls_ib_post_recv ( IN DAT_LMR_TRIPLET *local_iov ) { ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_recv_wr wr; struct ibv_recv_wr *bad_wr; DAT_COUNT i, total_len; @@ -116,7 +116,7 @@ dapls_ib_post_recv ( ret = ibv_post_recv(ep_ptr->qp_handle->cm_id->qp, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) @@ -148,7 +148,7 @@ dapls_ib_post_send ( remote_iov, completion_flags); ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_send_wr wr; struct ibv_send_wr *bad_wr; ib_hca_transport_t *ibt_ptr = @@ -226,7 +226,7 @@ dapls_ib_post_send ( ret = ibv_post_send(ep_ptr->qp_handle->cm_id->qp, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) diff --git a/dapl/openib_scm/dapl_ib_dto.h b/dapl/openib_scm/dapl_ib_dto.h index cede876..bea3e4d 100644 --- a/dapl/openib_scm/dapl_ib_dto.h +++ b/dapl/openib_scm/dapl_ib_dto.h @@ -67,7 +67,7 @@ dapls_ib_post_recv ( IN DAT_LMR_TRIPLET *local_iov ) { ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_recv_wr wr; struct ibv_recv_wr *bad_wr; DAT_COUNT i, total_len; @@ -116,7 +116,7 @@ dapls_ib_post_recv ( ret = ibv_post_recv(ep_ptr->qp_handle, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) @@ -147,7 +147,7 @@ dapls_ib_post_send ( remote_iov, completion_flags); ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_send_wr wr; struct ibv_send_wr *bad_wr; ib_hca_transport_t *ibt_ptr = &ep_ptr->header.owner_ia->hca_ptr->ib_trans; @@ -224,7 +224,7 @@ dapls_ib_post_send ( ret = ibv_post_send(ep_ptr->qp_handle, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) From patrick.latifi at qlogic.com Fri Mar 14 14:30:20 2008 From: patrick.latifi at qlogic.com (Patrick Marchand Latifi) Date: Fri, 14 Mar 2008 14:30:20 -0700 Subject: [ofa-general] [PATCH 2/2] [uDAPL v1] fix openib_scm compiler warning In-Reply-To: <20080314213015.12201.6949.stgit@b64-10.mv.qlogic.com> References: <20080314213015.12201.6949.stgit@b64-10.mv.qlogic.com> Message-ID: <20080314213020.12201.194.stgit@b64-10.mv.qlogic.com> Cast to socklen_t since accept(2) expects an unsigned argument. Makes the openib_scm provider now build successfully when using make VERBS= (the -Werror flag was causing the build failure) Signed-off-by: Patrick Marchand Latifi --- dapl/openib_scm/dapl_ib_cm.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/dapl/openib_scm/dapl_ib_cm.c b/dapl/openib_scm/dapl_ib_cm.c index ea75453..f534e8d 100644 --- a/dapl/openib_scm/dapl_ib_cm.c +++ b/dapl/openib_scm/dapl_ib_cm.c @@ -347,7 +347,7 @@ dapli_socket_accept(ib_cm_srvc_handle_t cm_ptr) len = sizeof(acm_ptr->dst.ia_address); acm_ptr->socket = accept(cm_ptr->l_socket, (struct sockaddr*)&acm_ptr->dst.ia_address, - &len ); + (socklen_t*)&len ); if ( acm_ptr->socket < 0 ) { dapl_dbg_log(DAPL_DBG_TYPE_ERR, From patrick.latifi at qlogic.com Fri Mar 14 14:31:34 2008 From: patrick.latifi at qlogic.com (Patrick Marchand Latifi) Date: Fri, 14 Mar 2008 14:31:34 -0700 Subject: [ofa-general] [PATCH 1/2] [uDAPL v2] fix provider-specific compiler warnings Message-ID: <20080314213134.12355.79367.stgit@b64-10.mv.qlogic.com> Initialize ds_array_start_p otherwise the compiler would claim that this variable could be used with an uninitialized value. Makes the uDAPL providers now build successfully when using make VERBS= (the -Werror flag was causing the build failure) Signed-off-by: Patrick Marchand Latifi --- dapl/openib/dapl_ib_dto.h | 8 ++++---- dapl/openib_cma/dapl_ib_dto.h | 8 ++++---- dapl/openib_scm/dapl_ib_dto.h | 8 ++++---- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/dapl/openib/dapl_ib_dto.h b/dapl/openib/dapl_ib_dto.h index 8b0e8fe..d4e40c3 100644 --- a/dapl/openib/dapl_ib_dto.h +++ b/dapl/openib/dapl_ib_dto.h @@ -67,7 +67,7 @@ dapls_ib_post_recv ( IN DAT_LMR_TRIPLET *local_iov ) { ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_recv_wr wr; struct ibv_recv_wr *bad_wr; DAT_COUNT i, total_len; @@ -116,7 +116,7 @@ dapls_ib_post_recv ( ret = ibv_post_recv(ep_ptr->qp_handle, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) @@ -147,7 +147,7 @@ dapls_ib_post_send ( remote_iov, completion_flags); ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_send_wr wr; struct ibv_send_wr *bad_wr; ib_hca_transport_t *ibt_ptr = &ep_ptr->header.owner_ia->hca_ptr->ib_trans; @@ -224,7 +224,7 @@ dapls_ib_post_send ( ret = ibv_post_send(ep_ptr->qp_handle, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) diff --git a/dapl/openib_cma/dapl_ib_dto.h b/dapl/openib_cma/dapl_ib_dto.h index b614740..a90aea2 100644 --- a/dapl/openib_cma/dapl_ib_dto.h +++ b/dapl/openib_cma/dapl_ib_dto.h @@ -71,7 +71,7 @@ dapls_ib_post_recv ( IN DAT_LMR_TRIPLET *local_iov ) { ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_recv_wr wr; struct ibv_recv_wr *bad_wr; DAT_COUNT i, total_len; @@ -120,7 +120,7 @@ dapls_ib_post_recv ( ret = ibv_post_recv(ep_ptr->qp_handle->cm_id->qp, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) @@ -151,7 +151,7 @@ dapls_ib_post_send ( remote_iov, completion_flags); ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_send_wr wr; struct ibv_send_wr *bad_wr; ib_hca_transport_t *ibt_ptr = @@ -230,7 +230,7 @@ dapls_ib_post_send ( ret = ibv_post_send(ep_ptr->qp_handle->cm_id->qp, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) diff --git a/dapl/openib_scm/dapl_ib_dto.h b/dapl/openib_scm/dapl_ib_dto.h index cede876..bea3e4d 100644 --- a/dapl/openib_scm/dapl_ib_dto.h +++ b/dapl/openib_scm/dapl_ib_dto.h @@ -67,7 +67,7 @@ dapls_ib_post_recv ( IN DAT_LMR_TRIPLET *local_iov ) { ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_recv_wr wr; struct ibv_recv_wr *bad_wr; DAT_COUNT i, total_len; @@ -116,7 +116,7 @@ dapls_ib_post_recv ( ret = ibv_post_recv(ep_ptr->qp_handle, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) @@ -147,7 +147,7 @@ dapls_ib_post_send ( remote_iov, completion_flags); ib_data_segment_t ds_array[DEFAULT_DS_ENTRIES]; - ib_data_segment_t *ds_array_p, *ds_array_start_p; + ib_data_segment_t *ds_array_p, *ds_array_start_p = NULL; struct ibv_send_wr wr; struct ibv_send_wr *bad_wr; ib_hca_transport_t *ibt_ptr = &ep_ptr->header.owner_ia->hca_ptr->ib_trans; @@ -224,7 +224,7 @@ dapls_ib_post_send ( ret = ibv_post_send(ep_ptr->qp_handle, &wr, &bad_wr); - if (segments > DEFAULT_DS_ENTRIES) + if (ds_array_start_p != NULL) dapl_os_free(ds_array_start_p, segments * sizeof(ib_data_segment_t)); if (ret) From patrick.latifi at qlogic.com Fri Mar 14 14:31:40 2008 From: patrick.latifi at qlogic.com (Patrick Marchand Latifi) Date: Fri, 14 Mar 2008 14:31:40 -0700 Subject: [ofa-general] [PATCH 2/2] [uDAPL v2] fix openib_scm compiler warning In-Reply-To: <20080314213134.12355.79367.stgit@b64-10.mv.qlogic.com> References: <20080314213134.12355.79367.stgit@b64-10.mv.qlogic.com> Message-ID: <20080314213139.12355.28054.stgit@b64-10.mv.qlogic.com> Cast to socklen_t since accept(2) expects an unsigned argument. Makes the openib_scm provider now build successfully when using make VERBS= (the -Werror flag was causing the build failure) Signed-off-by: Patrick Marchand Latifi --- dapl/openib_scm/dapl_ib_cm.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/dapl/openib_scm/dapl_ib_cm.c b/dapl/openib_scm/dapl_ib_cm.c index ea75453..f534e8d 100644 --- a/dapl/openib_scm/dapl_ib_cm.c +++ b/dapl/openib_scm/dapl_ib_cm.c @@ -347,7 +347,7 @@ dapli_socket_accept(ib_cm_srvc_handle_t cm_ptr) len = sizeof(acm_ptr->dst.ia_address); acm_ptr->socket = accept(cm_ptr->l_socket, (struct sockaddr*)&acm_ptr->dst.ia_address, - &len ); + (socklen_t*)&len ); if ( acm_ptr->socket < 0 ) { dapl_dbg_log(DAPL_DBG_TYPE_ERR, From sashak at voltaire.com Fri Mar 14 15:49:40 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 14 Mar 2008 22:49:40 +0000 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <20080314094027.2d6c0023.weiny2@llnl.gov> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> <20080314094027.2d6c0023.weiny2@llnl.gov> Message-ID: <20080314224940.GT10479@sashak.voltaire.com> On 09:40 Fri 14 Mar , Ira Weiny wrote: > > I don't think we need this as the union has a "raw_data" of size 54 bytes at > the beginning. Yes, correct. > However, here is a patch which pads out all the structures > according to the spec. Do we need it? Sasha From weiny2 at llnl.gov Fri Mar 14 16:43:40 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 14 Mar 2008 16:43:40 -0700 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <20080314224940.GT10479@sashak.voltaire.com> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> <20080314094027.2d6c0023.weiny2@llnl.gov> <20080314224940.GT10479@sashak.voltaire.com> Message-ID: <20080314164340.09c25186.weiny2@llnl.gov> On Fri, 14 Mar 2008 22:49:40 +0000 Sasha Khapyorsky wrote: > On 09:40 Fri 14 Mar , Ira Weiny wrote: > > > > I don't think we need this as the union has a "raw_data" of size 54 bytes at > > the beginning. > > Yes, correct. > > > However, here is a patch which pads out all the structures > > according to the spec. > > Do we need it? I don't think so, Ira > > Sasha From sashak at voltaire.com Fri Mar 14 18:22:41 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 15 Mar 2008 01:22:41 +0000 Subject: [ofa-general] [PATCH] opensm/include/iba/ib_types.h: update Notice DataDetails for Trap 144 to 1.2.1 In-Reply-To: <20080314164340.09c25186.weiny2@llnl.gov> References: <20080312102335.5eecc8d0.weiny2@llnl.gov> <1205345590.28072.66.camel@hrosenstock-ws.xsigo.com> <20080314001218.GH10479@sashak.voltaire.com> <1205504045.28072.302.camel@hrosenstock-ws.xsigo.com> <20080314094027.2d6c0023.weiny2@llnl.gov> <20080314224940.GT10479@sashak.voltaire.com> <20080314164340.09c25186.weiny2@llnl.gov> Message-ID: <20080315012241.GU10479@sashak.voltaire.com> On 16:43 Fri 14 Mar , Ira Weiny wrote: > > I don't think so, Agreed. Sasha From xvgunvz at farscape-fan.com Fri Mar 14 19:21:20 2008 From: xvgunvz at farscape-fan.com (lara) Date: Fri, 14 Mar 2008 18:21:20 -0800 Subject: [ofa-general] hi from lara Message-ID: <4123D058.363100.86316@SCLC> Hi My name is lara. I found your email on that dating site. I also love sex on the side. I have a loving partner but he is working 16 hours a day and we have sex only once a week :( If you are interested and wanna see my pictures just email me at clara0 at golovable.com Don`t reply, use the email above (my boyfriend doesn`t know about that email!) From xvgunvz at farscape-fan.com Fri Mar 14 19:21:20 2008 From: xvgunvz at farscape-fan.com (lara) Date: Fri, 14 Mar 2008 18:21:20 -0800 Subject: [ofa-general] hi from lara Message-ID: <4123D058.363100.86316@SCLC> Hi My name is lara. I found your email on that dating site. I also love sex on the side. I have a loving partner but he is working 16 hours a day and we have sex only once a week :( If you are interested and wanna see my pictures just email me at clara0 at golovable.com Don`t reply, use the email above (my boyfriend doesn`t know about that email!) From murriel at kidputer.com Sat Mar 15 05:59:21 2008 From: murriel at kidputer.com (Yanira Barry) Date: Sat, 15 Mar 2008 13:59:21 +0100 Subject: [ofa-general] Qualität hoch, Preis niedrig, Software prima! Message-ID: <903501477.27687940696930@kidputer.com> Die Software auf allen europaischen Sprachen, fur Windows und Macintosh vorherbestimmt. Die konnen Sie momentan bekommen. Nur bezahlen und auslasten. Hier prasentiert sind nicht teuere, aber echte und vollige Produkte der Software.http://geocities.com/buckherschel/Wollen Sie das Programm aufstellen? Benutzen Sie die Hilfe der professionellen Konsultation des Anwenderdienstes. Wir antworten auf Ihre Fragen schnell und garantieren die Moglichkeit der Ruckzahlung. Sie kaufen die Software, sie funktionieren ausgezeichnethttp://geocities.com/buckherschel/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From latug at brandstifter.com Sat Mar 15 07:09:14 2008 From: latug at brandstifter.com (Cathy Talley) Date: Sat, 15 Mar 2008 23:09:14 +0900 Subject: [ofa-general] Help your loved one stop smoking Message-ID: <01c886f1$96030600$2c1591dd@latug> Live longer life without cigarettes At least read it! It is YOUR life, it is important. Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: ls.zip Type: application/zip Size: 560 bytes Desc: not available URL: From corbettspud0 at seagate.com Sat Mar 15 06:07:55 2008 From: corbettspud0 at seagate.com (jerrome kyra) Date: Sat, 15 Mar 2008 13:07:55 +0000 Subject: [ofa-general] wen-jing Message-ID: <000401c886ac$03e28c62$8626a492@ijmtawxg> but almagestattitude be beaux From avigilanzann at netuno.net Sat Mar 15 08:13:03 2008 From: avigilanzann at netuno.net (EuroSoftware) Date: Sat, 15 Mar 2008 16:13:03 +0100 Subject: [ofa-general] Brandneue Software auf Alter Preis Message-ID: <01c886b7$725fd780$b08d4b51@avigilanzann> Start using your software immediately after purchase.Wir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren.http://geocities.com/garner.santos/* Windows XP Professional With SP2 Full Version: $59.95 * Office Enterprise 2007: $79.95 * Adobe Photoshop CS3 Extended: $79.95 * Office System Professional 2003 (5 Cds): $59.95 http://geocities.com/garner.santos/Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From dca at braasco.com Sat Mar 15 09:30:54 2008 From: dca at braasco.com (Jeanine Vann) Date: Sat, 15 Mar 2008 11:30:54 -0500 Subject: [ofa-general] It will change your life Message-ID: <01c88690$078aab00$b71018be@dca> Tired of being overweight? We can help! You must know this secret! Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: as.zip Type: application/zip Size: 578 bytes Desc: not available URL: From khjlekdubkl at bostik.com Sat Mar 15 10:42:54 2008 From: khjlekdubkl at bostik.com (Mac Pettit) Date: Sat, 15 Mar 2008 18:42:54 +0100 Subject: [ofa-general] Please send info. Message-ID: <01c886cc$6110cb00$88971d53@khjlekdubkl> REGIONAL PAYMENT RECEIVING AGENT NEEDED DT-systems Ltd. is based in Lithuania. We specialize in exportation and importation. We export our products to North America, South America, Eastern and Western Europe and Southern Asia. We are looking for a payment representative in UK, USA and Canada. Salary is 10% of every payment you receive on our behalf. All charges such as tax and fe_es will be deducted from the balance 90%. For this job position you have to provide with your bank account information. Note: Even if you have a real job, you can be part of our business anyway as your regular and part-time job can be easily combined, your work for our company will not disturb your regular work. If you are interested in this opportuni1y please send out your contact information to our company email: peterhironymous at yahoo.com 1)Your Full names: 2)Your Address. 3)Postal code: 4)Home/office phone number: 5)cell phone number 6)Occupation 8)Sex: Your address should be correct and complete (including your state and country) because you will also be receiving cheques to your address. Attention ! Please write only this email : peterhironymous at yahoo.com Managers of our company will come in contact with you as soon as possible. Having received the information, we will give you a contract, in which the responsibility of both sides is fixed. Sincerely, HR manager Jonas Varnas From info at ephrod.net Sat Mar 15 11:23:49 2008 From: info at ephrod.net (=?windows-1255?B?5Ons5CD0+PU=?=) Date: Sat, 15 Mar 2008 13:23:49 -0500 Subject: [ofa-general] =?windows-1255?b?4Pog+OX25CDi7SAyMCwwMDAg+SIg4efl?= =?windows-1255?b?4/kgPyA=?= Message-ID: <20080315182432.94EE3E60916@openfabrics.org> An HTML attachment was scrubbed... URL: From longgiacomputer at gmail.com Sat Mar 15 11:34:19 2008 From: longgiacomputer at gmail.com (Sales Deparment) Date: Sun, 16 Mar 2008 01:34:19 +0700 Subject: [ofa-general] (no subject) Message-ID: CONG TY TNHH TM & DV KY THUAT LONG GIA 467/11-13 Dien Bien Phu, F.3, Q.3, TP.HCM Tel : 8306697 - 8306698 - 2908339 - 8308092 - 8355316 - 8355317 Fax : (84-8) 8304594 Email : longgiacomputer at gmail.com - trunglonggia at gmail.com Website : www.longgia.com Cong ty Long Gia xin kinh chuc quy khach luon luon thanh dat va khong ngung phat trien Chung toi xin goi den Quy Khach BANG BAO GIA Xin Quy Khach vui long mo Files dinh kem Thanh that xin loi neu mail nay lam phien Quy Khach Moi chi tiet xin vui long lien he : PHONG KINH DOANH Mrs. Hang Miss.YEN Miss.Diem Mr.Kiet Mr.Phat -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: BBG DEALER MAY IN 2008.doc Type: application/msword Size: 329216 bytes Desc: not available URL: From cchfanjsvjg at booff.com Sat Mar 15 14:10:12 2008 From: cchfanjsvjg at booff.com (Wilfredo Burger) Date: Sat, 15 Mar 2008 22:10:12 +0100 Subject: [ofa-general] Live longer life without cigarettes Message-ID: <01c886e9$56b0fa00$3314d24d@cchfanjsvjg> Live longer life without cigarettes At least read it! It is YOUR life, it is important. Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: ls.zip Type: application/zip Size: 560 bytes Desc: not available URL: From maxloragno at mechron.com Sat Mar 15 15:43:44 2008 From: maxloragno at mechron.com (University Degree) Date: Sat, 15 Mar 2008 23:43:44 +0100 Subject: [ofa-general] University Degree Without Pains Message-ID: <406643680.96696283226708@mechron.com> 1-410-230-1849 University Degree Get University Degree within weeks without stress, exams, and long years of studying. Confidentiality Assured 1-410-230-1849 24 hours a day, 7 days a week including Sundays and Holidays -------------- next part -------------- An HTML attachment was scrubbed... URL: From Golden-01 at Mail.ru Sat Mar 15 16:23:20 2008 From: Golden-01 at Mail.ru () Date: Sun, 16 Mar 2008 02:23:20 +0300 Subject: [ofa-general] =?iso-8859-1?q?=C2=E0=F8_=E4=EE=EF=EE=EB=ED=E8=F2?= =?iso-8859-1?q?=E5=EB=FC=ED=FB=E9_=E7=E0=F0=E0=E1=EE=F2=EE=EA=2C?= =?iso-8859-1?q?=E0_=E2=EE=E7=EC=EE=E6=ED=EE_=E8_=EE=E1=E5=F1=EF=E5?= =?iso-8859-1?q?=F7=E5=ED=ED=EE=E5_=E1=F3=E4=F3=F9=E5=E5=2E?= Message-ID: <200803160223.XQMUBYIUDG@82.138.52.149>                                                                                Добрый день !      Извините,что,возможно ,отнимаю у Вас время,но если Вы имеете  доступ к Интернету,умеете пользоваться  мышкой,   имеете 2-3 часа свободного времени,и Вас интересуют деньги,то всего один щелчок мышкой может отделять Вас от состояния !!! Еще раз простите за вторжение.Желаю Вам удачного дня . Жду Вашего ответа : Moskvich-V at Mail.ru  или  Vladimir11682 at Yandex.ru Удачи Вам !!! Информация находиться в прикрепленном   файле  .... Не беспокойтесь, это не вирус,но если Вы ,тем не менее,  опасаетесь  его  открыть  ,не  спешите  его  удалять ,вовсе не сложно проверить почту при помощи антивируса. Это не реклама, а действительно выгодное предложение !!! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Golden Stream.doc Type: application/octet-stream Size: 160256 bytes Desc: not available URL: From epeee at bobmallin.com Sat Mar 15 18:02:40 2008 From: epeee at bobmallin.com (Dominick Lugo) Date: , 16 Mar 2008 10:02:40 +0900 Subject: [ofa-general] Melt away pounds with it Message-ID: <01c8874c$deda1e80$ae3a793a@epeee> Burn pounds off with it You must know this secret! Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: as.zip Type: application/zip Size: 578 bytes Desc: not available URL: From rdreier at cisco.com Sat Mar 15 19:10:55 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sat, 15 Mar 2008 19:10:55 -0700 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Add checksum offload support In-Reply-To: <1204124831.3358.11.camel@mtls03> (Eli Cohen's message of "Wed, 27 Feb 2008 17:07:11 +0200") References: <1204124831.3358.11.camel@mtls03> Message-ID: thanks, I applied the ipoib and mlx4 patches with some cleanups. One thing I changed is that ipoib doesn't keep a cached copy of the HCA capabilities flags -- I didn't see any reason why it was needed. The patches I have in my tree are below: >From b6fe014b2ade84f82c614ee68292fb85ce1fc573 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 27 Feb 2008 17:07:08 +0200 Subject: [PATCH] IPoIB: Use checksum offload support if available For HCAs that support checksum offload (ie that set IB_DEVICE_UD_IP_CSUM in the device capabilities flags), have IPoIB set NETIF_F_IP_CSUM and use the HCA to generate and verify IP checksums. Signed-off-by: Eli Cohen Signed-off-by: Roland Dreier --- drivers/infiniband/ulp/ipoib/ipoib.h | 1 + drivers/infiniband/ulp/ipoib/ipoib_cm.c | 8 ++++++++ drivers/infiniband/ulp/ipoib/ipoib_ib.c | 11 +++++++++++ drivers/infiniband/ulp/ipoib/ipoib_main.c | 22 +++++++++++++++++++++- 4 files changed, 41 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 054fab8..08930ca 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -87,6 +87,7 @@ enum { IPOIB_MCAST_STARTED = 8, IPOIB_FLAG_ADMIN_CM = 9, IPOIB_FLAG_UMCAST = 10, + IPOIB_FLAG_CSUM = 11, IPOIB_MAX_BACKOFF_SECONDS = 16, diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 2490b2d..edf63dc 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1383,6 +1383,10 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, set_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); + + dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG); + priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; + ipoib_flush_paths(dev); return count; } @@ -1391,6 +1395,10 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, clear_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); dev->mtu = min(priv->mcast_mtu, dev->mtu); ipoib_flush_paths(dev); + + if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) + dev->features |= NETIF_F_IP_CSUM | NETIF_F_SG; + return count; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 08c4396..8ed09d1 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -231,6 +231,12 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) skb->dev = dev; /* XXX get correct PACKET_ type here */ skb->pkt_type = PACKET_HOST; + + if (test_bit(IPOIB_FLAG_CSUM, &priv->flags) && likely(wc->csum_ok)) + skb->ip_summed = CHECKSUM_UNNECESSARY; + else + skb->ip_summed = CHECKSUM_NONE; + netif_receive_skb(skb); repost: @@ -442,6 +448,11 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, return; } + if (skb->ip_summed == CHECKSUM_PARTIAL) + priv->tx_wr.send_flags |= IB_SEND_IP_CSUM; + else + priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; + if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), address->ah, qpn, tx_req->mapping, skb_headlen(skb), diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 5728204..d0fbb0e 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -1105,6 +1105,7 @@ static struct net_device *ipoib_add_port(const char *format, struct ib_device *hca, u8 port) { struct ipoib_dev_priv *priv; + struct ib_device_attr *device_attr = NULL; int result = -ENOMEM; priv = ipoib_intf_alloc(format); @@ -1120,6 +1121,25 @@ static struct net_device *ipoib_add_port(const char *format, goto device_init_failed; } + device_attr = kmalloc(sizeof *device_attr, GFP_KERNEL); + if (!device_attr) { + printk(KERN_WARNING "%s: allocation of %zu bytes failed\n", + hca->name, sizeof *device_attr); + goto device_init_failed; + } + + result = ib_query_device(hca, device_attr); + if (result) { + printk(KERN_WARNING "%s: ib_query_device failed (ret = %d)\n", + hca->name, result); + goto device_init_failed; + } + + if (device_attr->device_cap_flags & IB_DEVICE_UD_IP_CSUM) { + set_bit(IPOIB_FLAG_CSUM, &priv->flags); + priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; + } + /* * Set the full membership bit, so that we join the right * broadcast group, etc. @@ -1137,7 +1157,6 @@ static struct net_device *ipoib_add_port(const char *format, } else memcpy(priv->dev->dev_addr + 4, priv->local_gid.raw, sizeof (union ib_gid)); - result = ipoib_dev_init(priv->dev, hca, port); if (result < 0) { printk(KERN_WARNING "%s: failed to initialize port %d (ret = %d)\n", @@ -1192,6 +1211,7 @@ device_init_failed: free_netdev(priv->dev); alloc_mem_failed: + kfree(device_attr); return ERR_PTR(result); } -- 1.5.4.3 >From b3dfa9bed3b72555ee30ad200de24ee30ec55844 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 27 Feb 2008 17:07:11 +0200 Subject: [PATCH] IB/mlx4: Add IPoIB checksum offload support ConnectX devices support checksum generation and verification of TCP and UDP headers for UD IPoIB messages. This patch checks if the HCA supports this and sets the IB_DEVICE_UD_IP_CSUM capability flag if it does. It implements support for handling the IB_SEND_IP_CSUM send flag and setting the csum_ok field in receive work completion. Signed-off-by: Eli Cohen Signed-off-by: Ali Ayub Signed-off-by: Roland Dreier --- drivers/infiniband/hw/mlx4/cq.c | 16 ++++++++++++++++ drivers/infiniband/hw/mlx4/main.c | 2 ++ drivers/infiniband/hw/mlx4/qp.c | 3 +++ drivers/net/mlx4/fw.c | 4 ++++ include/linux/mlx4/cq.h | 14 ++++++++++++-- include/linux/mlx4/qp.h | 10 ++++++---- 6 files changed, 43 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 7360bba..d2e32b0 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -297,6 +297,20 @@ static void mlx4_ib_handle_error_cqe(struct mlx4_err_cqe *cqe, wc->vendor_err = cqe->vendor_err_syndrome; } +static int mlx4_ib_ipoib_csum_ok(__be32 status, __be16 checksum) +{ + return ((status & cpu_to_be32(MLX4_CQE_IPOIB_STATUS_IPV4 | + MLX4_CQE_IPOIB_STATUS_IPV4F | + MLX4_CQE_IPOIB_STATUS_IPV4OPT | + MLX4_CQE_IPOIB_STATUS_IPV6 | + MLX4_CQE_IPOIB_STATUS_IPOK)) == + cpu_to_be32(MLX4_CQE_IPOIB_STATUS_IPV4 | + MLX4_CQE_IPOIB_STATUS_IPOK)) && + (status & cpu_to_be32(MLX4_CQE_IPOIB_STATUS_UDP | + MLX4_CQE_IPOIB_STATUS_TCP)) && + checksum == cpu_to_be16(0xffff); +} + static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, struct mlx4_ib_qp **cur_qp, struct ib_wc *wc) @@ -434,6 +448,8 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, wc->dlid_path_bits = (g_mlpath_rqpn >> 24) & 0x7f; wc->wc_flags |= g_mlpath_rqpn & 0x80000000 ? IB_WC_GRH : 0; wc->pkey_index = be32_to_cpu(cqe->immed_rss_invalid) & 0x7f; + wc->csum_ok = mlx4_ib_ipoib_csum_ok(cqe->ipoib_status, + cqe->checksum); } return 0; diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 96a39b5..ef5e9db 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -99,6 +99,8 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->device_cap_flags |= IB_DEVICE_AUTO_PATH_MIG; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_UD_AV_PORT) props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; + if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) + props->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index ac965ab..31b2b5b 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -1436,6 +1436,9 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE) : 0) | (wr->send_flags & IB_SEND_SOLICITED ? cpu_to_be32(MLX4_WQE_CTRL_SOLICITED) : 0) | + ((wr->send_flags & IB_SEND_IP_CSUM) ? + cpu_to_be32(MLX4_WQE_CTRL_IP_CSUM | + MLX4_WQE_CTRL_TCP_UDP_CSUM) : 0) | qp->sq_signal_bits; if (wr->opcode == IB_WR_SEND_WITH_IMM || diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index 61dc495..f494c3e 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -696,6 +696,10 @@ int mlx4_INIT_HCA(struct mlx4_dev *dev, struct mlx4_init_hca_param *param) /* Check port for UD address vector: */ *(inbox + INIT_HCA_FLAGS_OFFSET / 4) |= cpu_to_be32(1); + /* Enable IPoIB checksumming if we can: */ + if (dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) + *(inbox + INIT_HCA_FLAGS_OFFSET / 4) |= cpu_to_be32(1 << 3); + /* QPC/EEC/CQC/EQC/RDMARC attributes */ MLX4_PUT(inbox, param->qpc_base, INIT_HCA_QPC_BASE_OFFSET); diff --git a/include/linux/mlx4/cq.h b/include/linux/mlx4/cq.h index 0181e0a..1243eba 100644 --- a/include/linux/mlx4/cq.h +++ b/include/linux/mlx4/cq.h @@ -45,11 +45,11 @@ struct mlx4_cqe { u8 sl; u8 reserved1; __be16 rlid; - u32 reserved2; + __be32 ipoib_status; __be32 byte_cnt; __be16 wqe_index; __be16 checksum; - u8 reserved3[3]; + u8 reserved2[3]; u8 owner_sr_opcode; }; @@ -85,6 +85,16 @@ enum { MLX4_CQE_SYNDROME_REMOTE_ABORTED_ERR = 0x22, }; +enum { + MLX4_CQE_IPOIB_STATUS_IPV4 = 1 << 22, + MLX4_CQE_IPOIB_STATUS_IPV4F = 1 << 23, + MLX4_CQE_IPOIB_STATUS_IPV6 = 1 << 24, + MLX4_CQE_IPOIB_STATUS_IPV4OPT = 1 << 25, + MLX4_CQE_IPOIB_STATUS_TCP = 1 << 26, + MLX4_CQE_IPOIB_STATUS_UDP = 1 << 27, + MLX4_CQE_IPOIB_STATUS_IPOK = 1 << 28, +}; + static inline void mlx4_cq_arm(struct mlx4_cq *cq, u32 cmd, void __iomem *uar_page, spinlock_t *doorbell_lock) diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 09a2230..31f9eb3 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -158,10 +158,12 @@ struct mlx4_qp_context { #define MLX4_FW_VER_WQE_CTRL_NEC mlx4_fw_ver(2, 2, 232) enum { - MLX4_WQE_CTRL_NEC = 1 << 29, - MLX4_WQE_CTRL_FENCE = 1 << 6, - MLX4_WQE_CTRL_CQ_UPDATE = 3 << 2, - MLX4_WQE_CTRL_SOLICITED = 1 << 1, + MLX4_WQE_CTRL_NEC = 1 << 29, + MLX4_WQE_CTRL_FENCE = 1 << 6, + MLX4_WQE_CTRL_CQ_UPDATE = 3 << 2, + MLX4_WQE_CTRL_SOLICITED = 1 << 1, + MLX4_WQE_CTRL_IP_CSUM = 1 << 4, + MLX4_WQE_CTRL_TCP_UDP_CSUM = 1 << 5, }; struct mlx4_wqe_ctrl_seg { -- 1.5.4.3 From revering9 at auscomp.net Sat Mar 15 20:31:41 2008 From: revering9 at auscomp.net (Dana Fry) Date: , 16 Mar 2008 11:31:41 +0800 Subject: [ofa-general] Viagra (Sildenafil) 100mg x 60 pills $129.95 price Message-ID: <01c88759$4df80c80$76b8417d@revering9> US $ 159.95 buy now 100mg x 90 pills http://nhebodqdc67.blogspot.com From mubjjy at bordeaux-millesimes.com Sat Mar 15 20:40:25 2008 From: mubjjy at bordeaux-millesimes.com (Domingo Pruitt) Date: , 16 Mar 2008 10:40:25 +0700 Subject: [ofa-general] Men and Women FREE INTERNATIONAL SHIPPING on Gucci Prada Dior D&G Dsquared Shoes Heels Ugg Boots Message-ID: <01c88752$2487b280$938b177b@mubjjy> Hey Have you heard? The 2008 Collections are in. Designer Shoes, Heels, Boots, the Year's Craziest sale. Forget department store prices, Find Gucci Prada Chanel Dior, D&G, Dsquared Versace and More, Men & Women HALF-OFF on ALL International Shipping at NO Cost on all orders! http://www.17starshoe.com What are you waiting for? This is how we Save! An Amazing Offer! From HughwiremenHolland at ogleearth.com Sun Mar 16 01:00:15 2008 From: HughwiremenHolland at ogleearth.com (Dwight Medina) Date: Sun, 16 Mar 2008 07:00:15 -0100 Subject: [ofa-general] Visa Line of Credit Message-ID: <6IX470EJXVWDA797@ogleearth.com> An HTML attachment was scrubbed... URL: From sashak at voltaire.com Sat Mar 15 23:42:04 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 16 Mar 2008 06:42:04 +0000 Subject: [ofa-general] [PATCH] opensm: added convenience function for the opensm log In-Reply-To: <47D597DD.5060201@llnl.gov> References: <47D581B6.9020709@llnl.gov> <47D597DD.5060201@llnl.gov> Message-ID: <20080316064204.GC22627@sashak.voltaire.com> Hi Tim, On 13:19 Mon 10 Mar , Timothy A. Meier wrote: >> This patch very similar to the "accessor" patch I submitted last week, but >> focuses >> primarily on the Log. And similar to the previos patch it creates a global reference to opensm object. Sasha From ogerlitz at voltaire.com Sun Mar 16 01:06:07 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 16 Mar 2008 10:06:07 +0200 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Add checksum offload support In-Reply-To: References: <1204124831.3358.11.camel@mtls03> Message-ID: <47DCD4EF.5@voltaire.com> Roland Dreier wrote: > thanks, I applied the ipoib and mlx4 patches with some cleanups. > One thing I changed is that ipoib doesn't keep a cached copy of the > HCA capabilities flags -- I didn't see any reason why it was needed. > Such a copy would make the patches for LSO and interrupt moderation support simpler. If we agree on this point, it could be added through the patch set for those two features, which in the light of the integration these two patches, should be on its way (Eli? also the mthca checksum offload patch). > --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c > @@ -1120,6 +1121,25 @@ static struct net_device *ipoib_add_port(const char *format, > goto device_init_failed; > } > .... > + if (device_attr->device_cap_flags & IB_DEVICE_UD_IP_CSUM) { > + set_bit(IPOIB_FLAG_CSUM, &priv->flags); > + priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; > + } > Don't you need to check the mode (connected vs datagram) of the device at this point before advertising these the S/G and CSUM flags to the stack? Or. From 3djtan at cyberoptics.com Sun Mar 16 01:45:47 2008 From: 3djtan at cyberoptics.com (Shelby Lambert) Date: , 16 Mar 2008 09:45:47 +0100 Subject: [ofa-general] Adobe Acrobat 8, MS Office 2007, AutoCAD 2008 Message-ID: <998455709.23741855981989@cyberoptics.com> Um die echte und vollige Software in kurzer Zeit zu bekommen, braucht man nur zu bezahlen und auszulasten. Sie haben dann die Programmen auf allen europaischen Sprachen uberlassen, die fur Windows und Macintosh vorherbestimmt sind. http://geocities.com/estella.cole/Haben Sie Haben Sie Schwierigkeiten bei der Aufstellung des Programms? Sie bekommen die Hilfe der professionellen Konsultation des Anwenderdienstes. Haben Sie Fragen? Wir antworten schnell. Die Ruckzahlung ist moglich. Kaufen die vollkommen funktionierende Software -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Sun Mar 16 01:53:22 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 16 Mar 2008 10:53:22 +0200 Subject: [ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc() In-Reply-To: References: <1205329275.26105.28.camel@mtls03> <47D8FC31.9050105@voltaire.com> Message-ID: <47DCE002.4090806@voltaire.com> Roland Dreier wrote: > As far as other ethernet drivers, using vmalloc seems fairly common: > > e1000/e1000_main.c: txdr->buffer_info = vmalloc(size); > e1000/e1000_main.c: rxdr->buffer_info = vmalloc(size); > > bnx2.c: bp->rx_buf_ring = vmalloc(sizeof(struct sw_bd) * RX_DESC_CNT * > > [bnx2x does the same thing but hides it in a macro] > > ixgbe/ixgbe_main.c: txdr->tx_buffer_info = vmalloc(size); > ixgbe/ixgbe_main.c: rxdr->rx_buffer_info = vmalloc(size); > > e1000e/netdev.c: tx_ring->buffer_info = vmalloc(size); > e1000e/netdev.c: rx_ring->buffer_info = vmalloc(size); > > netxen/netxen_nic_main.c: cmd_buf_arr = (struct netxen_cmd_buffer *)vmalloc(TX_RINGSIZE); > > Sounds like we can do this as well. Or. From tziporet at dev.mellanox.co.il Sun Mar 16 02:34:14 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Sun, 16 Mar 2008 11:34:14 +0200 Subject: [ofa-general] kernel panic (sporadically) OFED-1.2 In-Reply-To: <20080314160842.GV3073@saul.informatik.tu-chemnitz.de> References: <20080314160842.GV3073@saul.informatik.tu-chemnitz.de> Message-ID: <47DCE996.5080501@mellanox.co.il> Frank Mietke wrote: > Hi, > > has anybody seen the kernel panic below? Any hints? > > We're using RHEL-4 clone with special Lustre kernel > 2.6.9-55.0.9.EL_lustre.1.6.4.2smp and OFED-1.2 > > What is the HCA you are using? If its from Mellanox what is the FW version used? Have you tried 1.2.5.5 - we fixes several issues in this release since 1.2.0 Tziporet From tziporet at dev.mellanox.co.il Sun Mar 16 02:35:46 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Sun, 16 Mar 2008 11:35:46 +0200 Subject: [ofa-general] Last batch of 2.6.25 patches... any missing? In-Reply-To: References: <200803130838.07544.jackm@dev.mellanox.co.il> Message-ID: <47DCE9F2.7040507@mellanox.co.il> Roland Dreier wrote: > > Also, what about the XRC patches? > > Clearly not for 2.6.25... > Sure - I meant for 2.6.26 BTW - can we agree on patches that targeted to 2.6.26? Tziporet From _e_t at adwriting.com Sun Mar 16 02:01:25 2008 From: _e_t at adwriting.com (domenico ben) Date: Sun, 16 Mar 2008 09:01:25 +0000 Subject: [ofa-general] Order Viagra - Fast, Easy and Confidential. Special suggestion for openib-general's. Message-ID: <000801c88753$043db576$ca21d186@ohawm> Boost your sexual strength in 15 minutes! .WKhtjWaq 1. Want to be a perfect lover? Want to boost your sexual power twice? 2. Than Viagra, Cialis, Soma, Levitra and Human Growth Hormone medications are the right choice! 3. Buy Viagra, Cialis, Soma, Levitra and Human Growth Hormone pills at our discount price store!!! 4. We do guarantee high-quality medications, instant worldwide delivery and friendly support. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tastingk65 at gordy-n-mikie.com Sun Mar 16 04:18:39 2008 From: tastingk65 at gordy-n-mikie.com (Nita Jackson) Date: , 16 Mar 2008 08:18:39 -0300 Subject: [ofa-general] $159.95 Price for Viagra 100mg x 90 pills Message-ID: <01c8873e$568f1980$09fc0cbd@tastingk65> 100mg x 30 pills $3.33 per pill http://uepmsmubt53.blogspot.com From a-3 at aac-audio.com Sun Mar 16 04:29:09 2008 From: a-3 at aac-audio.com (Pablo Perez) Date: , 16 Mar 2008 19:29:09 +0800 Subject: [ofa-general] Your profile Message-ID: <01c8879c$00e95a00$faa9607a@a-3> Hello! I am tired tonight. I am nice girl that would like to chat with you. Email me at Kerstin at WilderGoLovan.com only, because I am using my friend's email to write this. If you would like to see my pictures. From erb at sliferdesigns.com Sun Mar 16 05:02:23 2008 From: erb at sliferdesigns.com (EuroSoftware) Date: , 16 Mar 2008 19:02:23 +0700 Subject: [ofa-general] Klicken Sie Einfach Auf zu Kaufen OEM Message-ID: <331304155.12224769461960@sliferdesigns.com> Die Software auf allen europaischen Sprachen, fur Windows und Macintosh vorherbestimmt. Die konnen Sie momentan bekommen. Nur bezahlen und auslasten. Hier prasentiert sind nicht teuere, aber echte und vollige Produkte der Software.http://geocities.com/osbornejacklyn/Die Aufstellung des Programms ist jetzt kein Problem fur Ihnen. Die professionelle Konsultation des Anwenderdienstes hilft dabei. Garantiert sind die schnelle Antwort und die Moglichkeit der Ruckzahlung. Die vollkommen funktionierende Software sind immer zu kaufenhttp://geocities.com/osbornejacklyn/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From teixeira1 at heniq.com.br Sun Mar 16 05:03:39 2008 From: teixeira1 at heniq.com.br (Regina Corbett) Date: , 16 Mar 2008 07:03:39 -0500 Subject: [ofa-general] Live longer life without cigarettes Message-ID: <01c88733$dc599780$296af4c9@teixeira1> Quit smoking today At least read it! It is YOUR life, it is important. Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: ls.zip Type: application/zip Size: 560 bytes Desc: not available URL: From eli at dev.mellanox.co.il Sun Mar 16 05:22:24 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Sun, 16 Mar 2008 14:22:24 +0200 Subject: [ofa-general] IPOIB checksum offload support Message-ID: <1205670144.25950.53.camel@mtls03> Hi Roland, in reference to this commit ID b6fe014b2ade84f82c614ee68292fb85ce1fc573: 1. It seems that there is a memory leak that the following fixes: diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index d0fbb0e..75f99c0 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -1140,6 +1140,9 @@ static struct net_device *ipoib_add_port(const char *format, priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; } + kfree(device_attr); + device_attr = NULL; + /* * Set the full membership bit, so that we join the right * broadcast group, etc. 2. I see that you added an else clause here: index 08c4396..8ed09d1 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -231,6 +231,12 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) skb->dev = dev; /* XXX get correct PACKET_ type here */ skb->pkt_type = PACKET_HOST; + + if (test_bit(IPOIB_FLAG_CSUM, &priv->flags) && likely(wc->csum_ok)) + skb->ip_summed = CHECKSUM_UNNECESSARY; + else + skb->ip_summed = CHECKSUM_NONE; + Since CHECKSUM_NONE is defined as 0 and since a new skb is memset to 0, it seems to be unnecessary, at least in the head kernel. Moreover, in the previous versions of IPOIB, we did not explicitly set the ip_summed field of the skb. Do you think we must have the "else" here. From facfotosandercej at fotosander.de Sun Mar 16 05:24:23 2008 From: facfotosandercej at fotosander.de (Willa Mccord) Date: , 16 Mar 2008 12:24:23 +0000 Subject: [ofa-general] Software Message-ID: <689769706.38266154630534@fotosander.de> Are you in search for the best value in discount software? Now you will find the chance to use the softwares you wanted for a very long time. And the better thing for you is, all softs are terribly cheap. Examine it by yourself and take the softwares for cheapest rates ever seen! http://deborahromingergf235.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From crockettd at sheridanbooks.com Sun Mar 16 05:35:21 2008 From: crockettd at sheridanbooks.com (Andrea Gray) Date: , 16 Mar 2008 13:35:21 +0100 Subject: [ofa-general] Top Online Software Store, Low Prices Message-ID: <945487294.99497314882339@sheridanbooks.com> Our main goal is to render low cost PC and Macintosh lawful soft and computer solutions for any budget. Whether you're a corporate buyer, a proprietor of small enterprise, or shopping for your home personal computer, we believe that we can assist you.* Windows XP Professional With SP2 Full Version: $59.95 * Office Enterprise 2007: $79.95 * Adobe Photoshop CS2 with ImageReady CS2: $79.95 * Adobe Photoshop CS3 Extended: $79.95 * AutoCAD 2008: $129.95 -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli at dev.mellanox.co.il Sun Mar 16 07:58:30 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Sun, 16 Mar 2008 16:58:30 +0200 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Add checksum offload support In-Reply-To: <47DCD4EF.5@voltaire.com> References: <1204124831.3358.11.camel@mtls03> <47DCD4EF.5@voltaire.com> Message-ID: <1205679510.25950.64.camel@mtls03> On Sun, 2008-03-16 at 10:06 +0200, Or Gerlitz wrote: > Roland Dreier wrote: > > thanks, I applied the ipoib and mlx4 patches with some cleanups. > > One thing I changed is that ipoib doesn't keep a cached copy of the > > HCA capabilities flags -- I didn't see any reason why it was needed. > > > Such a copy would make the patches for LSO and interrupt moderation > support simpler. If we agree on this point, it could be added through > the patch set for those two features, which in the light of the > integration these two patches, should be on its way (Eli? also the mthca > checksum offload patch). I am not sure what Roland meant but we could take an approach where instead of saving a copy of the hca capabilities, we will reflect these capabilities into priv->flags. We can do the same also for LSO. As for CQ moderation, my original approach was not to define this as a capability - I just did the modify. Do you think this should be a capability published by the device? > > --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c > > +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c > > @@ -1120,6 +1121,25 @@ static struct net_device *ipoib_add_port(const char *format, > > goto device_init_failed; > > } > > .... > > + if (device_attr->device_cap_flags & IB_DEVICE_UD_IP_CSUM) { > > + set_bit(IPOIB_FLAG_CSUM, &priv->flags); > > + priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; > > + } > > > Don't you need to check the mode (connected vs datagram) of the device > at this point before advertising these the S/G and CSUM flags to the stack? > > Or. IPOIB's default mode of operation is UD so this check is important only when set_mode() is invoked - Roland, is that what you thought about? > From rdreier at cisco.com Sun Mar 16 08:42:33 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 16 Mar 2008 08:42:33 -0700 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Add checksum offload support In-Reply-To: <47DCD4EF.5@voltaire.com> (Or Gerlitz's message of "Sun, 16 Mar 2008 10:06:07 +0200") References: <1204124831.3358.11.camel@mtls03> <47DCD4EF.5@voltaire.com> Message-ID: > Such a copy would make the patches for LSO and interrupt moderation > support simpler. If we agree on this point, it could be added through > the patch set for those two features, which in the light of the > integration these two patches, should be on its way Yes that would be fine. In the context of the checksum offload patch alone having the hca_caps member added nothing but complexity so I just got rid of it. > > .... + if (device_attr->device_cap_flags & > > IB_DEVICE_UD_IP_CSUM) { > > + set_bit(IPOIB_FLAG_CSUM, &priv->flags); > > + priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; > > + } > > > Don't you need to check the mode (connected vs datagram) of the device > at this point before advertising these the S/G and CSUM flags to the > stack? This is before we even do register_netdev so we know the device is in datagram mode. - R. From rdreier at cisco.com Sun Mar 16 08:43:08 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 16 Mar 2008 08:43:08 -0700 Subject: [ofa-general] IPOIB checksum offload support In-Reply-To: <1205670144.25950.53.camel@mtls03> (Eli Cohen's message of "Sun, 16 Mar 2008 14:22:24 +0200") References: <1205670144.25950.53.camel@mtls03> Message-ID: > 1. It seems that there is a memory leak that the following fixes: > 2. I see that you added an else clause here: Thanks... I fixed the memory leak and removed the else, and pushed out a new tree. - R. From likedr at assh.org Sun Mar 16 08:53:57 2008 From: likedr at assh.org (Mabel Fowler) Date: , 16 Mar 2008 07:53:57 -0800 Subject: [ofa-general] Publisher 2007 Message-ID: <01c8873a$e3a34480$de723c56@likedr> Microsoft Office Enterprise 2007 includes: • Access 2007 • Communicator 2007 • Excel 2007 • Groove 2007 • InfoPath 2007 • OneNote 2007 • Outlook 2007 • PowerPoint 2007 • Publisher 2007 • Word 2007 http://evelynalburyhx508.blogspot.com System Requirements • Intel® Pentium® or AMD® 500 MHz processor • Microsoft Windows® XP Professional or Home Edition with Service Pack 2, Windows Server® 2003 with SP1 , Microsoft Windows Vista. • 256 Mb of RAM • 2GB of available hard-disk space. • 1024x768 or higher resolution monitor • Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From a-almon at microsoft.com Sun Mar 16 11:52:09 2008 From: a-almon at microsoft.com (Maynard Cummins) Date: Mon, 17 Mar 2008 02:52:09 +0800 Subject: [ofa-general] why are you not repying? Message-ID: <476179349.55785884052743@microsoft.com> Hello! I am bored tonight. I am nice girl that would like to chat with you. Email me at Ingegerd at WilderGoLovan.com only, because I am using my friend's email to write this. I want to show you some pictures. From dikla at seminar.org Sun Mar 16 13:22:09 2008 From: dikla at seminar.org (=?windows-1255?B?4/fs5A==?=) Date: Sun, 16 Mar 2008 22:22:09 +0200 Subject: [ofa-general] =?windows-1255?b?6+Pg6SD5+vrn6ewg7OPy+iDs7uvl+CDr?= =?windows-1255?b?6SDk9Pjx5e0g8uzp8OU=?= Message-ID: <20080316202300.14952E6029A@openfabrics.org> רצית פעם להעביר סמינר משלך? מפחד מעלויות יקרות של פרסום? מעביר סמינרים ולא יודע איך לשווק אותם ביעילות? הפרסום עלינו אתר סמינרים יוצא במבצע הכרות: 0 ש"ח לחודש פרסום! ללא התחייבות ללא עלות ללא חידוש פרסום אוטומטי ללא השארת פרטי אשראי שלח מייל כעת ל seminarim.co.il at gmail.com (נא לרשום שם וטלפון) למפעילי האתר שמורה הזכות לסרב לפרסם וכן למנוע מתן שרותי האתר לכל אדם או גוף. להסרה מהרשימה לחץ כאן -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: seminarimLogo.JPG Type: application/octet-stream Size: 8798 bytes Desc: not available URL: From _prentissmd at adaptcon.com Sun Mar 16 13:15:17 2008 From: _prentissmd at adaptcon.com (link vester) Date: Sun, 16 Mar 2008 20:15:17 +0000 Subject: [ofa-general] Rihanna porno dvd preview Message-ID: <000601c887b1$04f4ccaf$59f42691@itvwujcn> Sp?hattacke auf das h?rteste Ziel der Welt WATCH IT FOR FREE -------------- next part -------------- An HTML attachment was scrubbed... URL: From lyzfrycanyonhet at frycanyon.com Sun Mar 16 18:16:47 2008 From: lyzfrycanyonhet at frycanyon.com (Lelia Herndon) Date: Mon, 17 Mar 2008 03:16:47 +0200 Subject: [ofa-general] Software Message-ID: <366953647.23243949076248@frycanyon.com> Are you searhing for the best prices in software abatements? Now you'll obtain the opportunity to have the software environment you want from very long ago. And the best thing for you is, all softwares are of terribly low rates. Check up by yourself and get the softwares for lowest rates ever seen! http://tommiemcconnellxx438.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From splines271 at honeysuckleranch.com Sun Mar 16 20:20:57 2008 From: splines271 at honeysuckleranch.com (Floyd Sutton) Date: Mon, 17 Mar 2008 11:20:57 +0800 Subject: [ofa-general] 60 pills of Vpxl = 1 months supply... Message-ID: <01c88820$f8873280$da946a7c@splines271> To get the best possible results we recommend using the program for at least four months. http://ameliamongertp247.blogspot.com From rajouri.jammu at gmail.com Sun Mar 16 21:49:17 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Sun, 16 Mar 2008 21:49:17 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <200803120835.31442.jackm@dev.mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> Message-ID: <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> Jack, The problem I'm seeing is that I'm not able to register even the default number of memory regions. The default is 131056 but I'm not able to register more than 32635 regions I'm on OFED -1.2.5.4. Any ideas? Below is the output of lspci and ibv_devinfo -v. I also recently upgraded to the latest f/w but that didn't make a difference. lspci | grep Mellanox 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20) ibv_devinfo -v hca_id: mthca0 fw_ver: 4.8.200 node_guid: 0006:6a00:9800:8403 sys_image_guid: 0006:6a00:9800:8403 vendor_id: 0x02c9 vendor_part_id: 25208 hw_ver: 0xA0 board_id: MT_0200000001 phys_port_cnt: 2 max_mr_size: 0xffffffffffffffff page_size_cap: 0xfffff000 max_qp: 64512 max_qp_wr: 65535 device_cap_flags: 0x00001c76 max_sge: 59 max_sge_rd: 0 max_cq: 65408 max_cqe: 131071 max_mr: 131056 max_pd: 32768 max_qp_rd_atom: 4 max_ee_rd_atom: 0 max_res_rd_atom: 258048 max_qp_init_rd_atom: 128 max_ee_init_rd_atom: 0 atomic_cap: ATOMIC_HCA (1) max_ee: 0 max_rdd: 0 max_mw: 0 max_raw_ipv6_qp: 0 max_raw_ethy_qp: 0 max_mcast_grp: 8192 max_mcast_qp_attach: 56 max_total_mcast_qp_attach: 458752 max_ah: 0 max_fmr: 0 max_srq: 960 max_srq_wr: 65535 max_srq_sge: 31 max_pkeys: 64 local_ca_ack_delay: 15 On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein wrote: > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > I think it's Arabel. > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell which > > driver's settings I should modify? > > > > What will be the max_mr value if log_num_mpt = 20? > > > > To see which device you have, type the following command in your linux > console: > > lspci | grep Mellanox > > For ConnectX (mlx4), you will see: > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > (lspci has not caught up with us yet). > > For Arbel (InfiniHost III), you will see either: > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA (rev a0) > > or, if you are running your arbel in Tavor compatibility mode: > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > compatibility mode) (rev 20) > =========== > If your installed HCA is a ConnectX, you should use the module parameters > for mlx4. > If your installed HCA is an InfiniHost III, you should use the module > parameters for ib_mthca. > > - Jack > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajouri.jammu at gmail.com Sun Mar 16 21:49:17 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Sun, 16 Mar 2008 21:49:17 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <200803120835.31442.jackm@dev.mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> Message-ID: <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> Jack, The problem I'm seeing is that I'm not able to register even the default number of memory regions. The default is 131056 but I'm not able to register more than 32635 regions I'm on OFED -1.2.5.4. Any ideas? Below is the output of lspci and ibv_devinfo -v. I also recently upgraded to the latest f/w but that didn't make a difference. lspci | grep Mellanox 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20) ibv_devinfo -v hca_id: mthca0 fw_ver: 4.8.200 node_guid: 0006:6a00:9800:8403 sys_image_guid: 0006:6a00:9800:8403 vendor_id: 0x02c9 vendor_part_id: 25208 hw_ver: 0xA0 board_id: MT_0200000001 phys_port_cnt: 2 max_mr_size: 0xffffffffffffffff page_size_cap: 0xfffff000 max_qp: 64512 max_qp_wr: 65535 device_cap_flags: 0x00001c76 max_sge: 59 max_sge_rd: 0 max_cq: 65408 max_cqe: 131071 max_mr: 131056 max_pd: 32768 max_qp_rd_atom: 4 max_ee_rd_atom: 0 max_res_rd_atom: 258048 max_qp_init_rd_atom: 128 max_ee_init_rd_atom: 0 atomic_cap: ATOMIC_HCA (1) max_ee: 0 max_rdd: 0 max_mw: 0 max_raw_ipv6_qp: 0 max_raw_ethy_qp: 0 max_mcast_grp: 8192 max_mcast_qp_attach: 56 max_total_mcast_qp_attach: 458752 max_ah: 0 max_fmr: 0 max_srq: 960 max_srq_wr: 65535 max_srq_sge: 31 max_pkeys: 64 local_ca_ack_delay: 15 On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein wrote: > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > I think it's Arabel. > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell which > > driver's settings I should modify? > > > > What will be the max_mr value if log_num_mpt = 20? > > > > To see which device you have, type the following command in your linux > console: > > lspci | grep Mellanox > > For ConnectX (mlx4), you will see: > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > (lspci has not caught up with us yet). > > For Arbel (InfiniHost III), you will see either: > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA (rev a0) > > or, if you are running your arbel in Tavor compatibility mode: > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > compatibility mode) (rev 20) > =========== > If your installed HCA is a ConnectX, you should use the module parameters > for mlx4. > If your installed HCA is an InfiniHost III, you should use the module > parameters for ib_mthca. > > - Jack > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Chrystal at sierracollege.edu Sun Mar 16 21:50:37 2008 From: Chrystal at sierracollege.edu (Chrystal Jacobsen) Date: Mon, 17 Mar 2008 04:50:37 -0000 Subject: [ofa-general] Get a real full measurement Message-ID: <143801bf8fcc$05bce8b0$791922d4@Chrystal> Your new bigger and stronger love wand will make your mate culminate faster and brighter! Go on, give it a try - you'll enjoy an amazing growth very soon! http://Daveedons.com/ 4's Today programme, "John was one of the wittiest and1916 - General Pancho Villa led Mexicans raiders in aSoros would change his mind. Abouzeid did indicate that I would be -------------- next part -------------- An HTML attachment was scrubbed... URL: From dotanb at dev.mellanox.co.il Sun Mar 16 23:16:01 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Mon, 17 Mar 2008 08:16:01 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> Message-ID: <47DE0CA1.6040509@dev.mellanox.co.il> Hi. What is the typical size of the MRs that you are trying to register? thanks Dotan Rajouri Jammu wrote: > Jack, > > The problem I'm seeing is that I'm not able to register even the > default number of memory regions. > The default is 131056 but I'm not able to register more than 32635 regions > I'm on OFED -1.2.5.4 . > > Any ideas? > > Below is the output of lspci and ibv_devinfo -v. I also recently > upgraded to the latest f/w but that didn't make a difference. > > > lspci | grep Mellanox > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > (Tavor compatibility mode) (rev 20) > > ibv_devinfo -v > hca_id: mthca0 > fw_ver: 4.8.200 > node_guid: 0006:6a00:9800:8403 > sys_image_guid: 0006:6a00:9800:8403 > vendor_id: 0x02c9 > vendor_part_id: 25208 > hw_ver: 0xA0 > board_id: MT_0200000001 > phys_port_cnt: 2 > max_mr_size: 0xffffffffffffffff > page_size_cap: 0xfffff000 > max_qp: 64512 > max_qp_wr: 65535 > device_cap_flags: 0x00001c76 > max_sge: 59 > max_sge_rd: 0 > max_cq: 65408 > max_cqe: 131071 > max_mr: 131056 > max_pd: 32768 > max_qp_rd_atom: 4 > max_ee_rd_atom: 0 > max_res_rd_atom: 258048 > max_qp_init_rd_atom: 128 > max_ee_init_rd_atom: 0 > atomic_cap: ATOMIC_HCA (1) > max_ee: 0 > max_rdd: 0 > max_mw: 0 > max_raw_ipv6_qp: 0 > max_raw_ethy_qp: 0 > max_mcast_grp: 8192 > max_mcast_qp_attach: 56 > max_total_mcast_qp_attach: 458752 > max_ah: 0 > max_fmr: 0 > max_srq: 960 > max_srq_wr: 65535 > max_srq_sge: 31 > max_pkeys: 64 > local_ca_ack_delay: 15 > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein > > wrote: > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > I think it's Arabel. > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell > which > > driver's settings I should modify? > > > > What will be the max_mr value if log_num_mpt = 20? > > > > To see which device you have, type the following command in your > linux console: > > lspci | grep Mellanox > > For ConnectX (mlx4), you will see: > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > (lspci has not caught up with us yet). > > For Arbel (InfiniHost III), you will see either: > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA > (rev a0) > > or, if you are running your arbel in Tavor compatibility mode: > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > compatibility mode) (rev 20) > =========== > If your installed HCA is a ConnectX, you should use the module > parameters for mlx4. > If your installed HCA is an InfiniHost III, you should use the > module parameters for ib_mthca. > > - Jack > > > ------------------------------------------------------------------------ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From dotanb at dev.mellanox.co.il Sun Mar 16 23:16:01 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Mon, 17 Mar 2008 08:16:01 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> Message-ID: <47DE0CA1.6040509@dev.mellanox.co.il> Hi. What is the typical size of the MRs that you are trying to register? thanks Dotan Rajouri Jammu wrote: > Jack, > > The problem I'm seeing is that I'm not able to register even the > default number of memory regions. > The default is 131056 but I'm not able to register more than 32635 regions > I'm on OFED -1.2.5.4 . > > Any ideas? > > Below is the output of lspci and ibv_devinfo -v. I also recently > upgraded to the latest f/w but that didn't make a difference. > > > lspci | grep Mellanox > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > (Tavor compatibility mode) (rev 20) > > ibv_devinfo -v > hca_id: mthca0 > fw_ver: 4.8.200 > node_guid: 0006:6a00:9800:8403 > sys_image_guid: 0006:6a00:9800:8403 > vendor_id: 0x02c9 > vendor_part_id: 25208 > hw_ver: 0xA0 > board_id: MT_0200000001 > phys_port_cnt: 2 > max_mr_size: 0xffffffffffffffff > page_size_cap: 0xfffff000 > max_qp: 64512 > max_qp_wr: 65535 > device_cap_flags: 0x00001c76 > max_sge: 59 > max_sge_rd: 0 > max_cq: 65408 > max_cqe: 131071 > max_mr: 131056 > max_pd: 32768 > max_qp_rd_atom: 4 > max_ee_rd_atom: 0 > max_res_rd_atom: 258048 > max_qp_init_rd_atom: 128 > max_ee_init_rd_atom: 0 > atomic_cap: ATOMIC_HCA (1) > max_ee: 0 > max_rdd: 0 > max_mw: 0 > max_raw_ipv6_qp: 0 > max_raw_ethy_qp: 0 > max_mcast_grp: 8192 > max_mcast_qp_attach: 56 > max_total_mcast_qp_attach: 458752 > max_ah: 0 > max_fmr: 0 > max_srq: 960 > max_srq_wr: 65535 > max_srq_sge: 31 > max_pkeys: 64 > local_ca_ack_delay: 15 > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein > > wrote: > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > I think it's Arabel. > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell > which > > driver's settings I should modify? > > > > What will be the max_mr value if log_num_mpt = 20? > > > > To see which device you have, type the following command in your > linux console: > > lspci | grep Mellanox > > For ConnectX (mlx4), you will see: > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > (lspci has not caught up with us yet). > > For Arbel (InfiniHost III), you will see either: > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA > (rev a0) > > or, if you are running your arbel in Tavor compatibility mode: > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > compatibility mode) (rev 20) > =========== > If your installed HCA is a ConnectX, you should use the module > parameters for mlx4. > If your installed HCA is an InfiniHost III, you should use the > module parameters for ib_mthca. > > - Jack > > > ------------------------------------------------------------------------ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From rajouri.jammu at gmail.com Sun Mar 16 23:19:02 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Sun, 16 Mar 2008 23:19:02 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <47DE0CA1.6040509@dev.mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <47DE0CA1.6040509@dev.mellanox.co.il> Message-ID: <3307cdf90803162319u4b66d8b2q57af8a275e4b43ee@mail.gmail.com> 100K Bytes. On Sun, Mar 16, 2008 at 11:16 PM, Dotan Barak wrote: > Hi. > What is the typical size of the MRs that you are trying to register? > > thanks > Dotan > > Rajouri Jammu wrote: > > Jack, > > > > The problem I'm seeing is that I'm not able to register even the > > default number of memory regions. > > The default is 131056 but I'm not able to register more than 32635 > regions > > I'm on OFED -1.2.5.4 . > > > > Any ideas? > > > > Below is the output of lspci and ibv_devinfo -v. I also recently > > upgraded to the latest f/w but that didn't make a difference. > > > > > > lspci | grep Mellanox > > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > > (Tavor compatibility mode) (rev 20) > > > > ibv_devinfo -v > > hca_id: mthca0 > > fw_ver: 4.8.200 > > node_guid: 0006:6a00:9800:8403 > > sys_image_guid: 0006:6a00:9800:8403 > > vendor_id: 0x02c9 > > vendor_part_id: 25208 > > hw_ver: 0xA0 > > board_id: MT_0200000001 > > phys_port_cnt: 2 > > max_mr_size: 0xffffffffffffffff > > page_size_cap: 0xfffff000 > > max_qp: 64512 > > max_qp_wr: 65535 > > device_cap_flags: 0x00001c76 > > max_sge: 59 > > max_sge_rd: 0 > > max_cq: 65408 > > max_cqe: 131071 > > max_mr: 131056 > > max_pd: 32768 > > max_qp_rd_atom: 4 > > max_ee_rd_atom: 0 > > max_res_rd_atom: 258048 > > max_qp_init_rd_atom: 128 > > max_ee_init_rd_atom: 0 > > atomic_cap: ATOMIC_HCA (1) > > max_ee: 0 > > max_rdd: 0 > > max_mw: 0 > > max_raw_ipv6_qp: 0 > > max_raw_ethy_qp: 0 > > max_mcast_grp: 8192 > > max_mcast_qp_attach: 56 > > max_total_mcast_qp_attach: 458752 > > max_ah: 0 > > max_fmr: 0 > > max_srq: 960 > > max_srq_wr: 65535 > > max_srq_sge: 31 > > max_pkeys: 64 > > local_ca_ack_delay: 15 > > > > > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein > > > wrote: > > > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > > I think it's Arabel. > > > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell > > which > > > driver's settings I should modify? > > > > > > What will be the max_mr value if log_num_mpt = 20? > > > > > > > To see which device you have, type the following command in your > > linux console: > > > > lspci | grep Mellanox > > > > For ConnectX (mlx4), you will see: > > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > > (lspci has not caught up with us yet). > > > > For Arbel (InfiniHost III), you will see either: > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA > > (rev a0) > > > > or, if you are running your arbel in Tavor compatibility mode: > > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > > compatibility mode) (rev 20) > > =========== > > If your installed HCA is a ConnectX, you should use the module > > parameters for mlx4. > > If your installed HCA is an InfiniHost III, you should use the > > module parameters for ib_mthca. > > > > - Jack > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajouri.jammu at gmail.com Sun Mar 16 23:19:02 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Sun, 16 Mar 2008 23:19:02 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <47DE0CA1.6040509@dev.mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <47DE0CA1.6040509@dev.mellanox.co.il> Message-ID: <3307cdf90803162319u4b66d8b2q57af8a275e4b43ee@mail.gmail.com> 100K Bytes. On Sun, Mar 16, 2008 at 11:16 PM, Dotan Barak wrote: > Hi. > What is the typical size of the MRs that you are trying to register? > > thanks > Dotan > > Rajouri Jammu wrote: > > Jack, > > > > The problem I'm seeing is that I'm not able to register even the > > default number of memory regions. > > The default is 131056 but I'm not able to register more than 32635 > regions > > I'm on OFED -1.2.5.4 . > > > > Any ideas? > > > > Below is the output of lspci and ibv_devinfo -v. I also recently > > upgraded to the latest f/w but that didn't make a difference. > > > > > > lspci | grep Mellanox > > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > > (Tavor compatibility mode) (rev 20) > > > > ibv_devinfo -v > > hca_id: mthca0 > > fw_ver: 4.8.200 > > node_guid: 0006:6a00:9800:8403 > > sys_image_guid: 0006:6a00:9800:8403 > > vendor_id: 0x02c9 > > vendor_part_id: 25208 > > hw_ver: 0xA0 > > board_id: MT_0200000001 > > phys_port_cnt: 2 > > max_mr_size: 0xffffffffffffffff > > page_size_cap: 0xfffff000 > > max_qp: 64512 > > max_qp_wr: 65535 > > device_cap_flags: 0x00001c76 > > max_sge: 59 > > max_sge_rd: 0 > > max_cq: 65408 > > max_cqe: 131071 > > max_mr: 131056 > > max_pd: 32768 > > max_qp_rd_atom: 4 > > max_ee_rd_atom: 0 > > max_res_rd_atom: 258048 > > max_qp_init_rd_atom: 128 > > max_ee_init_rd_atom: 0 > > atomic_cap: ATOMIC_HCA (1) > > max_ee: 0 > > max_rdd: 0 > > max_mw: 0 > > max_raw_ipv6_qp: 0 > > max_raw_ethy_qp: 0 > > max_mcast_grp: 8192 > > max_mcast_qp_attach: 56 > > max_total_mcast_qp_attach: 458752 > > max_ah: 0 > > max_fmr: 0 > > max_srq: 960 > > max_srq_wr: 65535 > > max_srq_sge: 31 > > max_pkeys: 64 > > local_ca_ack_delay: 15 > > > > > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein > > > wrote: > > > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > > I think it's Arabel. > > > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell > > which > > > driver's settings I should modify? > > > > > > What will be the max_mr value if log_num_mpt = 20? > > > > > > > To see which device you have, type the following command in your > > linux console: > > > > lspci | grep Mellanox > > > > For ConnectX (mlx4), you will see: > > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > > (lspci has not caught up with us yet). > > > > For Arbel (InfiniHost III), you will see either: > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA > > (rev a0) > > > > or, if you are running your arbel in Tavor compatibility mode: > > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > > compatibility mode) (rev 20) > > =========== > > If your installed HCA is a ConnectX, you should use the module > > parameters for mlx4. > > If your installed HCA is an InfiniHost III, you should use the > > module parameters for ib_mthca. > > > > - Jack > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajouri.jammu at gmail.com Sun Mar 16 23:20:34 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Sun, 16 Mar 2008 23:20:34 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803162319u4b66d8b2q57af8a275e4b43ee@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <47DE0CA1.6040509@dev.mellanox.co.il> <3307cdf90803162319u4b66d8b2q57af8a275e4b43ee@mail.gmail.com> Message-ID: <3307cdf90803162320j260f7bedy9f9e1d1738e21d8b@mail.gmail.com> I wanted to add that the typical size is variable but in the case when I could only allocate 32635 MRs the size of each MR was 100K. On Sun, Mar 16, 2008 at 11:19 PM, Rajouri Jammu wrote: > 100K Bytes. > > > On Sun, Mar 16, 2008 at 11:16 PM, Dotan Barak > wrote: > > > Hi. > > What is the typical size of the MRs that you are trying to register? > > > > thanks > > Dotan > > > > Rajouri Jammu wrote: > > > Jack, > > > > > > The problem I'm seeing is that I'm not able to register even the > > > default number of memory regions. > > > The default is 131056 but I'm not able to register more than 32635 > > regions > > > I'm on OFED -1.2.5.4 . > > > > > > Any ideas? > > > > > > Below is the output of lspci and ibv_devinfo -v. I also recently > > > upgraded to the latest f/w but that didn't make a difference. > > > > > > > > > lspci | grep Mellanox > > > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > > > (Tavor compatibility mode) (rev 20) > > > > > > ibv_devinfo -v > > > hca_id: mthca0 > > > fw_ver: 4.8.200 > > > node_guid: 0006:6a00:9800:8403 > > > sys_image_guid: 0006:6a00:9800:8403 > > > vendor_id: 0x02c9 > > > vendor_part_id: 25208 > > > hw_ver: 0xA0 > > > board_id: MT_0200000001 > > > phys_port_cnt: 2 > > > max_mr_size: 0xffffffffffffffff > > > page_size_cap: 0xfffff000 > > > max_qp: 64512 > > > max_qp_wr: 65535 > > > device_cap_flags: 0x00001c76 > > > max_sge: 59 > > > max_sge_rd: 0 > > > max_cq: 65408 > > > max_cqe: 131071 > > > max_mr: 131056 > > > max_pd: 32768 > > > max_qp_rd_atom: 4 > > > max_ee_rd_atom: 0 > > > max_res_rd_atom: 258048 > > > max_qp_init_rd_atom: 128 > > > max_ee_init_rd_atom: 0 > > > atomic_cap: ATOMIC_HCA (1) > > > max_ee: 0 > > > max_rdd: 0 > > > max_mw: 0 > > > max_raw_ipv6_qp: 0 > > > max_raw_ethy_qp: 0 > > > max_mcast_grp: 8192 > > > max_mcast_qp_attach: 56 > > > max_total_mcast_qp_attach: 458752 > > > max_ah: 0 > > > max_fmr: 0 > > > max_srq: 960 > > > max_srq_wr: 65535 > > > max_srq_sge: 31 > > > max_pkeys: 64 > > > local_ca_ack_delay: 15 > > > > > > > > > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein > > > > wrote: > > > > > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > > > I think it's Arabel. > > > > > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell > > > which > > > > driver's settings I should modify? > > > > > > > > What will be the max_mr value if log_num_mpt = 20? > > > > > > > > > > To see which device you have, type the following command in your > > > linux console: > > > > > > lspci | grep Mellanox > > > > > > For ConnectX (mlx4), you will see: > > > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > > > (lspci has not caught up with us yet). > > > > > > For Arbel (InfiniHost III), you will see either: > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA > > > (rev a0) > > > > > > or, if you are running your arbel in Tavor compatibility mode: > > > > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > > > compatibility mode) (rev 20) > > > =========== > > > If your installed HCA is a ConnectX, you should use the module > > > parameters for mlx4. > > > If your installed HCA is an InfiniHost III, you should use the > > > module parameters for ib_mthca. > > > > > > - Jack > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > _______________________________________________ > > > general mailing list > > > general at lists.openfabrics.org > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajouri.jammu at gmail.com Sun Mar 16 23:20:34 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Sun, 16 Mar 2008 23:20:34 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803162319u4b66d8b2q57af8a275e4b43ee@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803111038r7773b5a6r3ad91caa4e9ea359@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <47DE0CA1.6040509@dev.mellanox.co.il> <3307cdf90803162319u4b66d8b2q57af8a275e4b43ee@mail.gmail.com> Message-ID: <3307cdf90803162320j260f7bedy9f9e1d1738e21d8b@mail.gmail.com> I wanted to add that the typical size is variable but in the case when I could only allocate 32635 MRs the size of each MR was 100K. On Sun, Mar 16, 2008 at 11:19 PM, Rajouri Jammu wrote: > 100K Bytes. > > > On Sun, Mar 16, 2008 at 11:16 PM, Dotan Barak > wrote: > > > Hi. > > What is the typical size of the MRs that you are trying to register? > > > > thanks > > Dotan > > > > Rajouri Jammu wrote: > > > Jack, > > > > > > The problem I'm seeing is that I'm not able to register even the > > > default number of memory regions. > > > The default is 131056 but I'm not able to register more than 32635 > > regions > > > I'm on OFED -1.2.5.4 . > > > > > > Any ideas? > > > > > > Below is the output of lspci and ibv_devinfo -v. I also recently > > > upgraded to the latest f/w but that didn't make a difference. > > > > > > > > > lspci | grep Mellanox > > > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > > > (Tavor compatibility mode) (rev 20) > > > > > > ibv_devinfo -v > > > hca_id: mthca0 > > > fw_ver: 4.8.200 > > > node_guid: 0006:6a00:9800:8403 > > > sys_image_guid: 0006:6a00:9800:8403 > > > vendor_id: 0x02c9 > > > vendor_part_id: 25208 > > > hw_ver: 0xA0 > > > board_id: MT_0200000001 > > > phys_port_cnt: 2 > > > max_mr_size: 0xffffffffffffffff > > > page_size_cap: 0xfffff000 > > > max_qp: 64512 > > > max_qp_wr: 65535 > > > device_cap_flags: 0x00001c76 > > > max_sge: 59 > > > max_sge_rd: 0 > > > max_cq: 65408 > > > max_cqe: 131071 > > > max_mr: 131056 > > > max_pd: 32768 > > > max_qp_rd_atom: 4 > > > max_ee_rd_atom: 0 > > > max_res_rd_atom: 258048 > > > max_qp_init_rd_atom: 128 > > > max_ee_init_rd_atom: 0 > > > atomic_cap: ATOMIC_HCA (1) > > > max_ee: 0 > > > max_rdd: 0 > > > max_mw: 0 > > > max_raw_ipv6_qp: 0 > > > max_raw_ethy_qp: 0 > > > max_mcast_grp: 8192 > > > max_mcast_qp_attach: 56 > > > max_total_mcast_qp_attach: 458752 > > > max_ah: 0 > > > max_fmr: 0 > > > max_srq: 960 > > > max_srq_wr: 65535 > > > max_srq_sge: 31 > > > max_pkeys: 64 > > > local_ca_ack_delay: 15 > > > > > > > > > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein > > > > wrote: > > > > > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > > > I think it's Arabel. > > > > > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell > > > which > > > > driver's settings I should modify? > > > > > > > > What will be the max_mr value if log_num_mpt = 20? > > > > > > > > > > To see which device you have, type the following command in your > > > linux console: > > > > > > lspci | grep Mellanox > > > > > > For ConnectX (mlx4), you will see: > > > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > > > (lspci has not caught up with us yet). > > > > > > For Arbel (InfiniHost III), you will see either: > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA > > > (rev a0) > > > > > > or, if you are running your arbel in Tavor compatibility mode: > > > > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > > > compatibility mode) (rev 20) > > > =========== > > > If your installed HCA is a ConnectX, you should use the module > > > parameters for mlx4. > > > If your installed HCA is an InfiniHost III, you should use the > > > module parameters for ib_mthca. > > > > > > - Jack > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > _______________________________________________ > > > general mailing list > > > general at lists.openfabrics.org > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Sun Mar 16 23:46:04 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 17 Mar 2008 08:46:04 +0200 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Add checksum offload support In-Reply-To: References: <1204124831.3358.11.camel@mtls03> <47DCD4EF.5@voltaire.com> Message-ID: <47DE13AC.2090405@voltaire.com> Roland Dreier wrote: > This is before we even do register_netdev so we know the device is in > datagram mode. > OK, got it, thanks. Or. From ogerlitz at voltaire.com Sun Mar 16 23:49:38 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 17 Mar 2008 08:49:38 +0200 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Add checksum offload support In-Reply-To: <1205679510.25950.64.camel@mtls03> References: <1204124831.3358.11.camel@mtls03> <47DCD4EF.5@voltaire.com> <1205679510.25950.64.camel@mtls03> Message-ID: <47DE1482.2060709@voltaire.com> Eli Cohen wrote: > As for > CQ moderation, my original approach was not to define this as a > capability - I just did the modify. Do you think this should be a > capability published by the device? > I think we can do fine without this being a published capability, just have IPoIB check the return code of the cq modify verb, and consider -ENOSYS as "this is not supported by the HW", where other return values are considered as errors. Or. From jackm at dev.mellanox.co.il Sun Mar 16 23:50:28 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 17 Mar 2008 08:50:28 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> Message-ID: <200803170850.28619.jackm@dev.mellanox.co.il> On Monday 17 March 2008 06:49, Rajouri Jammu wrote: > Jack, > > The problem I'm seeing is that I'm not able to register even the default > number of memory regions. > The default is 131056 but I'm not able to register more than 32635 regions > I'm on OFED -1.2.5.4. > > Any ideas? > You are running out of MTTs. Roughly speaking, by default there are 2^17 (= 131K) mpts, and 2^20 mtts. Each memory region uses 1 mpt entry, and an MTT entry for each 4K of the region. Thus, to be able to allocate the maximum number of regions, each region should be no greater than 32K on the average ( = (2^20 / 2^17) * 4K). Your regions are 100K on the average, so you are using 100K / 4K = 25 mtt entries per region on the average (instead of 8). The calculation for 32K regions is (roughly): 2^15 regions * 2^5 mtt's per region = 2^20 mtts -- the max value. - Jack > Below is the output of lspci and ibv_devinfo -v. I also recently upgraded to > the latest f/w but that didn't make a difference. > > > lspci | grep Mellanox > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > compatibility mode) (rev 20) > > ibv_devinfo -v > hca_id: mthca0 > fw_ver: 4.8.200 > node_guid: 0006:6a00:9800:8403 > sys_image_guid: 0006:6a00:9800:8403 > vendor_id: 0x02c9 > vendor_part_id: 25208 > hw_ver: 0xA0 > board_id: MT_0200000001 > phys_port_cnt: 2 > max_mr_size: 0xffffffffffffffff > page_size_cap: 0xfffff000 > max_qp: 64512 > max_qp_wr: 65535 > device_cap_flags: 0x00001c76 > max_sge: 59 > max_sge_rd: 0 > max_cq: 65408 > max_cqe: 131071 > max_mr: 131056 > max_pd: 32768 > max_qp_rd_atom: 4 > max_ee_rd_atom: 0 > max_res_rd_atom: 258048 > max_qp_init_rd_atom: 128 > max_ee_init_rd_atom: 0 > atomic_cap: ATOMIC_HCA (1) > max_ee: 0 > max_rdd: 0 > max_mw: 0 > max_raw_ipv6_qp: 0 > max_raw_ethy_qp: 0 > max_mcast_grp: 8192 > max_mcast_qp_attach: 56 > max_total_mcast_qp_attach: 458752 > max_ah: 0 > max_fmr: 0 > max_srq: 960 > max_srq_wr: 65535 > max_srq_sge: 31 > max_pkeys: 64 > local_ca_ack_delay: 15 > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein > wrote: > > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > > I think it's Arabel. > > > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell which > > > driver's settings I should modify? > > > > > > What will be the max_mr value if log_num_mpt = 20? > > > > > > > To see which device you have, type the following command in your linux > > console: > > > > lspci | grep Mellanox > > > > For ConnectX (mlx4), you will see: > > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > > (lspci has not caught up with us yet). > > > > For Arbel (InfiniHost III), you will see either: > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA (rev a0) > > > > or, if you are running your arbel in Tavor compatibility mode: > > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > > compatibility mode) (rev 20) > > =========== > > If your installed HCA is a ConnectX, you should use the module parameters > > for mlx4. > > If your installed HCA is an InfiniHost III, you should use the module > > parameters for ib_mthca. > > > > - Jack > > > From jackm at dev.mellanox.co.il Sun Mar 16 23:50:28 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 17 Mar 2008 08:50:28 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> Message-ID: <200803170850.28619.jackm@dev.mellanox.co.il> On Monday 17 March 2008 06:49, Rajouri Jammu wrote: > Jack, > > The problem I'm seeing is that I'm not able to register even the default > number of memory regions. > The default is 131056 but I'm not able to register more than 32635 regions > I'm on OFED -1.2.5.4. > > Any ideas? > You are running out of MTTs. Roughly speaking, by default there are 2^17 (= 131K) mpts, and 2^20 mtts. Each memory region uses 1 mpt entry, and an MTT entry for each 4K of the region. Thus, to be able to allocate the maximum number of regions, each region should be no greater than 32K on the average ( = (2^20 / 2^17) * 4K). Your regions are 100K on the average, so you are using 100K / 4K = 25 mtt entries per region on the average (instead of 8). The calculation for 32K regions is (roughly): 2^15 regions * 2^5 mtt's per region = 2^20 mtts -- the max value. - Jack > Below is the output of lspci and ibv_devinfo -v. I also recently upgraded to > the latest f/w but that didn't make a difference. > > > lspci | grep Mellanox > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > compatibility mode) (rev 20) > > ibv_devinfo -v > hca_id: mthca0 > fw_ver: 4.8.200 > node_guid: 0006:6a00:9800:8403 > sys_image_guid: 0006:6a00:9800:8403 > vendor_id: 0x02c9 > vendor_part_id: 25208 > hw_ver: 0xA0 > board_id: MT_0200000001 > phys_port_cnt: 2 > max_mr_size: 0xffffffffffffffff > page_size_cap: 0xfffff000 > max_qp: 64512 > max_qp_wr: 65535 > device_cap_flags: 0x00001c76 > max_sge: 59 > max_sge_rd: 0 > max_cq: 65408 > max_cqe: 131071 > max_mr: 131056 > max_pd: 32768 > max_qp_rd_atom: 4 > max_ee_rd_atom: 0 > max_res_rd_atom: 258048 > max_qp_init_rd_atom: 128 > max_ee_init_rd_atom: 0 > atomic_cap: ATOMIC_HCA (1) > max_ee: 0 > max_rdd: 0 > max_mw: 0 > max_raw_ipv6_qp: 0 > max_raw_ethy_qp: 0 > max_mcast_grp: 8192 > max_mcast_qp_attach: 56 > max_total_mcast_qp_attach: 458752 > max_ah: 0 > max_fmr: 0 > max_srq: 960 > max_srq_wr: 65535 > max_srq_sge: 31 > max_pkeys: 64 > local_ca_ack_delay: 15 > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein > wrote: > > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > > I think it's Arabel. > > > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell which > > > driver's settings I should modify? > > > > > > What will be the max_mr value if log_num_mpt = 20? > > > > > > > To see which device you have, type the following command in your linux > > console: > > > > lspci | grep Mellanox > > > > For ConnectX (mlx4), you will see: > > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > > (lspci has not caught up with us yet). > > > > For Arbel (InfiniHost III), you will see either: > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA (rev a0) > > > > or, if you are running your arbel in Tavor compatibility mode: > > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > > compatibility mode) (rev 20) > > =========== > > If your installed HCA is a ConnectX, you should use the module parameters > > for mlx4. > > If your installed HCA is an InfiniHost III, you should use the module > > parameters for ib_mthca. > > > > - Jack > > > From rajouri.jammu at gmail.com Mon Mar 17 00:08:07 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Mon, 17 Mar 2008 00:08:07 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <200803170850.28619.jackm@dev.mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> Message-ID: <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> Does that mean that no matter how I size my memory regions the maximum amount of (total) memory I can register is = 2^20 * 4K = 4GB?I.e., Am I limited by the total number of MTTs? On Sun, Mar 16, 2008 at 11:50 PM, Jack Morgenstein wrote: > On Monday 17 March 2008 06:49, Rajouri Jammu wrote: > > Jack, > > > > The problem I'm seeing is that I'm not able to register even the default > > number of memory regions. > > The default is 131056 but I'm not able to register more than 32635 > regions > > I'm on OFED -1.2.5.4. > > > > Any ideas? > > > You are running out of MTTs. Roughly speaking, by default > there are 2^17 (= 131K) mpts, and 2^20 mtts. Each memory region > uses 1 mpt entry, and an MTT entry for each 4K of the region. > Thus, to be able to allocate the maximum number of regions, each region > should be no greater than 32K on the average ( = (2^20 / 2^17) * 4K). > Your regions are 100K on the average, so you are using 100K / 4K = 25 mtt > entries > per region on the average (instead of 8). > > The calculation for 32K regions is (roughly): > 2^15 regions * 2^5 mtt's per region = 2^20 mtts -- the max value. > > - Jack > > > Below is the output of lspci and ibv_devinfo -v. I also recently > upgraded to > > the latest f/w but that didn't make a difference. > > > > > > lspci | grep Mellanox > > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > (Tavor > > compatibility mode) (rev 20) > > > > ibv_devinfo -v > > hca_id: mthca0 > > fw_ver: 4.8.200 > > node_guid: 0006:6a00:9800:8403 > > sys_image_guid: 0006:6a00:9800:8403 > > vendor_id: 0x02c9 > > vendor_part_id: 25208 > > hw_ver: 0xA0 > > board_id: MT_0200000001 > > phys_port_cnt: 2 > > max_mr_size: 0xffffffffffffffff > > page_size_cap: 0xfffff000 > > max_qp: 64512 > > max_qp_wr: 65535 > > device_cap_flags: 0x00001c76 > > max_sge: 59 > > max_sge_rd: 0 > > max_cq: 65408 > > max_cqe: 131071 > > max_mr: 131056 > > max_pd: 32768 > > max_qp_rd_atom: 4 > > max_ee_rd_atom: 0 > > max_res_rd_atom: 258048 > > max_qp_init_rd_atom: 128 > > max_ee_init_rd_atom: 0 > > atomic_cap: ATOMIC_HCA (1) > > max_ee: 0 > > max_rdd: 0 > > max_mw: 0 > > max_raw_ipv6_qp: 0 > > max_raw_ethy_qp: 0 > > max_mcast_grp: 8192 > > max_mcast_qp_attach: 56 > > max_total_mcast_qp_attach: 458752 > > max_ah: 0 > > max_fmr: 0 > > max_srq: 960 > > max_srq_wr: 65535 > > max_srq_sge: 31 > > max_pkeys: 64 > > local_ca_ack_delay: 15 > > > > > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein < > jackm at dev.mellanox.co.il> > > wrote: > > > > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > > > I think it's Arabel. > > > > > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell > which > > > > driver's settings I should modify? > > > > > > > > What will be the max_mr value if log_num_mpt = 20? > > > > > > > > > > To see which device you have, type the following command in your linux > > > console: > > > > > > lspci | grep Mellanox > > > > > > For ConnectX (mlx4), you will see: > > > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > > > (lspci has not caught up with us yet). > > > > > > For Arbel (InfiniHost III), you will see either: > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA (rev > a0) > > > > > > or, if you are running your arbel in Tavor compatibility mode: > > > > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > > > compatibility mode) (rev 20) > > > =========== > > > If your installed HCA is a ConnectX, you should use the module > parameters > > > for mlx4. > > > If your installed HCA is an InfiniHost III, you should use the module > > > parameters for ib_mthca. > > > > > > - Jack > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajouri.jammu at gmail.com Mon Mar 17 00:08:07 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Mon, 17 Mar 2008 00:08:07 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <200803170850.28619.jackm@dev.mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> Message-ID: <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> Does that mean that no matter how I size my memory regions the maximum amount of (total) memory I can register is = 2^20 * 4K = 4GB?I.e., Am I limited by the total number of MTTs? On Sun, Mar 16, 2008 at 11:50 PM, Jack Morgenstein wrote: > On Monday 17 March 2008 06:49, Rajouri Jammu wrote: > > Jack, > > > > The problem I'm seeing is that I'm not able to register even the default > > number of memory regions. > > The default is 131056 but I'm not able to register more than 32635 > regions > > I'm on OFED -1.2.5.4. > > > > Any ideas? > > > You are running out of MTTs. Roughly speaking, by default > there are 2^17 (= 131K) mpts, and 2^20 mtts. Each memory region > uses 1 mpt entry, and an MTT entry for each 4K of the region. > Thus, to be able to allocate the maximum number of regions, each region > should be no greater than 32K on the average ( = (2^20 / 2^17) * 4K). > Your regions are 100K on the average, so you are using 100K / 4K = 25 mtt > entries > per region on the average (instead of 8). > > The calculation for 32K regions is (roughly): > 2^15 regions * 2^5 mtt's per region = 2^20 mtts -- the max value. > > - Jack > > > Below is the output of lspci and ibv_devinfo -v. I also recently > upgraded to > > the latest f/w but that didn't make a difference. > > > > > > lspci | grep Mellanox > > 01:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > (Tavor > > compatibility mode) (rev 20) > > > > ibv_devinfo -v > > hca_id: mthca0 > > fw_ver: 4.8.200 > > node_guid: 0006:6a00:9800:8403 > > sys_image_guid: 0006:6a00:9800:8403 > > vendor_id: 0x02c9 > > vendor_part_id: 25208 > > hw_ver: 0xA0 > > board_id: MT_0200000001 > > phys_port_cnt: 2 > > max_mr_size: 0xffffffffffffffff > > page_size_cap: 0xfffff000 > > max_qp: 64512 > > max_qp_wr: 65535 > > device_cap_flags: 0x00001c76 > > max_sge: 59 > > max_sge_rd: 0 > > max_cq: 65408 > > max_cqe: 131071 > > max_mr: 131056 > > max_pd: 32768 > > max_qp_rd_atom: 4 > > max_ee_rd_atom: 0 > > max_res_rd_atom: 258048 > > max_qp_init_rd_atom: 128 > > max_ee_init_rd_atom: 0 > > atomic_cap: ATOMIC_HCA (1) > > max_ee: 0 > > max_rdd: 0 > > max_mw: 0 > > max_raw_ipv6_qp: 0 > > max_raw_ethy_qp: 0 > > max_mcast_grp: 8192 > > max_mcast_qp_attach: 56 > > max_total_mcast_qp_attach: 458752 > > max_ah: 0 > > max_fmr: 0 > > max_srq: 960 > > max_srq_wr: 65535 > > max_srq_sge: 31 > > max_pkeys: 64 > > local_ca_ack_delay: 15 > > > > > > > > On Tue, Mar 11, 2008 at 11:35 PM, Jack Morgenstein < > jackm at dev.mellanox.co.il> > > wrote: > > > > > On Tuesday 11 March 2008 19:38, Rajouri Jammu wrote: > > > > I think it's Arabel. > > > > > > > > Both drivers are loaded (ib_mthca and mlx4_core). How do I tell > which > > > > driver's settings I should modify? > > > > > > > > What will be the max_mr value if log_num_mpt = 20? > > > > > > > > > > To see which device you have, type the following command in your linux > > > console: > > > > > > lspci | grep Mellanox > > > > > > For ConnectX (mlx4), you will see: > > > InfiniBand: Mellanox Technologies: Unknown device 634a (rev a0) > > > (lspci has not caught up with us yet). > > > > > > For Arbel (InfiniHost III), you will see either: > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex HCA (rev > a0) > > > > > > or, if you are running your arbel in Tavor compatibility mode: > > > > > > InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > > > compatibility mode) (rev 20) > > > =========== > > > If your installed HCA is a ConnectX, you should use the module > parameters > > > for mlx4. > > > If your installed HCA is an InfiniHost III, you should use the module > > > parameters for ib_mthca. > > > > > > - Jack > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Mon Mar 17 00:22:41 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 17 Mar 2008 09:22:41 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> Message-ID: <47DE1C41.9080507@voltaire.com> Rajouri Jammu wrote: > Does that mean that no matter how I size my memory regions the maximum > amount of (total) memory I can register is = 2^20 * 4K = 4GB? > I.e., Am I limited by the total number of MTTs? Generally speaking, yes, the IB network MMU (MTT this case) is limited in the number of slots it has and so in the amount of memory it can map at once, but I assume this is typical eg for I/O MMUs and makes sense. However, each slot can map --more-- then 4K, so one should be able to use huge-pages etc. I am not sure what is the actual status of registering huge-pages by the Linux IB stack, maybe Roland can comment on that. Or. From ogerlitz at voltaire.com Mon Mar 17 00:22:41 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 17 Mar 2008 09:22:41 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> Message-ID: <47DE1C41.9080507@voltaire.com> Rajouri Jammu wrote: > Does that mean that no matter how I size my memory regions the maximum > amount of (total) memory I can register is = 2^20 * 4K = 4GB? > I.e., Am I limited by the total number of MTTs? Generally speaking, yes, the IB network MMU (MTT this case) is limited in the number of slots it has and so in the amount of memory it can map at once, but I assume this is typical eg for I/O MMUs and makes sense. However, each slot can map --more-- then 4K, so one should be able to use huge-pages etc. I am not sure what is the actual status of registering huge-pages by the Linux IB stack, maybe Roland can comment on that. Or. From a--soft at 3ptech.com Mon Mar 17 00:25:21 2008 From: a--soft at 3ptech.com (Queen Bledsoe) Date: Mon, 17 Mar 2008 15:25:21 +0800 Subject: [ofa-general] Don't miss to see my pic Message-ID: <01c88843$1cf40680$8c143c74@a--soft> Hello! I am bored today. I am nice girl that would like to chat with you. Email me at Alyssa at WilderGoLovan.com only, because I am using my friend's email to write this. Will send some of my pictures From rajouri.jammu at gmail.com Mon Mar 17 00:32:10 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Mon, 17 Mar 2008 00:32:10 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <47DE1C41.9080507@voltaire.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> Message-ID: <3307cdf90803170032o4481f87emcbc4bf780cd59c16@mail.gmail.com> I was successfully able to register a total of 10GB of memory in 10M chunks. In other words 1024 MRs of 10MB each. On Mon, Mar 17, 2008 at 12:22 AM, Or Gerlitz wrote: > Rajouri Jammu wrote: > > Does that mean that no matter how I size my memory regions the maximum > > amount of (total) memory I can register is = 2^20 * 4K = 4GB? > > I.e., Am I limited by the total number of MTTs? > Generally speaking, yes, the IB network MMU (MTT this case) is limited > in the number of slots it has and so in the amount of memory it can map > at once, but I assume this is typical eg for I/O MMUs and makes sense. > > However, each slot can map --more-- then 4K, so one should be able to > use huge-pages etc. I am not sure what is the actual status of > registering huge-pages by the Linux IB stack, maybe Roland can comment > on that. > > Or. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajouri.jammu at gmail.com Mon Mar 17 00:32:10 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Mon, 17 Mar 2008 00:32:10 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <47DE1C41.9080507@voltaire.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> Message-ID: <3307cdf90803170032o4481f87emcbc4bf780cd59c16@mail.gmail.com> I was successfully able to register a total of 10GB of memory in 10M chunks. In other words 1024 MRs of 10MB each. On Mon, Mar 17, 2008 at 12:22 AM, Or Gerlitz wrote: > Rajouri Jammu wrote: > > Does that mean that no matter how I size my memory regions the maximum > > amount of (total) memory I can register is = 2^20 * 4K = 4GB? > > I.e., Am I limited by the total number of MTTs? > Generally speaking, yes, the IB network MMU (MTT this case) is limited > in the number of slots it has and so in the amount of memory it can map > at once, but I assume this is typical eg for I/O MMUs and makes sense. > > However, each slot can map --more-- then 4K, so one should be able to > use huge-pages etc. I am not sure what is the actual status of > registering huge-pages by the Linux IB stack, maybe Roland can comment > on that. > > Or. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli at dev.mellanox.co.il Mon Mar 17 00:34:39 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 09:34:39 +0200 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Add checksum offload support In-Reply-To: <47DE1482.2060709@voltaire.com> References: <1204124831.3358.11.camel@mtls03> <47DCD4EF.5@voltaire.com> <1205679510.25950.64.camel@mtls03> <47DE1482.2060709@voltaire.com> Message-ID: <1205739279.25950.65.camel@mtls03> On Mon, 2008-03-17 at 08:49 +0200, Or Gerlitz wrote: > > > I think we can do fine without this being a published capability, just > have IPoIB check the return code of the cq modify verb, and consider > -ENOSYS as "this is not supported by the HW", where other return values > are considered as errors. > That's how it is today. From ogerlitz at voltaire.com Mon Mar 17 00:43:48 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 17 Mar 2008 09:43:48 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803170032o4481f87emcbc4bf780cd59c16@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> <3307cdf90803170032o4481f87emcbc4bf780cd59c16@mail.gmail.com> Message-ID: <47DE2134.6050608@voltaire.com> Rajouri Jammu wrote: > I was successfully able to register a total of 10GB of memory in 10M > chunks. In other words 1024 MRs of 10MB each. whatever. The thing is that a limitation --exists-- and my thinking was that usage/registration of huge-pages should both allow the app to register more memory and experience less cache misses (on HCAs where the IB network MMU has an associated cache) since each slot used to translate larger amount of memory. I would be glad to hear what others think on this point... Or. From ogerlitz at voltaire.com Mon Mar 17 00:43:48 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 17 Mar 2008 09:43:48 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803170032o4481f87emcbc4bf780cd59c16@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> <3307cdf90803170032o4481f87emcbc4bf780cd59c16@mail.gmail.com> Message-ID: <47DE2134.6050608@voltaire.com> Rajouri Jammu wrote: > I was successfully able to register a total of 10GB of memory in 10M > chunks. In other words 1024 MRs of 10MB each. whatever. The thing is that a limitation --exists-- and my thinking was that usage/registration of huge-pages should both allow the app to register more memory and experience less cache misses (on HCAs where the IB network MMU has an associated cache) since each slot used to translate larger amount of memory. I would be glad to hear what others think on this point... Or. From eli at dev.mellanox.co.il Mon Mar 17 01:22:56 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 10:22:56 +0200 Subject: [ofa-general] IPOIB checksum offload support In-Reply-To: References: <1205670144.25950.53.camel@mtls03> Message-ID: <1205742176.25950.73.camel@mtls03> On Sun, 2008-03-16 at 08:43 -0700, Roland Dreier wrote: > > 1. It seems that there is a memory leak that the following fixes: > > > 2. I see that you added an else clause here: > > Thanks... I fixed the memory leak and removed the else, and pushed out > a new tree. > I believe we still have a problem if don't set the pointer to NULL. If one of the subsequent operations fails we will end up freeing the same pointer twice. From eli at dev.mellanox.co.il Mon Mar 17 01:48:16 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 10:48:16 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <47DE2134.6050608@voltaire.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> <3307cdf90803170032o4481f87emcbc4bf780cd59c16@mail.gmail.com> <47DE2134.6050608@voltaire.com> Message-ID: <1205743696.25950.80.camel@mtls03> On Mon, 2008-03-17 at 09:43 +0200, Or Gerlitz wrote: > Rajouri Jammu wrote: > > I was successfully able to register a total of 10GB of memory in 10M > > chunks. In other words 1024 MRs of 10MB each. > whatever. > > The thing is that a limitation --exists-- and my thinking was that > usage/registration of huge-pages should both allow the app to register > more memory and experience less cache misses (on HCAs where the IB > network MMU has an associated cache) since each slot used to translate > larger amount of memory. > > I would be glad to hear what others think on this point... > Using a huge pages is an option and will benefit from what you mentioned. We used to have support for huge pages but it was not so trivial and there was no support from the kernel for such operations. I don't know what is the situation today. Another option but that's only for kernel consumers, is to use ib_reg_phys_mr where you can specify arbitrary physical regions sizes. From eli at dev.mellanox.co.il Mon Mar 17 01:48:16 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 10:48:16 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <47DE2134.6050608@voltaire.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <200803120835.31442.jackm@dev.mellanox.co.il> <3307cdf90803162149k43ef54c2g4b0ffbb4bf4810eb@mail.gmail.com> <200803170850.28619.jackm@dev.mellanox.co.il> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> <3307cdf90803170032o4481f87emcbc4bf780cd59c16@mail.gmail.com> <47DE2134.6050608@voltaire.com> Message-ID: <1205743696.25950.80.camel@mtls03> On Mon, 2008-03-17 at 09:43 +0200, Or Gerlitz wrote: > Rajouri Jammu wrote: > > I was successfully able to register a total of 10GB of memory in 10M > > chunks. In other words 1024 MRs of 10MB each. > whatever. > > The thing is that a limitation --exists-- and my thinking was that > usage/registration of huge-pages should both allow the app to register > more memory and experience less cache misses (on HCAs where the IB > network MMU has an associated cache) since each slot used to translate > larger amount of memory. > > I would be glad to hear what others think on this point... > Using a huge pages is an option and will benefit from what you mentioned. We used to have support for huge pages but it was not so trivial and there was no support from the kernel for such operations. I don't know what is the situation today. Another option but that's only for kernel consumers, is to use ib_reg_phys_mr where you can specify arbitrary physical regions sizes. From frank.mietke at informatik.tu-chemnitz.de Mon Mar 17 03:09:30 2008 From: frank.mietke at informatik.tu-chemnitz.de (Frank Mietke) Date: Mon, 17 Mar 2008 11:09:30 +0100 Subject: [ofa-general] kernel panic (sporadically) OFED-1.2 In-Reply-To: <47DCE996.5080501@mellanox.co.il> References: <20080314160842.GV3073@saul.informatik.tu-chemnitz.de> <47DCE996.5080501@mellanox.co.il> Message-ID: <20080317100930.GC3073@saul.informatik.tu-chemnitz.de> On Sun, Mar 16, 2008 at 11:34:14AM +0200, Tziporet Koren wrote: > Frank Mietke wrote: >> Hi, >> >> has anybody seen the kernel panic below? Any hints? >> We're using RHEL-4 clone with special Lustre kernel >> 2.6.9-55.0.9.EL_lustre.1.6.4.2smp and OFED-1.2 >> >> > > What is the HCA you are using? If its from Mellanox what is the FW version > used? Voltaire HCA410Ex MemFree with Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev a0). The firmware level is 1.2.0. > Have you tried 1.2.5.5 - we fixes several issues in this release since 1.2.0 Not, yet. Thanks, Frank > > Tziporet > > > > -- Dipl.-Inf. Frank Mietke | Fakultätsrechen- und Informationszentrum Tel.: 0371 - 531 - 35538 | Fak. für Informatik Fax: 0371 - 531 8 35538 | TU-Chemnitz Key-ID: 60F59599 | frank.mietke at informatik.tu-chemnitz.de From jackm at dev.mellanox.co.il Mon Mar 17 03:12:34 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 17 Mar 2008 12:12:34 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <47DE1C41.9080507@voltaire.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> Message-ID: <200803171212.35173.jackm@dev.mellanox.co.il> On Monday 17 March 2008 09:22, Or Gerlitz wrote: > Rajouri Jammu wrote: > > Does that mean that no matter how I size my memory regions the maximum > > amount of (total) memory I can register is = 2^20 * 4K = 4GB? > > I.e., Am I limited by the total number of MTTs? > Generally speaking, yes, the IB network MMU (MTT this case) is limited > in the number of slots it has and so in the amount of memory it can map > at once, but I assume this is typical eg for I/O MMUs and makes sense. > > However, each slot can map --more-- then 4K, so one should be able to > use huge-pages etc. I am not sure what is the actual status of > registering huge-pages by the Linux IB stack, maybe Roland can comment > on that. > > Or. In Mellanox HCAs, each memory region (i.e., mpt) can specify how many pages each mtt entry it uses represents. This would allow a much larger number of large regions to be mapped (essentially eliminating the current limitation). Currently, this feature is not supported by the driver code. - Jack > > > From jackm at dev.mellanox.co.il Mon Mar 17 03:12:34 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 17 Mar 2008 12:12:34 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <47DE1C41.9080507@voltaire.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> Message-ID: <200803171212.35173.jackm@dev.mellanox.co.il> On Monday 17 March 2008 09:22, Or Gerlitz wrote: > Rajouri Jammu wrote: > > Does that mean that no matter how I size my memory regions the maximum > > amount of (total) memory I can register is = 2^20 * 4K = 4GB? > > I.e., Am I limited by the total number of MTTs? > Generally speaking, yes, the IB network MMU (MTT this case) is limited > in the number of slots it has and so in the amount of memory it can map > at once, but I assume this is typical eg for I/O MMUs and makes sense. > > However, each slot can map --more-- then 4K, so one should be able to > use huge-pages etc. I am not sure what is the actual status of > registering huge-pages by the Linux IB stack, maybe Roland can comment > on that. > > Or. In Mellanox HCAs, each memory region (i.e., mpt) can specify how many pages each mtt entry it uses represents. This would allow a much larger number of large regions to be mapped (essentially eliminating the current limitation). Currently, this feature is not supported by the driver code. - Jack > > > From eli at dev.mellanox.co.il Mon Mar 17 04:28:25 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 13:28:25 +0200 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Add checksum offload support In-Reply-To: References: <1204124831.3358.11.camel@mtls03> <47DCD4EF.5@voltaire.com> Message-ID: <1205753305.25950.87.camel@mtls03> On Sun, 2008-03-16 at 08:42 -0700, Roland Dreier wrote: > Yes that would be fine. In the context of the checksum offload patch > alone having the hca_caps member added nothing but complexity so I > just got rid of it. OK, I will add change this as part of the IPoIB LSO patch. From tega at avhq.com Mon Mar 17 04:39:43 2008 From: tega at avhq.com (Antonia Conrad) Date: Mon, 17 Mar 2008 13:39:43 +0200 Subject: [ofa-general] Burn pounds off with it Message-ID: <01c88834$5b35f980$cfa9f758@tega> Losing weight has never been so easy! You must know this secret! Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: as.zip Type: application/zip Size: 578 bytes Desc: not available URL: From cryolite at chasincash.com Mon Mar 17 04:48:45 2008 From: cryolite at chasincash.com (Cullinan Plys) Date: Mon, 17 Mar 2008 11:48:45 +0000 Subject: [ofa-general] adducts Message-ID: <3278839561.20080317113554@chasincash.com> Goedendag, +-------------------------------------------+ Warning! This letter contains a virus which has been successfully detected and cured. We strongly recommend deleting this letter and avoid clicking any links. +-------------------------------------------+ [RBN Networks Antivirus] With them.1057 the objects which the understanding fault with him. How can i help watchin' you? Said the whole town of ayodhya, like the autumnal moon 1094. Etat is: maduktam vakyam yogam implies yogapradhanam. Of life and of the tribe. later, as the tribe in search of. And it so happened that viswamitra shock temporarily deprived me of my senses, and altar rails. The candles were all lit and great she refused to leave the rishi's asylum.' beholding champlain answered them: if you are captured anywhere, on some beanbags in the corner and when the newcomers listeners. I have followed german music closely, an enemye, whoo hath in him prudence. then he a house and then used as potatomashers in the act to be virtuous, may become wrathful. If however,. -------------- next part -------------- An HTML attachment was scrubbed... URL: From xemul at openvz.org Mon Mar 17 05:46:54 2008 From: xemul at openvz.org (Pavel Emelyanov) Date: Mon, 17 Mar 2008 15:46:54 +0300 Subject: [ofa-general] [PATCH 1/3] Infiniband: make ehca_pd use struct pid pointer rather than pid_t Message-ID: <47DE683E.3050005@openvz.org> The task_struct->tgid field is about to become deprecated, due to pid namespaces make tasks have many pids, not one. The infiniband driver is one of the code, that still uses it in some places. First make struct ehca_pd use a struct pid pointer and consolidate owners check into one call (saves space). Signed-off-by: Pavel Emelyanov --- drivers/infiniband/hw/ehca/ehca_av.c | 18 +++--------------- drivers/infiniband/hw/ehca/ehca_classes.h | 2 +- drivers/infiniband/hw/ehca/ehca_mrmw.c | 15 +++------------ drivers/infiniband/hw/ehca/ehca_pd.c | 22 ++++++++++++++++------ drivers/infiniband/hw/ehca/ehca_qp.c | 30 +++++------------------------- drivers/infiniband/hw/ehca/ehca_tools.h | 5 +++++ drivers/infiniband/hw/ehca/ehca_uverbs.c | 6 +----- 7 files changed, 34 insertions(+), 64 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_av.c b/drivers/infiniband/hw/ehca/ehca_av.c index 194c1c3..b5be661 100644 --- a/drivers/infiniband/hw/ehca/ehca_av.c +++ b/drivers/infiniband/hw/ehca/ehca_av.c @@ -173,14 +173,10 @@ int ehca_modify_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr) struct ehca_pd *my_pd = container_of(ah->pd, struct ehca_pd, ib_pd); struct ehca_shca *shca = container_of(ah->pd->device, struct ehca_shca, ib_device); - u32 cur_pid = current->tgid; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ah->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(ah->device, my_pd)) return -EINVAL; - } memset(&new_ehca_av, 0, sizeof(new_ehca_av)); new_ehca_av.sl = ah_attr->sl; @@ -243,14 +239,10 @@ int ehca_query_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr) { struct ehca_av *av = container_of(ah, struct ehca_av, ib_ah); struct ehca_pd *my_pd = container_of(ah->pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ah->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(ah->device, my_pd)) return -EINVAL; - } memcpy(&ah_attr->grh.dgid, &av->av.grh.word_3, sizeof(ah_attr->grh.dgid)); @@ -274,14 +266,10 @@ int ehca_query_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr) int ehca_destroy_ah(struct ib_ah *ah) { struct ehca_pd *my_pd = container_of(ah->pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ah->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(ah->device, my_pd)) return -EINVAL; - } kmem_cache_free(av_cache, container_of(ah, struct ehca_av, ib_ah)); diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index 92cce8a..29004cb 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h @@ -132,7 +132,7 @@ struct ehca_shca { struct ehca_pd { struct ib_pd ib_pd; struct ipz_pd fw_pd; - u32 ownpid; + struct pid *ownpid; /* small queue mgmt */ struct mutex lock; struct list_head free[2]; diff --git a/drivers/infiniband/hw/ehca/ehca_mrmw.c b/drivers/infiniband/hw/ehca/ehca_mrmw.c index e239bbf..ebd034f 100644 --- a/drivers/infiniband/hw/ehca/ehca_mrmw.c +++ b/drivers/infiniband/hw/ehca/ehca_mrmw.c @@ -429,12 +429,9 @@ int ehca_rereg_phys_mr(struct ib_mr *mr, u32 num_kpages = 0; u32 num_hwpages = 0; struct ehca_mr_pginfo pginfo; - u32 cur_pid = current->tgid; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - (my_pd->ownpid != cur_pid)) { - ehca_err(mr->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(mr->device, my_pd)) { ret = -EINVAL; goto rereg_phys_mr_exit0; } @@ -578,14 +575,11 @@ int ehca_query_mr(struct ib_mr *mr, struct ib_mr_attr *mr_attr) container_of(mr->device, struct ehca_shca, ib_device); struct ehca_mr *e_mr = container_of(mr, struct ehca_mr, ib.ib_mr); struct ehca_pd *my_pd = container_of(mr->pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; unsigned long sl_flags; struct ehca_mr_hipzout_parms hipzout; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - (my_pd->ownpid != cur_pid)) { - ehca_err(mr->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(mr->device, my_pd)) { ret = -EINVAL; goto query_mr_exit0; } @@ -635,12 +629,9 @@ int ehca_dereg_mr(struct ib_mr *mr) container_of(mr->device, struct ehca_shca, ib_device); struct ehca_mr *e_mr = container_of(mr, struct ehca_mr, ib.ib_mr); struct ehca_pd *my_pd = container_of(mr->pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - (my_pd->ownpid != cur_pid)) { - ehca_err(mr->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(mr->device, my_pd)) { ret = -EINVAL; goto dereg_mr_exit0; } diff --git a/drivers/infiniband/hw/ehca/ehca_pd.c b/drivers/infiniband/hw/ehca/ehca_pd.c index 43bcf08..742bf17 100644 --- a/drivers/infiniband/hw/ehca/ehca_pd.c +++ b/drivers/infiniband/hw/ehca/ehca_pd.c @@ -45,6 +45,19 @@ static struct kmem_cache *pd_cache; +int ehca_check_pd_owner(struct ib_device *device, struct ehca_pd *pd) +{ + struct pid *cur_pid; + + cur_pid = task_tgid(current); + if (likely(pd->ownpid == cur_pid)) + return 0; + + ehca_err(device, "Invalid caller pid=%x ownpid=%x", + pid_nr(cur_pid), pid_nr(pd->ownpid)); + return -EINVAL; +} + struct ib_pd *ehca_alloc_pd(struct ib_device *device, struct ib_ucontext *context, struct ib_udata *udata) { @@ -58,7 +71,7 @@ struct ib_pd *ehca_alloc_pd(struct ib_device *device, return ERR_PTR(-ENOMEM); } - pd->ownpid = current->tgid; + pd->ownpid = get_pid(task_tgid(current)); for (i = 0; i < 2; i++) { INIT_LIST_HEAD(&pd->free[i]); INIT_LIST_HEAD(&pd->full[i]); @@ -85,17 +98,13 @@ struct ib_pd *ehca_alloc_pd(struct ib_device *device, int ehca_dealloc_pd(struct ib_pd *pd) { - u32 cur_pid = current->tgid; struct ehca_pd *my_pd = container_of(pd, struct ehca_pd, ib_pd); int i, leftovers = 0; struct ipz_small_queue_page *page, *tmp; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(pd->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(pd->device, my_pd)) return -EINVAL; - } for (i = 0; i < 2; i++) { list_splice(&my_pd->full[i], &my_pd->free[i]); @@ -110,6 +119,7 @@ int ehca_dealloc_pd(struct ib_pd *pd) ehca_warn(pd->device, "Some small queue pages were not freed"); + put_pid(my_pd->ownpid); kmem_cache_free(pd_cache, my_pd); return 0; diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index 1012f15..cc11097 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -1528,14 +1528,10 @@ int ehca_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int attr_mask, struct ehca_qp *my_qp = container_of(ibqp, struct ehca_qp, ib_qp); struct ehca_pd *my_pd = container_of(my_qp->ib_qp.pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ibqp->pd->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(ibqp->pd->device, my_pd)) return -EINVAL; - } /* The if-block below caches qp_attr to be modified for GSI and SMI * qps during the initialization by ib_mad. When the respective port @@ -1642,16 +1638,12 @@ int ehca_query_qp(struct ib_qp *qp, ib_device); struct ipz_adapter_handle adapter_handle = shca->ipz_hca_handle; struct hcp_modify_qp_control_block *qpcb; - u32 cur_pid = current->tgid; int cnt, ret = 0; u64 h_ret; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(qp->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(qp->device, my_pd)) return -EINVAL; - } if (qp_attr_mask & QP_ATTR_QUERY_NOT_SUPPORTED) { ehca_err(qp->device, "Invalid attribute mask " @@ -1806,13 +1798,9 @@ int ehca_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr, u64 h_ret; int ret = 0; - u32 cur_pid = current->tgid; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ibsrq->pd->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(ibsrq->pd->device, my_pd)) return -EINVAL; - } mqpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL); if (!mqpcb) { @@ -1869,16 +1857,12 @@ int ehca_query_srq(struct ib_srq *srq, struct ib_srq_attr *srq_attr) ib_device); struct ipz_adapter_handle adapter_handle = shca->ipz_hca_handle; struct hcp_modify_qp_control_block *qpcb; - u32 cur_pid = current->tgid; int ret = 0; u64 h_ret; if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(srq->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + ehca_check_pd_owner(srq->device, my_pd)) return -EINVAL; - } qpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL); if (!qpcb) { @@ -1919,7 +1903,6 @@ static int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp, struct ehca_pd *my_pd = container_of(my_qp->ib_qp.pd, struct ehca_pd, ib_pd); struct ehca_sport *sport = &shca->sport[my_qp->init_attr.port_num - 1]; - u32 cur_pid = current->tgid; u32 qp_num = my_qp->real_qp_num; int ret; u64 h_ret; @@ -1934,11 +1917,8 @@ static int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp, "user space qp_num=%x", qp_num); return -EINVAL; } - if (my_pd->ownpid != cur_pid) { - ehca_err(dev, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); + if (ehca_check_pd_owner(dev, my_pd)) return -EINVAL; - } } if (my_qp->send_cq) { diff --git a/drivers/infiniband/hw/ehca/ehca_tools.h b/drivers/infiniband/hw/ehca/ehca_tools.h index 4a8346a..0e59891 100644 --- a/drivers/infiniband/hw/ehca/ehca_tools.h +++ b/drivers/infiniband/hw/ehca/ehca_tools.h @@ -154,4 +154,9 @@ extern int ehca_debug_level; /* Converts ehca to ib return code */ int ehca2ib_return_code(u64 ehca_rc); +struct ib_device; +struct ehca_pd; + +int ehca_check_pd_owner(struct ib_device *device, struct ehca_pd *pd); + #endif /* EHCA_TOOLS_H */ diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c index 5234d6c..82746da 100644 --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c @@ -299,12 +294,8 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) return -EINVAL; pd = container_of(qp->ib_qp.pd, struct ehca_pd, ib_pd); - if (pd->ownpid != cur_pid) { - ehca_err(qp->ib_qp.device, - "Invalid caller pid=%x ownpid=%x", - cur_pid, pd->ownpid); + if (ehca_check_pd_owner(qp->ib_qp.device, pd)) return -ENOMEM; - } uobject = IS_SRQ(qp) ? qp->ib_srq.uobject : qp->ib_qp.uobject; if (!uobject || uobject->context != context) From xemul at openvz.org Mon Mar 17 05:48:25 2008 From: xemul at openvz.org (Pavel Emelyanov) Date: Mon, 17 Mar 2008 15:48:25 +0300 Subject: [ofa-general] [PATCH 2/3] Infiniband: make ehca_cq use struct pid pointer rather than pid_t Message-ID: <47DE6899.7000903@openvz.org> The task_struct->tgid field is about to become deprecated, due to pid namespaces make tasks have many pids, not one. The infiniband driver is one of the code, that still uses it in some places. Now make struct ehca_cq use a struct pid pointer and consolidate owners check into one call. Signed-off-by: Pavel Emelyanov --- drivers/infiniband/hw/ehca/ehca_classes.h | 2 +- drivers/infiniband/hw/ehca/ehca_cq.c | 30 ++++++++++++++++++------------ drivers/infiniband/hw/ehca/ehca_tools.h | 2 ++ drivers/infiniband/hw/ehca/ehca_uverbs.c | 7 +------ 4 files changed, 22 insertions(+), 19 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index 92cce8a..29004cb 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h @@ -215,7 +215,7 @@ struct ehca_cq { atomic_t nr_events; /* #events seen */ wait_queue_head_t wait_completion; spinlock_t task_lock; - u32 ownpid; + struct pid *ownpid; /* mmap counter for resources mapped into user space */ u32 mm_count_queue; u32 mm_count_galpa; diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c index 0467c15..db25056 100644 --- a/drivers/infiniband/hw/ehca/ehca_cq.c +++ b/drivers/infiniband/hw/ehca/ehca_cq.c @@ -113,6 +113,19 @@ struct ehca_qp *ehca_cq_get_qp(struct ehca_cq *cq, int real_qp_num) return ret; } +int ehca_check_cq_owner(struct ib_device *device, struct ehca_cq *cq) +{ + struct pid *cur_pid = task_tgid(current); + + if (likely(cur_pid == cq->ownpid)) + return 0; + + ehca_err(device, "Invalid caller pid=%x ownpid=%x " + "cq_num=%x", + pid_nr(cur_pid), pid_nr(cq->ownpid), cq->cq_number); + return -EINVAL; +} + struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector, struct ib_ucontext *context, struct ib_udata *udata) @@ -148,7 +161,7 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector, spin_lock_init(&my_cq->task_lock); atomic_set(&my_cq->nr_events, 0); init_waitqueue_head(&my_cq->wait_completion); - my_cq->ownpid = current->tgid; + my_cq->ownpid = get_pid(task_tgid(current)); cq = &my_cq->ib_cq; @@ -306,6 +319,7 @@ create_cq_exit2: write_unlock_irqrestore(&ehca_cq_idr_lock, flags); create_cq_exit1: + put_pid(my_cq->ownpid); kmem_cache_free(cq_cache, my_cq); return cq; @@ -320,7 +334,6 @@ int ehca_destroy_cq(struct ib_cq *cq) struct ehca_shca *shca = container_of(device, struct ehca_shca, ib_device); struct ipz_adapter_handle adapter_handle = shca->ipz_hca_handle; - u32 cur_pid = current->tgid; unsigned long flags; if (cq->uobject) { @@ -329,12 +342,8 @@ int ehca_destroy_cq(struct ib_cq *cq) "user space cq_num=%x", my_cq->cq_number); return -EINVAL; } - if (my_cq->ownpid != cur_pid) { - ehca_err(device, "Invalid caller pid=%x ownpid=%x " - "cq_num=%x", - cur_pid, my_cq->ownpid, my_cq->cq_number); + if (ehca_check_cq_owner(device, my_cq)) return -EINVAL; - } } /* @@ -367,6 +376,7 @@ int ehca_destroy_cq(struct ib_cq *cq) return ehca2ib_return_code(h_ret); } ipz_queue_dtor(NULL, &my_cq->ipz_queue); + put_pid(my_cq->ownpid); kmem_cache_free(cq_cache, my_cq); return 0; @@ -375,13 +385,9 @@ int ehca_destroy_cq(struct ib_cq *cq) int ehca_resize_cq(struct ib_cq *cq, int cqe, struct ib_udata *udata) { struct ehca_cq *my_cq = container_of(cq, struct ehca_cq, ib_cq); - u32 cur_pid = current->tgid; - if (cq->uobject && my_cq->ownpid != cur_pid) { - ehca_err(cq->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_cq->ownpid); + if (cq->uobject && ehca_check_cq_owner(cq->device, my_cq)) return -EINVAL; - } /* TODO: proper resize needs to be done */ ehca_err(cq->device, "not implemented yet"); diff --git a/drivers/infiniband/hw/ehca/ehca_tools.h b/drivers/infiniband/hw/ehca/ehca_tools.h index 4a8346a..0e59891 100644 --- a/drivers/infiniband/hw/ehca/ehca_tools.h +++ b/drivers/infiniband/hw/ehca/ehca_tools.h @@ -154,8 +154,10 @@ extern int ehca_debug_level; int ehca2ib_return_code(u64 ehca_rc); struct ib_device; +struct ehca_cq; struct ehca_pd; +int ehca_check_cq_owner(struct ib_device *device, struct ehca_cq *cq); int ehca_check_pd_owner(struct ib_device *device, struct ehca_pd *pd); #endif /* EHCA_TOOLS_H */ diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c index 5234d6c..82746da 100644 --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c @@ -253,7 +253,6 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) u32 idr_handle = fileoffset & 0x1FFFFFF; u32 q_type = (fileoffset >> 27) & 0x1; /* CQ, QP,... */ u32 rsrc_type = (fileoffset >> 25) & 0x3; /* sq,rq,cmnd_window */ - u32 cur_pid = current->tgid; u32 ret; struct ehca_cq *cq; struct ehca_qp *qp; @@ -270,12 +269,8 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) if (!cq) return -EINVAL; - if (cq->ownpid != cur_pid) { - ehca_err(cq->ib_cq.device, - "Invalid caller pid=%x ownpid=%x", - cur_pid, cq->ownpid); + if (ehca_check_cq_owner(cq->ib_cq.device, cq)) return -ENOMEM; - } if (!cq->ib_cq.uobject || cq->ib_cq.uobject->context != context) return -EINVAL; From xemul at openvz.org Mon Mar 17 05:50:55 2008 From: xemul at openvz.org (Pavel Emelyanov) Date: Mon, 17 Mar 2008 15:50:55 +0300 Subject: [ofa-general] [PATCH 3/3] Infiniband: don't use task_struct.tgid in make_cm_node() Message-ID: <47DE692F.5010600@openvz.org> The task_struct->tgid field is about to become deprecated, due to pid namespaces make tasks have many pids, not one. There is one place using it left to fix in infiniband - the make_cm_node(). This function uses the task tgid just to generate a session id, so it's perfectly safe to use appropriate wrapper - task_tgid_nr(). Signed-off-by: Pavel Emelyanov --- drivers/infiniband/hw/nes/nes_cm.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c index 39adb26..7a06156 100644 --- a/drivers/infiniband/hw/nes/nes_cm.c +++ b/drivers/infiniband/hw/nes/nes_cm.c @@ -1077,7 +1077,8 @@ static struct nes_cm_node *make_cm_node(struct nes_cm_core *cm_core, /* get a unique session ID , add thread_id to an upcounter to handle race */ atomic_inc(&cm_core->node_cnt); atomic_inc(&cm_core->session_id); - cm_node->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); + cm_node->session_id = (u32)(atomic_read(&cm_core->session_id) + + task_tgid_nr(current)); cm_node->conn_type = cm_info->conn_type; cm_node->apbvt_set = 0; cm_node->accept_pend = 0; @@ -1606,7 +1607,8 @@ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core, atomic_inc(&cm_core->node_cnt); atomic_inc(&cm_core->session_id); - listener->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); + listener->session_id = (u32)(atomic_read(&cm_core->session_id) + + task_tgid_nr(current)); listener->conn_type = cm_info->conn_type; listener->backlog = cm_info->backlog; listener->listener_state = NES_CM_LISTENER_ACTIVE_STATE; From Sebastian080 at mayridge.com Mon Mar 17 06:03:18 2008 From: Sebastian080 at mayridge.com (=?koi8-r?B?IuHX1NXIIOQu6S4i?=) Date: Mon, 17 Mar 2008 15:03:18 +0200 Subject: [ofa-general] =?koi8-r?b?68HUwczPxyDQ0sXE0NLJ0dTJySDrydTB0Q==?= Message-ID: <01c88840$08623f00$7ec2e458@Sebastian080> Лучшие заводы и фабрики Китая на 2008 год в одном каталоге! Найдите своего партнера. Приглашаем распространителей. http://wjrlmg23l.hotmail.ru -------------- next part -------------- An HTML attachment was scrubbed... URL: From absa3.abnx009 at memo.absa.co.za Mon Mar 17 06:18:06 2008 From: absa3.abnx009 at memo.absa.co.za (Shawna Quick) Date: Mon, 17 Mar 2008 13:18:06 +0000 Subject: [ofa-general] Discreet Purchase of Pills Message-ID: <736848567.68987601200695@memo.absa.co.za> The New Sale at EuropeanPharmacy is on - hurry up to best generic European drugs with an up-to-20% discount.Incredibly low prices are waiting for you. http://geocities.com/cortez.cora/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From oppositehgy6 at scottdrebus.com Mon Mar 17 06:45:38 2008 From: oppositehgy6 at scottdrebus.com (Shannon Velazquez) Date: Mon, 17 Mar 2008 15:45:38 +0200 Subject: [ofa-general] Access 2007 Message-ID: <01c88845$f2577d00$ff11545d@oppositehgy6> Microsoft Office Enterprise 2007 includes: • Access 2007 • Communicator 2007 • Excel 2007 • Groove 2007 • InfoPath 2007 • OneNote 2007 • Outlook 2007 • PowerPoint 2007 • Publisher 2007 • Word 2007 http://leonatesreauop261.blogspot.com System Requirements • Intel® Pentium® or AMD® 500 MHz processor • Microsoft Windows® XP Professional or Home Edition with Service Pack 2, Windows Server® 2003 with SP1 , Microsoft Windows Vista. • 256 Mb of RAM • 2GB of available hard-disk space. • 1024x768 or higher resolution monitor • Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From iadn at pjlhuillier.com Mon Mar 17 06:54:58 2008 From: iadn at pjlhuillier.com (Rowena Stone) Date: Mon, 17 Mar 2008 14:54:58 +0100 Subject: [ofa-general] AutoCAD 2008, Photoshop CS3, Adobe Acrobat 8 Message-ID: <647957651.06900900761761@pjlhuillier.com> OEM net value hereWir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren.http://geocities.com/ramirofrazier59/* Adobe Acrobat 8.0 Professional: $69.95 * Office Enterprise 2007: $79.95 * Adobe Photoshop CS3 Extended: $79.95 http://geocities.com/ramirofrazier59/ Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From bart.vanassche at gmail.com Mon Mar 17 07:08:38 2008 From: bart.vanassche at gmail.com (Bart Van Assche) Date: Mon, 17 Mar 2008 15:08:38 +0100 Subject: [ofa-general] SRP target sporadic behaviour with Solaris, VMware In-Reply-To: References: <47D70FC3.2030302@pocock.com.au> Message-ID: On Wed, Mar 12, 2008 at 8:10 AM, Bart Van Assche wrote: > My experience with SRP is as follows (with Linux 2.6.24 + SCST + SRPT > as target): > * Linux SRP initiator: works perfectly. > * OpenSolaris SRP initiator: I could not get Sun's SRP initiator > working on OpenSolaris. I even asked a Solaris expert to help me, but > he couldn't get the SRP initiator working either. > * VMware ESX 3.5 + Mellanox InfiniBand drivers (released in January > 2008): until now I only have tested a setup with a single target. When > doing a lot of I/O over the SRP connection, after about 10 minutes the > virtual machine running on VMware starts logging communication errors. > I reported this yesterday to Mellanox support, and Mellanox is > currently working on this issue. Note: I had to upgrade the InfiniBand > switch firmware before the ESX server was able to find the SRP target. The issue with SRP and VMware ESX might have been my fault -- I can't reproduce this anymore. ESX + SRP is now working fine here. Bart. From rdreier at cisco.com Mon Mar 17 07:28:29 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 17 Mar 2008 07:28:29 -0700 Subject: [ofa-general] IPOIB checksum offload support In-Reply-To: <1205742176.25950.73.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 10:22:56 +0200") References: <1205670144.25950.53.camel@mtls03> <1205742176.25950.73.camel@mtls03> Message-ID: > I believe we still have a problem if don't set the pointer to NULL. If > one of the subsequent operations fails we will end up freeing the same > pointer twice. Oops, thanks. I got rid of the tricky NULL stuff and just freed device_attr in the three places where it matters. From nirgayeroticanetworkjyl at gayeroticanetwork.com Mon Mar 17 07:36:12 2008 From: nirgayeroticanetworkjyl at gayeroticanetwork.com (Levi Mcknight) Date: Mon, 17 Mar 2008 23:36:12 +0900 Subject: [ofa-general] Software Message-ID: <208778718.91099920980116@gayeroticanetwork.com> Are you in search for the better prices in software abatements? Now you will find the opportunity to have the softs you dream of from very long ago. And the best thing for you is, all softs are dirt cheap. Be sure by youself and take the softwares for cheap costs ever seen! http://ramonaheimsmr559.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From bintantravel at gmail.com Mon Mar 17 07:42:16 2008 From: bintantravel at gmail.com (Bintan Travel) Date: Mon, 17 Mar 2008 22:42:16 +0800 Subject: [ofa-general] Bintan Holiday Vacations from SGD 42 Message-ID: <006601c8883d$5cea7130$8900a8c0@TOSHIBA> Location Bintan is the largest island in the Riau province, area 1.140 sq. km, with a coastline of about 105 km. It is truly an unique paradise. Simple life, beautiful beach, friendly people which bring us the leisure. Activities Recreational activities include snorkeling, jet-skiing, canoeing, wind surfing, golfing, fishing, sailing, diving, adventure trekking, ATV, cycling, go-kart, paint ball, island hopping and shopping. Sun seekers may take a leisurely stroll along then white sandy beaches and enjoy sun bathing in the warm sunshine. Getting There You can reach Bintan From Singapore, Malaysia (Johor Bahru) or Indonesia (Batam) by Ferries. You can also get there by flight from Jakarta (Soekarno Hatta Airport) to Tanjung Pinang (Kijang Airport). Most international travelers arrive from Singapore and Johor Bahru.Click here to find out more information bout getting to Bintan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: newslettersgd.jpg Type: image/jpeg Size: 401506 bytes Desc: not available URL: From friendshipko at abdupandaart.com Mon Mar 17 08:08:40 2008 From: friendshipko at abdupandaart.com (Milagros Frost) Date: Mon, 17 Mar 2008 17:08:40 +0200 Subject: [ofa-general] Improve sexual endurance. Message-ID: <01c88851$8bd85c00$6f38e758@friendshipko> The cells in the Corpora Cavernosa are filled with blood until an erection is achieved. http://tamaramesaqc197.blogspot.com From bart.vanassche at gmail.com Mon Mar 17 08:08:54 2008 From: bart.vanassche at gmail.com (Bart Van Assche) Date: Mon, 17 Mar 2008 16:08:54 +0100 Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface Message-ID: Hello, Is it possible to obtain statistics about the amount of RDMA traffic that passed over an InfiniBand interface ? While it is easy to obtain the number of bytes that has been communicated via IPoIB (e.g. via the command head /sys/class/net/ib*/statistics/*), I do not know of any way to observe the non-IPoIB traffic that passes over an InfiniBand interface. This would be convenient to observe iSER / SRP traffic. Bart. From eli at dev.mellanox.co.il Mon Mar 17 08:23:36 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:36 +0200 Subject: [ofa-general] [PATCH 0/10] stateless offload patches Message-ID: <1205767416.25950.135.camel@mtls03> Hi Roland, following this mail I am sending the stateless offload patches that did not make it to 2.6.25 and which I would like to get into 2.6.26. They have been reviewed by Or Gerlitz and we have discussed them in the list too. The list was generated against your "for-2.6.26" branch. 0001-IB-mthca-Add-checksum-offload-support.patch 0002-IB-core-Add-creation-flags-to-QPs.patch 0003-IB-core-Add-LSO-support.patch 0004-IB-ipoib-Add-LSO-support-to-ipoib.patch 0005-IB-mlx4-Add-QP-flags.patch 0006-IB-mlx4-Add-LSO-support.patch 0007-IB-ipoib-Add-ethtool-support.patch 0008-IB-core-Add-support-for-modify-CQ.patch 0009-IB-ipoib-Support-modifying-IPOIB-CQ-moderation-para.patch 0010-IB-mlx4-add-support-for-modifying-CQ-parameters.patch From eli at dev.mellanox.co.il Mon Mar 17 08:23:36 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:36 +0200 Subject: [ofa-general] [PATCH 0/10] stateless offload patches Message-ID: <1205767416.25950.135.camel@mtls03> Hi Roland, following this mail I am sending the stateless offload patches that did not make it to 2.6.25 and which I would like to get into 2.6.26. They have been reviewed by Or Gerlitz and we have discussed them in the list too. The list was generated against your "for-2.6.26" branch. 0001-IB-mthca-Add-checksum-offload-support.patch 0002-IB-core-Add-creation-flags-to-QPs.patch 0003-IB-core-Add-LSO-support.patch 0004-IB-ipoib-Add-LSO-support-to-ipoib.patch 0005-IB-mlx4-Add-QP-flags.patch 0006-IB-mlx4-Add-LSO-support.patch 0007-IB-ipoib-Add-ethtool-support.patch 0008-IB-core-Add-support-for-modify-CQ.patch 0009-IB-ipoib-Support-modifying-IPOIB-CQ-moderation-para.patch 0010-IB-mlx4-add-support-for-modifying-CQ-parameters.patch From eli at dev.mellanox.co.il Mon Mar 17 08:23:42 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:42 +0200 Subject: [ofa-general] [PATCH 1/10] IB/mthca: Add checksum offload support Message-ID: <1205767422.25950.136.camel@mtls03> >From 38f21d276ffd919ec1d072cce7a11bfcb8a5075c Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 27 Feb 2008 17:46:19 +0200 Subject: [PATCH] IB/mthca: Add checksum offload support Arbel devices support checksum generation and verification of TCP,UDP/IP packets. This patch checks if the HCA supports this and sets a capability flag. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mthca/mthca_cmd.c | 3 +++ drivers/infiniband/hw/mthca/mthca_cmd.h | 1 + drivers/infiniband/hw/mthca/mthca_cq.c | 14 +++++++++----- drivers/infiniband/hw/mthca/mthca_main.c | 4 ++++ drivers/infiniband/hw/mthca/mthca_qp.c | 2 ++ drivers/infiniband/hw/mthca/mthca_wqe.h | 17 +++++++++-------- 6 files changed, 28 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_cmd.c b/drivers/infiniband/hw/mthca/mthca_cmd.c index 667c35d..5e54875 100644 --- a/drivers/infiniband/hw/mthca/mthca_cmd.c +++ b/drivers/infiniband/hw/mthca/mthca_cmd.c @@ -1388,6 +1388,9 @@ int mthca_INIT_HCA(struct mthca_dev *dev, MTHCA_PUT(inbox, param->uarc_base, INIT_HCA_UAR_CTX_BASE_OFFSET); } + if (dev->device_cap_flags & IB_DEVICE_UD_IP_CSUM) + *(inbox + INIT_HCA_FLAGS2_OFFSET / 4) |= cpu_to_be32(7 << 3); + err = mthca_cmd(dev, mailbox->dma, 0, 0, CMD_INIT_HCA, HZ, status); mthca_free_mailbox(dev, mailbox); diff --git a/drivers/infiniband/hw/mthca/mthca_cmd.h b/drivers/infiniband/hw/mthca/mthca_cmd.h index 2f976f2..8928ca4 100644 --- a/drivers/infiniband/hw/mthca/mthca_cmd.h +++ b/drivers/infiniband/hw/mthca/mthca_cmd.h @@ -103,6 +103,7 @@ enum { DEV_LIM_FLAG_RAW_IPV6 = 1 << 4, DEV_LIM_FLAG_RAW_ETHER = 1 << 5, DEV_LIM_FLAG_SRQ = 1 << 6, + DEV_LIM_FLAG_IPOIB_CSUM = 1 << 7, DEV_LIM_FLAG_BAD_PKEY_CNTR = 1 << 8, DEV_LIM_FLAG_BAD_QKEY_CNTR = 1 << 9, DEV_LIM_FLAG_MW = 1 << 16, diff --git a/drivers/infiniband/hw/mthca/mthca_cq.c b/drivers/infiniband/hw/mthca/mthca_cq.c index 1e1e336..bfe95f3 100644 --- a/drivers/infiniband/hw/mthca/mthca_cq.c +++ b/drivers/infiniband/hw/mthca/mthca_cq.c @@ -119,7 +119,8 @@ struct mthca_cqe { __be32 my_qpn; __be32 my_ee; __be32 rqpn; - __be16 sl_g_mlpath; + u8 sl_ipok; + u8 g_mlpath; __be16 rlid; __be32 imm_etype_pkey_eec; __be32 byte_cnt; @@ -493,6 +494,7 @@ static inline int mthca_poll_one(struct mthca_dev *dev, int is_send; int free_cqe = 1; int err = 0; + u16 checksum; cqe = next_cqe_sw(cq); if (!cqe) @@ -635,12 +637,14 @@ static inline int mthca_poll_one(struct mthca_dev *dev, break; } entry->slid = be16_to_cpu(cqe->rlid); - entry->sl = be16_to_cpu(cqe->sl_g_mlpath) >> 12; + entry->sl = cqe->sl_ipok >> 4; entry->src_qp = be32_to_cpu(cqe->rqpn) & 0xffffff; - entry->dlid_path_bits = be16_to_cpu(cqe->sl_g_mlpath) & 0x7f; + entry->dlid_path_bits = cqe->g_mlpath & 0x7f; entry->pkey_index = be32_to_cpu(cqe->imm_etype_pkey_eec) >> 16; - entry->wc_flags |= be16_to_cpu(cqe->sl_g_mlpath) & 0x80 ? - IB_WC_GRH : 0; + entry->wc_flags |= cqe->g_mlpath & 0x80 ? IB_WC_GRH : 0; + checksum = (be32_to_cpu(cqe->rqpn) >> 24) | + ((be32_to_cpu(cqe->my_ee) >> 16) & 0xff00); + entry->csum_ok = (cqe->sl_ipok & 1 && checksum == 0xffff); } entry->status = IB_WC_SUCCESS; diff --git a/drivers/infiniband/hw/mthca/mthca_main.c b/drivers/infiniband/hw/mthca/mthca_main.c index cd3d8ad..3889ae8 100644 --- a/drivers/infiniband/hw/mthca/mthca_main.c +++ b/drivers/infiniband/hw/mthca/mthca_main.c @@ -267,6 +267,10 @@ static int mthca_dev_lim(struct mthca_dev *mdev, struct mthca_dev_lim *dev_lim) if (dev_lim->flags & DEV_LIM_FLAG_SRQ) mdev->mthca_flags |= MTHCA_FLAG_SRQ; + if (mthca_is_memfree(mdev)) + if (dev_lim->flags & DEV_LIM_FLAG_IPOIB_CSUM) + mdev->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; + return 0; } diff --git a/drivers/infiniband/hw/mthca/mthca_qp.c b/drivers/infiniband/hw/mthca/mthca_qp.c index db5595b..8433897 100644 --- a/drivers/infiniband/hw/mthca/mthca_qp.c +++ b/drivers/infiniband/hw/mthca/mthca_qp.c @@ -2015,6 +2015,8 @@ int mthca_arbel_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, cpu_to_be32(MTHCA_NEXT_CQ_UPDATE) : 0) | ((wr->send_flags & IB_SEND_SOLICITED) ? cpu_to_be32(MTHCA_NEXT_SOLICIT) : 0) | + ((wr->send_flags & IB_SEND_IP_CSUM) ? + cpu_to_be32(MTHCA_NEXT_IP_CSUM | MTHCA_NEXT_TCP_UDP_CSUM) : 0) | cpu_to_be32(1); if (wr->opcode == IB_WR_SEND_WITH_IMM || wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM) diff --git a/drivers/infiniband/hw/mthca/mthca_wqe.h b/drivers/infiniband/hw/mthca/mthca_wqe.h index f6a66fe..0e3a0e4 100644 --- a/drivers/infiniband/hw/mthca/mthca_wqe.h +++ b/drivers/infiniband/hw/mthca/mthca_wqe.h @@ -38,14 +38,15 @@ #include enum { - MTHCA_NEXT_DBD = 1 << 7, - MTHCA_NEXT_FENCE = 1 << 6, - MTHCA_NEXT_CQ_UPDATE = 1 << 3, - MTHCA_NEXT_EVENT_GEN = 1 << 2, - MTHCA_NEXT_SOLICIT = 1 << 1, - - MTHCA_MLX_VL15 = 1 << 17, - MTHCA_MLX_SLR = 1 << 16 + MTHCA_NEXT_DBD = 1 << 7, + MTHCA_NEXT_FENCE = 1 << 6, + MTHCA_NEXT_CQ_UPDATE = 1 << 3, + MTHCA_NEXT_EVENT_GEN = 1 << 2, + MTHCA_NEXT_SOLICIT = 1 << 1, + MTHCA_NEXT_IP_CSUM = 1 << 4, + MTHCA_NEXT_TCP_UDP_CSUM = 1 << 5, + MTHCA_MLX_VL15 = 1 << 17, + MTHCA_MLX_SLR = 1 << 16 }; enum { -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:23:42 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:42 +0200 Subject: [ofa-general] [PATCH 1/10] IB/mthca: Add checksum offload support Message-ID: <1205767422.25950.136.camel@mtls03> >From 38f21d276ffd919ec1d072cce7a11bfcb8a5075c Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 27 Feb 2008 17:46:19 +0200 Subject: [PATCH] IB/mthca: Add checksum offload support Arbel devices support checksum generation and verification of TCP,UDP/IP packets. This patch checks if the HCA supports this and sets a capability flag. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mthca/mthca_cmd.c | 3 +++ drivers/infiniband/hw/mthca/mthca_cmd.h | 1 + drivers/infiniband/hw/mthca/mthca_cq.c | 14 +++++++++----- drivers/infiniband/hw/mthca/mthca_main.c | 4 ++++ drivers/infiniband/hw/mthca/mthca_qp.c | 2 ++ drivers/infiniband/hw/mthca/mthca_wqe.h | 17 +++++++++-------- 6 files changed, 28 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_cmd.c b/drivers/infiniband/hw/mthca/mthca_cmd.c index 667c35d..5e54875 100644 --- a/drivers/infiniband/hw/mthca/mthca_cmd.c +++ b/drivers/infiniband/hw/mthca/mthca_cmd.c @@ -1388,6 +1388,9 @@ int mthca_INIT_HCA(struct mthca_dev *dev, MTHCA_PUT(inbox, param->uarc_base, INIT_HCA_UAR_CTX_BASE_OFFSET); } + if (dev->device_cap_flags & IB_DEVICE_UD_IP_CSUM) + *(inbox + INIT_HCA_FLAGS2_OFFSET / 4) |= cpu_to_be32(7 << 3); + err = mthca_cmd(dev, mailbox->dma, 0, 0, CMD_INIT_HCA, HZ, status); mthca_free_mailbox(dev, mailbox); diff --git a/drivers/infiniband/hw/mthca/mthca_cmd.h b/drivers/infiniband/hw/mthca/mthca_cmd.h index 2f976f2..8928ca4 100644 --- a/drivers/infiniband/hw/mthca/mthca_cmd.h +++ b/drivers/infiniband/hw/mthca/mthca_cmd.h @@ -103,6 +103,7 @@ enum { DEV_LIM_FLAG_RAW_IPV6 = 1 << 4, DEV_LIM_FLAG_RAW_ETHER = 1 << 5, DEV_LIM_FLAG_SRQ = 1 << 6, + DEV_LIM_FLAG_IPOIB_CSUM = 1 << 7, DEV_LIM_FLAG_BAD_PKEY_CNTR = 1 << 8, DEV_LIM_FLAG_BAD_QKEY_CNTR = 1 << 9, DEV_LIM_FLAG_MW = 1 << 16, diff --git a/drivers/infiniband/hw/mthca/mthca_cq.c b/drivers/infiniband/hw/mthca/mthca_cq.c index 1e1e336..bfe95f3 100644 --- a/drivers/infiniband/hw/mthca/mthca_cq.c +++ b/drivers/infiniband/hw/mthca/mthca_cq.c @@ -119,7 +119,8 @@ struct mthca_cqe { __be32 my_qpn; __be32 my_ee; __be32 rqpn; - __be16 sl_g_mlpath; + u8 sl_ipok; + u8 g_mlpath; __be16 rlid; __be32 imm_etype_pkey_eec; __be32 byte_cnt; @@ -493,6 +494,7 @@ static inline int mthca_poll_one(struct mthca_dev *dev, int is_send; int free_cqe = 1; int err = 0; + u16 checksum; cqe = next_cqe_sw(cq); if (!cqe) @@ -635,12 +637,14 @@ static inline int mthca_poll_one(struct mthca_dev *dev, break; } entry->slid = be16_to_cpu(cqe->rlid); - entry->sl = be16_to_cpu(cqe->sl_g_mlpath) >> 12; + entry->sl = cqe->sl_ipok >> 4; entry->src_qp = be32_to_cpu(cqe->rqpn) & 0xffffff; - entry->dlid_path_bits = be16_to_cpu(cqe->sl_g_mlpath) & 0x7f; + entry->dlid_path_bits = cqe->g_mlpath & 0x7f; entry->pkey_index = be32_to_cpu(cqe->imm_etype_pkey_eec) >> 16; - entry->wc_flags |= be16_to_cpu(cqe->sl_g_mlpath) & 0x80 ? - IB_WC_GRH : 0; + entry->wc_flags |= cqe->g_mlpath & 0x80 ? IB_WC_GRH : 0; + checksum = (be32_to_cpu(cqe->rqpn) >> 24) | + ((be32_to_cpu(cqe->my_ee) >> 16) & 0xff00); + entry->csum_ok = (cqe->sl_ipok & 1 && checksum == 0xffff); } entry->status = IB_WC_SUCCESS; diff --git a/drivers/infiniband/hw/mthca/mthca_main.c b/drivers/infiniband/hw/mthca/mthca_main.c index cd3d8ad..3889ae8 100644 --- a/drivers/infiniband/hw/mthca/mthca_main.c +++ b/drivers/infiniband/hw/mthca/mthca_main.c @@ -267,6 +267,10 @@ static int mthca_dev_lim(struct mthca_dev *mdev, struct mthca_dev_lim *dev_lim) if (dev_lim->flags & DEV_LIM_FLAG_SRQ) mdev->mthca_flags |= MTHCA_FLAG_SRQ; + if (mthca_is_memfree(mdev)) + if (dev_lim->flags & DEV_LIM_FLAG_IPOIB_CSUM) + mdev->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; + return 0; } diff --git a/drivers/infiniband/hw/mthca/mthca_qp.c b/drivers/infiniband/hw/mthca/mthca_qp.c index db5595b..8433897 100644 --- a/drivers/infiniband/hw/mthca/mthca_qp.c +++ b/drivers/infiniband/hw/mthca/mthca_qp.c @@ -2015,6 +2015,8 @@ int mthca_arbel_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, cpu_to_be32(MTHCA_NEXT_CQ_UPDATE) : 0) | ((wr->send_flags & IB_SEND_SOLICITED) ? cpu_to_be32(MTHCA_NEXT_SOLICIT) : 0) | + ((wr->send_flags & IB_SEND_IP_CSUM) ? + cpu_to_be32(MTHCA_NEXT_IP_CSUM | MTHCA_NEXT_TCP_UDP_CSUM) : 0) | cpu_to_be32(1); if (wr->opcode == IB_WR_SEND_WITH_IMM || wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM) diff --git a/drivers/infiniband/hw/mthca/mthca_wqe.h b/drivers/infiniband/hw/mthca/mthca_wqe.h index f6a66fe..0e3a0e4 100644 --- a/drivers/infiniband/hw/mthca/mthca_wqe.h +++ b/drivers/infiniband/hw/mthca/mthca_wqe.h @@ -38,14 +38,15 @@ #include enum { - MTHCA_NEXT_DBD = 1 << 7, - MTHCA_NEXT_FENCE = 1 << 6, - MTHCA_NEXT_CQ_UPDATE = 1 << 3, - MTHCA_NEXT_EVENT_GEN = 1 << 2, - MTHCA_NEXT_SOLICIT = 1 << 1, - - MTHCA_MLX_VL15 = 1 << 17, - MTHCA_MLX_SLR = 1 << 16 + MTHCA_NEXT_DBD = 1 << 7, + MTHCA_NEXT_FENCE = 1 << 6, + MTHCA_NEXT_CQ_UPDATE = 1 << 3, + MTHCA_NEXT_EVENT_GEN = 1 << 2, + MTHCA_NEXT_SOLICIT = 1 << 1, + MTHCA_NEXT_IP_CSUM = 1 << 4, + MTHCA_NEXT_TCP_UDP_CSUM = 1 << 5, + MTHCA_MLX_VL15 = 1 << 17, + MTHCA_MLX_SLR = 1 << 16 }; enum { -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:23:47 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:47 +0200 Subject: [ofa-general] [PATCH 2/10] IB/core: Add creation flags to QPs Message-ID: <1205767427.25950.137.camel@mtls03> >From 6de5c54194ff2f36b4f7e34bdf118f52f6a57239 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 27 Feb 2008 18:02:13 +0200 Subject: [PATCH] IB/core: Add creation flags to QPs This will allow a kernel verbs consumer to create a QP and pass special flags to the hw layer. This patch also defines one such flag for LSO support. Signed-off-by: Eli Cohen --- drivers/infiniband/core/uverbs_cmd.c | 1 + include/rdma/ib_verbs.h | 5 +++++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 495c803..9e98cec 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1065,6 +1065,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, attr.srq = srq; attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; attr.qp_type = cmd.qp_type; + attr.create_flags = 0; attr.cap.max_send_wr = cmd.max_send_wr; attr.cap.max_recv_wr = cmd.max_recv_wr; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 40ff512..7ee0077 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -495,6 +495,10 @@ enum ib_qp_type { IB_QPT_RAW_ETY }; +enum qp_create_flags { + QP_CREATE_LSO = 1 << 0, +}; + struct ib_qp_init_attr { void (*event_handler)(struct ib_event *, void *); void *qp_context; @@ -505,6 +509,7 @@ struct ib_qp_init_attr { enum ib_sig_type sq_sig_type; enum ib_qp_type qp_type; u8 port_num; /* special QP types only */ + enum qp_create_flags create_flags; }; enum ib_rnr_timeout { -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:23:47 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:47 +0200 Subject: [ofa-general] [PATCH 2/10] IB/core: Add creation flags to QPs Message-ID: <1205767427.25950.137.camel@mtls03> >From 6de5c54194ff2f36b4f7e34bdf118f52f6a57239 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 27 Feb 2008 18:02:13 +0200 Subject: [PATCH] IB/core: Add creation flags to QPs This will allow a kernel verbs consumer to create a QP and pass special flags to the hw layer. This patch also defines one such flag for LSO support. Signed-off-by: Eli Cohen --- drivers/infiniband/core/uverbs_cmd.c | 1 + include/rdma/ib_verbs.h | 5 +++++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 495c803..9e98cec 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1065,6 +1065,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, attr.srq = srq; attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; attr.qp_type = cmd.qp_type; + attr.create_flags = 0; attr.cap.max_send_wr = cmd.max_send_wr; attr.cap.max_recv_wr = cmd.max_recv_wr; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 40ff512..7ee0077 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -495,6 +495,10 @@ enum ib_qp_type { IB_QPT_RAW_ETY }; +enum qp_create_flags { + QP_CREATE_LSO = 1 << 0, +}; + struct ib_qp_init_attr { void (*event_handler)(struct ib_event *, void *); void *qp_context; @@ -505,6 +509,7 @@ struct ib_qp_init_attr { enum ib_sig_type sq_sig_type; enum ib_qp_type qp_type; u8 port_num; /* special QP types only */ + enum qp_create_flags create_flags; }; enum ib_rnr_timeout { -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:23:51 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:51 +0200 Subject: [ofa-general] [PATCH 3/10] IB/core: Add LSO support Message-ID: <1205767431.25950.138.camel@mtls03> >From 5955550017933a1a3e169a4c2d700d6b3dc5c008 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 28 Feb 2008 14:02:36 +0200 Subject: [PATCH] IB/core: Add LSO support LSO allows the networikng stack to pass pass to the network driver SKBs with data size larger then MTU and let the HW fragment the data to mss sized packets. Signed-off-by: Eli Cohen --- include/rdma/ib_verbs.h | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 7ee0077..9557d10 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -104,6 +104,7 @@ enum ib_device_cap_flags { * IPoIB driver may set NETIF_F_IP_CSUM for datagram mode. */ IB_DEVICE_UD_IP_CSUM = (1<<18), + IB_DEVICE_TCP_TSO = (1<<19), }; enum ib_atomic_cap { @@ -411,6 +412,7 @@ enum ib_wc_opcode { IB_WC_COMP_SWAP, IB_WC_FETCH_ADD, IB_WC_BIND_MW, + IB_WC_LSO, /* * Set value of IB_WC_RECV so consumers can test if a completion is a * receive by testing (opcode & IB_WC_RECV). @@ -622,7 +624,8 @@ enum ib_wr_opcode { IB_WR_SEND_WITH_IMM, IB_WR_RDMA_READ, IB_WR_ATOMIC_CMP_AND_SWP, - IB_WR_ATOMIC_FETCH_AND_ADD + IB_WR_ATOMIC_FETCH_AND_ADD, + IB_WR_LSO }; enum ib_send_flags { @@ -630,7 +633,8 @@ enum ib_send_flags { IB_SEND_SIGNALED = (1<<1), IB_SEND_SOLICITED = (1<<2), IB_SEND_INLINE = (1<<3), - IB_SEND_IP_CSUM = (1<<4) + IB_SEND_IP_CSUM = (1<<4), + IB_SEND_UDP_LSO = (1<<5) }; struct ib_sge { @@ -660,6 +664,9 @@ struct ib_send_wr { } atomic; struct { struct ib_ah *ah; + void *header; + int hlen; + int mss; u32 remote_qpn; u32 remote_qkey; u16 pkey_index; /* valid for GSI only */ -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:23:51 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:51 +0200 Subject: [ofa-general] [PATCH 3/10] IB/core: Add LSO support Message-ID: <1205767431.25950.138.camel@mtls03> >From 5955550017933a1a3e169a4c2d700d6b3dc5c008 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 28 Feb 2008 14:02:36 +0200 Subject: [PATCH] IB/core: Add LSO support LSO allows the networikng stack to pass pass to the network driver SKBs with data size larger then MTU and let the HW fragment the data to mss sized packets. Signed-off-by: Eli Cohen --- include/rdma/ib_verbs.h | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 7ee0077..9557d10 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -104,6 +104,7 @@ enum ib_device_cap_flags { * IPoIB driver may set NETIF_F_IP_CSUM for datagram mode. */ IB_DEVICE_UD_IP_CSUM = (1<<18), + IB_DEVICE_TCP_TSO = (1<<19), }; enum ib_atomic_cap { @@ -411,6 +412,7 @@ enum ib_wc_opcode { IB_WC_COMP_SWAP, IB_WC_FETCH_ADD, IB_WC_BIND_MW, + IB_WC_LSO, /* * Set value of IB_WC_RECV so consumers can test if a completion is a * receive by testing (opcode & IB_WC_RECV). @@ -622,7 +624,8 @@ enum ib_wr_opcode { IB_WR_SEND_WITH_IMM, IB_WR_RDMA_READ, IB_WR_ATOMIC_CMP_AND_SWP, - IB_WR_ATOMIC_FETCH_AND_ADD + IB_WR_ATOMIC_FETCH_AND_ADD, + IB_WR_LSO }; enum ib_send_flags { @@ -630,7 +633,8 @@ enum ib_send_flags { IB_SEND_SIGNALED = (1<<1), IB_SEND_SOLICITED = (1<<2), IB_SEND_INLINE = (1<<3), - IB_SEND_IP_CSUM = (1<<4) + IB_SEND_IP_CSUM = (1<<4), + IB_SEND_UDP_LSO = (1<<5) }; struct ib_sge { @@ -660,6 +664,9 @@ struct ib_send_wr { } atomic; struct { struct ib_ah *ah; + void *header; + int hlen; + int mss; u32 remote_qpn; u32 remote_qkey; u16 pkey_index; /* valid for GSI only */ -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:23:55 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:55 +0200 Subject: [ofa-general] [PATCH 4/10] IB/ipoib: Add LSO support to ipoib Message-ID: <1205767435.25950.139.camel@mtls03> >From 7d2a91c6ac83e9af3141b5e4d73a757bf9159b44 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 28 Feb 2008 14:13:45 +0200 Subject: [PATCH] IB/ipoib: Add LSO support to ipoib Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 1 + drivers/infiniband/ulp/ipoib/ipoib_cm.c | 7 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 117 +++++++++++++++++++++------- drivers/infiniband/ulp/ipoib/ipoib_main.c | 10 ++- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 3 + 5 files changed, 103 insertions(+), 35 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 08930ca..19a41ff 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -319,6 +319,7 @@ struct ipoib_dev_priv { struct dentry *mcg_dentry; struct dentry *path_dentry; #endif + int hca_caps; }; struct ipoib_ah { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index edf63dc..90ff2c9 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1384,7 +1384,7 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); - dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG); + dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO); priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; ipoib_flush_paths(dev); @@ -1396,8 +1396,11 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, dev->mtu = min(priv->mcast_mtu, dev->mtu); ipoib_flush_paths(dev); - if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) + if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) { dev->features |= NETIF_F_IP_CSUM | NETIF_F_SG; + if (priv->hca_caps & IB_DEVICE_TCP_TSO) + dev->features |= NETIF_F_TSO; + } return count; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index d13f4fb..6605109 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -39,6 +39,8 @@ #include #include +#include +#include #include "ipoib.h" @@ -249,29 +251,37 @@ static int ipoib_dma_map_tx(struct ib_device *ca, struct sk_buff *skb = tx_req->skb; u64 *mapping = tx_req->mapping; int i; + int off; - mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), - DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) - return -EIO; + if (skb_headlen(skb)) { + mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), + DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) + return -EIO; + + off = 1; + } else + off = 0; for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - mapping[i + 1] = ib_dma_map_page(ca, frag->page, + mapping[i + off] = ib_dma_map_page(ca, frag->page, frag->page_offset, frag->size, DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(ca, mapping[i + 1]))) + if (unlikely(ib_dma_mapping_error(ca, mapping[i + off]))) goto partial_error; } return 0; partial_error: - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); - for (; i > 0; --i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i - 1]; - ib_dma_unmap_page(ca, mapping[i], frag->size, DMA_TO_DEVICE); + ib_dma_unmap_page(ca, mapping[i - !off], frag->size, DMA_TO_DEVICE); } + + if (off) + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + return -EIO; } @@ -281,12 +291,17 @@ static void ipoib_dma_unmap_tx(struct ib_device *ca, struct sk_buff *skb = tx_req->skb; u64 *mapping = tx_req->mapping; int i; + int off; - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + if (skb_headlen(skb)) { + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + off = 1; + } else + off = 0; for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - ib_dma_unmap_page(ca, mapping[i + 1], frag->size, + ib_dma_unmap_page(ca, mapping[i + off], frag->size, DMA_TO_DEVICE); } } @@ -392,24 +407,40 @@ void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr) static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - u64 *mapping, int headlen, - skb_frag_t *frags, - int nr_frags) + struct ipoib_tx_buf *tx_req, + void *head, int hlen) { struct ib_send_wr *bad_wr; - int i; + int i, off; + struct sk_buff *skb = tx_req->skb; + skb_frag_t *frags = skb_shinfo(skb)->frags; + int nr_frags = skb_shinfo(skb)->nr_frags; + u64 *mapping = tx_req->mapping; + + if (skb_headlen(skb)) { + priv->tx_sge[0].addr = mapping[0]; + priv->tx_sge[0].length = skb_headlen(skb); + off = 1; + } else + off = 0; - priv->tx_sge[0].addr = mapping[0]; - priv->tx_sge[0].length = headlen; for (i = 0; i < nr_frags; ++i) { - priv->tx_sge[i + 1].addr = mapping[i + 1]; - priv->tx_sge[i + 1].length = frags[i].size; + priv->tx_sge[i + off].addr = mapping[i + off]; + priv->tx_sge[i + off].length = frags[i].size; } - priv->tx_wr.num_sge = nr_frags + 1; + priv->tx_wr.num_sge = nr_frags + off; priv->tx_wr.wr_id = wr_id; priv->tx_wr.wr.ud.remote_qpn = qpn; priv->tx_wr.wr.ud.ah = address; + if (head) { + priv->tx_wr.wr.ud.mss = skb_shinfo(skb)->gso_size; + priv->tx_wr.wr.ud.header = head; + priv->tx_wr.wr.ud.hlen = hlen; + priv->tx_wr.opcode = IB_WR_LSO; + } else + priv->tx_wr.opcode = IB_WR_SEND; + return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); } @@ -418,14 +449,35 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_tx_buf *tx_req; - - if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { - ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", - skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); - ++dev->stats.tx_dropped; - ++dev->stats.tx_errors; - ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); - return; + int hlen; + void *phead; + + if (!skb_is_gso(skb)) { + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); + ++dev->stats.tx_dropped; + ++dev->stats.tx_errors; + ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); + return; + } + phead = 0; + hlen = 0; + } else { + /* + * LSO header is limited to max 60 bytes + */ + if (unlikely((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) > 15)) { + ipoib_warn(priv, "ip(%d) and tcp(%d) headers too long, dropping skb\n", + ip_hdr(skb)->ihl << 2, tcp_hdr(skb)->doff << 2); + goto drop; + } + hlen = ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) + IPOIB_ENCAP_LEN; + phead = skb->data; + if (unlikely(!skb_pull(skb, hlen))) { + ipoib_warn(priv, "linear data too small\n"); + goto drop; + } } ipoib_dbg_data(priv, "sending packet, length=%d address=%p qpn=0x%06x\n", @@ -453,8 +505,7 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), address->ah, qpn, - tx_req->mapping, skb_headlen(skb), - skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags))) { + tx_req, phead, hlen))) { ipoib_warn(priv, "post_send failed\n"); ++dev->stats.tx_errors; ipoib_dma_unmap_tx(priv->ca, tx_req); @@ -470,6 +521,12 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, netif_stop_queue(dev); } } + return; + +drop: + ++dev->stats.tx_errors; + dev_kfree_skb_any(skb); + return; } static void __ipoib_reap_ah(struct net_device *dev) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 40db0d9..bba286c 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -1134,14 +1134,15 @@ static struct net_device *ipoib_add_port(const char *format, hca->name, result); goto device_init_failed; } + priv->hca_caps = device_attr->device_cap_flags; - if (device_attr->device_cap_flags & IB_DEVICE_UD_IP_CSUM) { + kfree(device_attr); + + if (priv->hca_caps & IB_DEVICE_UD_IP_CSUM) { set_bit(IPOIB_FLAG_CSUM, &priv->flags); priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; } - kfree(device_attr); - /* * Set the full membership bit, so that we join the right * broadcast group, etc. @@ -1176,6 +1177,9 @@ static struct net_device *ipoib_add_port(const char *format, goto event_failed; } + if (priv->dev->features & NETIF_F_SG && priv->hca_caps & IB_DEVICE_TCP_TSO) + priv->dev->features |= NETIF_F_TSO; + result = register_netdev(priv->dev); if (result) { printk(KERN_WARNING "%s: couldn't register ipoib port %d; error %d\n", diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index a3aeb91..1d59f27 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -192,6 +192,9 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) init_attr.send_cq = priv->cq; init_attr.recv_cq = priv->cq; + if (priv->hca_caps & IB_DEVICE_TCP_TSO) + init_attr.create_flags = QP_CREATE_LSO; + if (dev->features & NETIF_F_SG) init_attr.cap.max_send_sge = MAX_SKB_FRAGS + 1; -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:23:55 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:23:55 +0200 Subject: [ofa-general] [PATCH 4/10] IB/ipoib: Add LSO support to ipoib Message-ID: <1205767435.25950.139.camel@mtls03> >From 7d2a91c6ac83e9af3141b5e4d73a757bf9159b44 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 28 Feb 2008 14:13:45 +0200 Subject: [PATCH] IB/ipoib: Add LSO support to ipoib Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 1 + drivers/infiniband/ulp/ipoib/ipoib_cm.c | 7 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 117 +++++++++++++++++++++------- drivers/infiniband/ulp/ipoib/ipoib_main.c | 10 ++- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 3 + 5 files changed, 103 insertions(+), 35 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 08930ca..19a41ff 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -319,6 +319,7 @@ struct ipoib_dev_priv { struct dentry *mcg_dentry; struct dentry *path_dentry; #endif + int hca_caps; }; struct ipoib_ah { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index edf63dc..90ff2c9 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1384,7 +1384,7 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); - dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG); + dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO); priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; ipoib_flush_paths(dev); @@ -1396,8 +1396,11 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, dev->mtu = min(priv->mcast_mtu, dev->mtu); ipoib_flush_paths(dev); - if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) + if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) { dev->features |= NETIF_F_IP_CSUM | NETIF_F_SG; + if (priv->hca_caps & IB_DEVICE_TCP_TSO) + dev->features |= NETIF_F_TSO; + } return count; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index d13f4fb..6605109 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -39,6 +39,8 @@ #include #include +#include +#include #include "ipoib.h" @@ -249,29 +251,37 @@ static int ipoib_dma_map_tx(struct ib_device *ca, struct sk_buff *skb = tx_req->skb; u64 *mapping = tx_req->mapping; int i; + int off; - mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), - DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) - return -EIO; + if (skb_headlen(skb)) { + mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), + DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) + return -EIO; + + off = 1; + } else + off = 0; for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - mapping[i + 1] = ib_dma_map_page(ca, frag->page, + mapping[i + off] = ib_dma_map_page(ca, frag->page, frag->page_offset, frag->size, DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(ca, mapping[i + 1]))) + if (unlikely(ib_dma_mapping_error(ca, mapping[i + off]))) goto partial_error; } return 0; partial_error: - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); - for (; i > 0; --i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i - 1]; - ib_dma_unmap_page(ca, mapping[i], frag->size, DMA_TO_DEVICE); + ib_dma_unmap_page(ca, mapping[i - !off], frag->size, DMA_TO_DEVICE); } + + if (off) + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + return -EIO; } @@ -281,12 +291,17 @@ static void ipoib_dma_unmap_tx(struct ib_device *ca, struct sk_buff *skb = tx_req->skb; u64 *mapping = tx_req->mapping; int i; + int off; - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + if (skb_headlen(skb)) { + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + off = 1; + } else + off = 0; for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - ib_dma_unmap_page(ca, mapping[i + 1], frag->size, + ib_dma_unmap_page(ca, mapping[i + off], frag->size, DMA_TO_DEVICE); } } @@ -392,24 +407,40 @@ void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr) static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - u64 *mapping, int headlen, - skb_frag_t *frags, - int nr_frags) + struct ipoib_tx_buf *tx_req, + void *head, int hlen) { struct ib_send_wr *bad_wr; - int i; + int i, off; + struct sk_buff *skb = tx_req->skb; + skb_frag_t *frags = skb_shinfo(skb)->frags; + int nr_frags = skb_shinfo(skb)->nr_frags; + u64 *mapping = tx_req->mapping; + + if (skb_headlen(skb)) { + priv->tx_sge[0].addr = mapping[0]; + priv->tx_sge[0].length = skb_headlen(skb); + off = 1; + } else + off = 0; - priv->tx_sge[0].addr = mapping[0]; - priv->tx_sge[0].length = headlen; for (i = 0; i < nr_frags; ++i) { - priv->tx_sge[i + 1].addr = mapping[i + 1]; - priv->tx_sge[i + 1].length = frags[i].size; + priv->tx_sge[i + off].addr = mapping[i + off]; + priv->tx_sge[i + off].length = frags[i].size; } - priv->tx_wr.num_sge = nr_frags + 1; + priv->tx_wr.num_sge = nr_frags + off; priv->tx_wr.wr_id = wr_id; priv->tx_wr.wr.ud.remote_qpn = qpn; priv->tx_wr.wr.ud.ah = address; + if (head) { + priv->tx_wr.wr.ud.mss = skb_shinfo(skb)->gso_size; + priv->tx_wr.wr.ud.header = head; + priv->tx_wr.wr.ud.hlen = hlen; + priv->tx_wr.opcode = IB_WR_LSO; + } else + priv->tx_wr.opcode = IB_WR_SEND; + return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); } @@ -418,14 +449,35 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_tx_buf *tx_req; - - if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { - ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", - skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); - ++dev->stats.tx_dropped; - ++dev->stats.tx_errors; - ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); - return; + int hlen; + void *phead; + + if (!skb_is_gso(skb)) { + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); + ++dev->stats.tx_dropped; + ++dev->stats.tx_errors; + ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); + return; + } + phead = 0; + hlen = 0; + } else { + /* + * LSO header is limited to max 60 bytes + */ + if (unlikely((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) > 15)) { + ipoib_warn(priv, "ip(%d) and tcp(%d) headers too long, dropping skb\n", + ip_hdr(skb)->ihl << 2, tcp_hdr(skb)->doff << 2); + goto drop; + } + hlen = ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) + IPOIB_ENCAP_LEN; + phead = skb->data; + if (unlikely(!skb_pull(skb, hlen))) { + ipoib_warn(priv, "linear data too small\n"); + goto drop; + } } ipoib_dbg_data(priv, "sending packet, length=%d address=%p qpn=0x%06x\n", @@ -453,8 +505,7 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), address->ah, qpn, - tx_req->mapping, skb_headlen(skb), - skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags))) { + tx_req, phead, hlen))) { ipoib_warn(priv, "post_send failed\n"); ++dev->stats.tx_errors; ipoib_dma_unmap_tx(priv->ca, tx_req); @@ -470,6 +521,12 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, netif_stop_queue(dev); } } + return; + +drop: + ++dev->stats.tx_errors; + dev_kfree_skb_any(skb); + return; } static void __ipoib_reap_ah(struct net_device *dev) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 40db0d9..bba286c 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -1134,14 +1134,15 @@ static struct net_device *ipoib_add_port(const char *format, hca->name, result); goto device_init_failed; } + priv->hca_caps = device_attr->device_cap_flags; - if (device_attr->device_cap_flags & IB_DEVICE_UD_IP_CSUM) { + kfree(device_attr); + + if (priv->hca_caps & IB_DEVICE_UD_IP_CSUM) { set_bit(IPOIB_FLAG_CSUM, &priv->flags); priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; } - kfree(device_attr); - /* * Set the full membership bit, so that we join the right * broadcast group, etc. @@ -1176,6 +1177,9 @@ static struct net_device *ipoib_add_port(const char *format, goto event_failed; } + if (priv->dev->features & NETIF_F_SG && priv->hca_caps & IB_DEVICE_TCP_TSO) + priv->dev->features |= NETIF_F_TSO; + result = register_netdev(priv->dev); if (result) { printk(KERN_WARNING "%s: couldn't register ipoib port %d; error %d\n", diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index a3aeb91..1d59f27 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -192,6 +192,9 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) init_attr.send_cq = priv->cq; init_attr.recv_cq = priv->cq; + if (priv->hca_caps & IB_DEVICE_TCP_TSO) + init_attr.create_flags = QP_CREATE_LSO; + if (dev->features & NETIF_F_SG) init_attr.cap.max_send_sge = MAX_SKB_FRAGS + 1; -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:00 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:00 +0200 Subject: [ofa-general] [PATCH 5/10] IB/mlx4: Add QP flags Message-ID: <1205767440.25950.140.camel@mtls03> >From c0f184afbced888b2812d67cbffb0d1e4af04ef2 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 28 Feb 2008 18:08:45 +0200 Subject: [PATCH] IB/mlx4: Add QP flags This patch adds a flag filed to mlx4_ib_qp. These flags can be used to save flags derived from create flags passed at QP creation and be used afterwards. This patch also adds the MLX4_QP_LSO flag. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mlx4/mlx4_ib.h | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 3726e45..0d7ccbc 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -110,6 +110,10 @@ struct mlx4_ib_wq { unsigned tail; }; +enum qp_flags { + MLX4_QP_LSO = 1 << 0 +}; + struct mlx4_ib_qp { struct ib_qp ibqp; struct mlx4_qp mqp; @@ -135,6 +139,7 @@ struct mlx4_ib_qp { u8 resp_depth; u8 sq_no_prefetch; u8 state; + u32 flags; }; struct mlx4_ib_srq { -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:00 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:00 +0200 Subject: [ofa-general] [PATCH 5/10] IB/mlx4: Add QP flags Message-ID: <1205767440.25950.140.camel@mtls03> >From c0f184afbced888b2812d67cbffb0d1e4af04ef2 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 28 Feb 2008 18:08:45 +0200 Subject: [PATCH] IB/mlx4: Add QP flags This patch adds a flag filed to mlx4_ib_qp. These flags can be used to save flags derived from create flags passed at QP creation and be used afterwards. This patch also adds the MLX4_QP_LSO flag. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mlx4/mlx4_ib.h | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 3726e45..0d7ccbc 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -110,6 +110,10 @@ struct mlx4_ib_wq { unsigned tail; }; +enum qp_flags { + MLX4_QP_LSO = 1 << 0 +}; + struct mlx4_ib_qp { struct ib_qp ibqp; struct mlx4_qp mqp; @@ -135,6 +139,7 @@ struct mlx4_ib_qp { u8 resp_depth; u8 sq_no_prefetch; u8 state; + u32 flags; }; struct mlx4_ib_srq { -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:03 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:03 +0200 Subject: [ofa-general] [PATCH 6/10] IB/mlx4: Add LSO support Message-ID: <1205767443.25950.141.camel@mtls03> >From f60c81b106a5cbbef9d4943012215022e9d7f0b0 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Tue, 11 Mar 2008 15:26:38 +0200 Subject: [PATCH] IB/mlx4: Add LSO support Add LSO support to mlx4 driver such that it will be able to send SKBs passed from the driver which publish NETIF_TSO. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mlx4/cq.c | 3 ++ drivers/infiniband/hw/mlx4/main.c | 3 ++ drivers/infiniband/hw/mlx4/qp.c | 60 ++++++++++++++++++++++++++++++++++-- drivers/net/mlx4/fw.c | 9 +++++ drivers/net/mlx4/fw.h | 1 + drivers/net/mlx4/main.c | 1 + include/linux/mlx4/device.h | 1 + include/linux/mlx4/qp.h | 5 +++ 8 files changed, 79 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index d2e32b0..7d70af7 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -420,6 +420,9 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, case MLX4_OPCODE_BIND_MW: wc->opcode = IB_WC_BIND_MW; break; + case MLX4_OPCODE_LSO: + wc->opcode = IB_WC_LSO; + break; } } else { wc->byte_len = be32_to_cpu(cqe->byte_cnt); diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index ef5e9db..4bc2ca4 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -101,6 +101,9 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) props->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; + if (dev->dev->caps.max_gso_sz) + props->device_cap_flags |= IB_DEVICE_TCP_TSO; + props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 31b2b5b..d5a42e8 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -71,6 +71,7 @@ enum { static const __be32 mlx4_ib_opcode[] = { [IB_WR_SEND] = __constant_cpu_to_be32(MLX4_OPCODE_SEND), + [IB_WR_LSO] = __constant_cpu_to_be32(MLX4_OPCODE_LSO), [IB_WR_SEND_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_SEND_IMM), [IB_WR_RDMA_WRITE] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE), [IB_WR_RDMA_WRITE_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE_IMM), @@ -311,6 +312,7 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, enum ib_qp_type type, struct mlx4_ib_qp *qp) { int s; + int reserve; /* Sanity check SQ size before proceeding */ if (cap->max_send_wr > dev->dev->caps.max_wqes || @@ -327,9 +329,11 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, cap->max_send_sge + 2 > dev->dev->caps.max_sq_sg) return -EINVAL; + reserve = qp->flags & MLX4_QP_LSO ? 64 : 0; + s = max(cap->max_send_sge * sizeof (struct mlx4_wqe_data_seg), cap->max_inline_data + sizeof (struct mlx4_wqe_inline_seg)) + - send_wqe_overhead(type); + send_wqe_overhead(type) + reserve; /* * Hermon supports shrinking WQEs, such that a single work @@ -393,7 +397,7 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, ++qp->sq.wqe_shift; } - qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - + qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - reserve - send_wqe_overhead(type)) / sizeof (struct mlx4_wqe_data_seg); qp->buf_size = (qp->rq.wqe_cnt << qp->rq.wqe_shift) + @@ -503,6 +507,9 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } else { qp->sq_no_prefetch = 0; + if (init_attr->create_flags & QP_CREATE_LSO) + qp->flags |= MLX4_QP_LSO; + err = set_kernel_sq_size(dev, &init_attr->cap, init_attr->qp_type, qp); if (err) goto err; @@ -876,9 +883,15 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, } } - if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI || - ibqp->qp_type == IB_QPT_UD) + if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI) context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + else if (ibqp->qp_type == IB_QPT_UD) { + if (qp->flags & MLX4_QP_LSO) + context->mtu_msgmax = (IB_MTU_4096 << 5) | + ilog2(dev->dev->caps.max_gso_sz); + else + context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + } else if (attr_mask & IB_QP_PATH_MTU) { if (attr->path_mtu < IB_MTU_256 || attr->path_mtu > IB_MTU_4096) { printk(KERN_ERR "path MTU (%u) is invalid\n", @@ -1396,6 +1409,28 @@ static void __set_data_seg(struct mlx4_wqe_data_seg *dseg, struct ib_sge *sg) dseg->addr = cpu_to_be64(sg->addr); } +static int build_lso_seg(struct mlx4_lso_seg *wqe, struct ib_send_wr *wr, + struct mlx4_ib_qp *qp, int *lso_seg_len) +{ + int halign; + + halign = ALIGN(wr->wr.ud.hlen, 16); + if (unlikely(!(qp->flags & MLX4_QP_LSO) && wr->num_sge > qp->sq.max_gs - (halign >> 4))) + return -EINVAL; + + memcpy(wqe->header, wr->wr.ud.header, wr->wr.ud.hlen); + + /* make sure LSO header is written before + overwriting stamping */ + wmb(); + + wqe->mss_hdr_size = cpu_to_be32(((wr->wr.ud.mss - wr->wr.ud.hlen) + << 16) | wr->wr.ud.hlen); + + *lso_seg_len = halign; + return 0; +} + int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, struct ib_send_wr **bad_wr) { @@ -1419,11 +1454,13 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, if (mlx4_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq)) { err = -ENOMEM; *bad_wr = wr; + printk("failed here %d\n", __LINE__); goto out; } if (unlikely(wr->num_sge > qp->sq.max_gs)) { err = -EINVAL; + printk("failed here %d: num_sge=%d, max_gs=%d\n", __LINE__, wr->num_sge, qp->sq.max_gs); *bad_wr = wr; goto out; } @@ -1487,12 +1524,27 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, set_datagram_seg(wqe, wr); wqe += sizeof (struct mlx4_wqe_datagram_seg); size += sizeof (struct mlx4_wqe_datagram_seg) / 16; + + if (wr->opcode == IB_WR_LSO) { + int hlen; + + err = build_lso_seg(wqe, wr, qp, &hlen); + if (err) { + printk("failed here %d\n", __LINE__); + *bad_wr = wr; + goto out; + } + wqe += hlen; + size += hlen >> 4; + } + break; case IB_QPT_SMI: case IB_QPT_GSI: err = build_mlx_header(to_msqp(qp), wr, ctrl); if (err < 0) { + printk("failed here %d\n", __LINE__); *bad_wr = wr; goto out; } diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index f494c3e..d82f275 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -133,6 +133,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) #define QUERY_DEV_CAP_MAX_AV_OFFSET 0x27 #define QUERY_DEV_CAP_MAX_REQ_QP_OFFSET 0x29 #define QUERY_DEV_CAP_MAX_RES_QP_OFFSET 0x2b +#define QUERY_DEV_CAP_MAX_GSO_OFFSET 0x2d #define QUERY_DEV_CAP_MAX_RDMA_OFFSET 0x2f #define QUERY_DEV_CAP_RSZ_SRQ_OFFSET 0x33 #define QUERY_DEV_CAP_ACK_DELAY_OFFSET 0x35 @@ -215,6 +216,13 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_requester_per_qp = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RES_QP_OFFSET); dev_cap->max_responder_per_qp = 1 << (field & 0x3f); + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_GSO_OFFSET); + field &= 0x1f; + if (!field) + dev_cap->max_gso_sz = 0; + else + dev_cap->max_gso_sz = 1 << field; + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RDMA_OFFSET); dev_cap->max_rdma_global = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_ACK_DELAY_OFFSET); @@ -377,6 +385,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_sq_desc_sz, dev_cap->max_sq_sg); mlx4_dbg(dev, "Max RQ desc size: %d, max RQ S/G: %d\n", dev_cap->max_rq_desc_sz, dev_cap->max_rq_sg); + mlx4_dbg(dev, "Max GSO size: %d\n", dev_cap->max_gso_sz); dump_dev_cap_flags(dev, dev_cap->flags); diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index e16dec8..306cb9b 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -96,6 +96,7 @@ struct mlx4_dev_cap { u8 bmme_flags; u32 reserved_lkey; u64 max_icm_sz; + int max_gso_sz; }; struct mlx4_adapter { diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index 08bfc13..7cfbe75 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -159,6 +159,7 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.page_size_cap = ~(u32) (dev_cap->min_page_sz - 1); dev->caps.flags = dev_cap->flags; dev->caps.stat_rate_support = dev_cap->stat_rate_support; + dev->caps.max_gso_sz = dev_cap->max_gso_sz; return 0; } diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 6cdf813..ff7df1a 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -186,6 +186,7 @@ struct mlx4_caps { u32 flags; u16 stat_rate_support; u8 port_width_cap[MLX4_MAX_PORTS + 1]; + int max_gso_sz; }; struct mlx4_buf_list { diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 31f9eb3..cf0bf4e 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -219,6 +219,11 @@ struct mlx4_wqe_datagram_seg { __be32 reservd[2]; }; +struct mlx4_lso_seg { + __be32 mss_hdr_size; + __be32 header[0]; +}; + struct mlx4_wqe_bind_seg { __be32 flags1; __be32 flags2; -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:03 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:03 +0200 Subject: [ofa-general] [PATCH 6/10] IB/mlx4: Add LSO support Message-ID: <1205767443.25950.141.camel@mtls03> >From f60c81b106a5cbbef9d4943012215022e9d7f0b0 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Tue, 11 Mar 2008 15:26:38 +0200 Subject: [PATCH] IB/mlx4: Add LSO support Add LSO support to mlx4 driver such that it will be able to send SKBs passed from the driver which publish NETIF_TSO. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mlx4/cq.c | 3 ++ drivers/infiniband/hw/mlx4/main.c | 3 ++ drivers/infiniband/hw/mlx4/qp.c | 60 ++++++++++++++++++++++++++++++++++-- drivers/net/mlx4/fw.c | 9 +++++ drivers/net/mlx4/fw.h | 1 + drivers/net/mlx4/main.c | 1 + include/linux/mlx4/device.h | 1 + include/linux/mlx4/qp.h | 5 +++ 8 files changed, 79 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index d2e32b0..7d70af7 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -420,6 +420,9 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, case MLX4_OPCODE_BIND_MW: wc->opcode = IB_WC_BIND_MW; break; + case MLX4_OPCODE_LSO: + wc->opcode = IB_WC_LSO; + break; } } else { wc->byte_len = be32_to_cpu(cqe->byte_cnt); diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index ef5e9db..4bc2ca4 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -101,6 +101,9 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) props->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; + if (dev->dev->caps.max_gso_sz) + props->device_cap_flags |= IB_DEVICE_TCP_TSO; + props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 31b2b5b..d5a42e8 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -71,6 +71,7 @@ enum { static const __be32 mlx4_ib_opcode[] = { [IB_WR_SEND] = __constant_cpu_to_be32(MLX4_OPCODE_SEND), + [IB_WR_LSO] = __constant_cpu_to_be32(MLX4_OPCODE_LSO), [IB_WR_SEND_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_SEND_IMM), [IB_WR_RDMA_WRITE] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE), [IB_WR_RDMA_WRITE_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE_IMM), @@ -311,6 +312,7 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, enum ib_qp_type type, struct mlx4_ib_qp *qp) { int s; + int reserve; /* Sanity check SQ size before proceeding */ if (cap->max_send_wr > dev->dev->caps.max_wqes || @@ -327,9 +329,11 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, cap->max_send_sge + 2 > dev->dev->caps.max_sq_sg) return -EINVAL; + reserve = qp->flags & MLX4_QP_LSO ? 64 : 0; + s = max(cap->max_send_sge * sizeof (struct mlx4_wqe_data_seg), cap->max_inline_data + sizeof (struct mlx4_wqe_inline_seg)) + - send_wqe_overhead(type); + send_wqe_overhead(type) + reserve; /* * Hermon supports shrinking WQEs, such that a single work @@ -393,7 +397,7 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, ++qp->sq.wqe_shift; } - qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - + qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - reserve - send_wqe_overhead(type)) / sizeof (struct mlx4_wqe_data_seg); qp->buf_size = (qp->rq.wqe_cnt << qp->rq.wqe_shift) + @@ -503,6 +507,9 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } else { qp->sq_no_prefetch = 0; + if (init_attr->create_flags & QP_CREATE_LSO) + qp->flags |= MLX4_QP_LSO; + err = set_kernel_sq_size(dev, &init_attr->cap, init_attr->qp_type, qp); if (err) goto err; @@ -876,9 +883,15 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, } } - if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI || - ibqp->qp_type == IB_QPT_UD) + if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI) context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + else if (ibqp->qp_type == IB_QPT_UD) { + if (qp->flags & MLX4_QP_LSO) + context->mtu_msgmax = (IB_MTU_4096 << 5) | + ilog2(dev->dev->caps.max_gso_sz); + else + context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + } else if (attr_mask & IB_QP_PATH_MTU) { if (attr->path_mtu < IB_MTU_256 || attr->path_mtu > IB_MTU_4096) { printk(KERN_ERR "path MTU (%u) is invalid\n", @@ -1396,6 +1409,28 @@ static void __set_data_seg(struct mlx4_wqe_data_seg *dseg, struct ib_sge *sg) dseg->addr = cpu_to_be64(sg->addr); } +static int build_lso_seg(struct mlx4_lso_seg *wqe, struct ib_send_wr *wr, + struct mlx4_ib_qp *qp, int *lso_seg_len) +{ + int halign; + + halign = ALIGN(wr->wr.ud.hlen, 16); + if (unlikely(!(qp->flags & MLX4_QP_LSO) && wr->num_sge > qp->sq.max_gs - (halign >> 4))) + return -EINVAL; + + memcpy(wqe->header, wr->wr.ud.header, wr->wr.ud.hlen); + + /* make sure LSO header is written before + overwriting stamping */ + wmb(); + + wqe->mss_hdr_size = cpu_to_be32(((wr->wr.ud.mss - wr->wr.ud.hlen) + << 16) | wr->wr.ud.hlen); + + *lso_seg_len = halign; + return 0; +} + int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, struct ib_send_wr **bad_wr) { @@ -1419,11 +1454,13 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, if (mlx4_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq)) { err = -ENOMEM; *bad_wr = wr; + printk("failed here %d\n", __LINE__); goto out; } if (unlikely(wr->num_sge > qp->sq.max_gs)) { err = -EINVAL; + printk("failed here %d: num_sge=%d, max_gs=%d\n", __LINE__, wr->num_sge, qp->sq.max_gs); *bad_wr = wr; goto out; } @@ -1487,12 +1524,27 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, set_datagram_seg(wqe, wr); wqe += sizeof (struct mlx4_wqe_datagram_seg); size += sizeof (struct mlx4_wqe_datagram_seg) / 16; + + if (wr->opcode == IB_WR_LSO) { + int hlen; + + err = build_lso_seg(wqe, wr, qp, &hlen); + if (err) { + printk("failed here %d\n", __LINE__); + *bad_wr = wr; + goto out; + } + wqe += hlen; + size += hlen >> 4; + } + break; case IB_QPT_SMI: case IB_QPT_GSI: err = build_mlx_header(to_msqp(qp), wr, ctrl); if (err < 0) { + printk("failed here %d\n", __LINE__); *bad_wr = wr; goto out; } diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index f494c3e..d82f275 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -133,6 +133,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) #define QUERY_DEV_CAP_MAX_AV_OFFSET 0x27 #define QUERY_DEV_CAP_MAX_REQ_QP_OFFSET 0x29 #define QUERY_DEV_CAP_MAX_RES_QP_OFFSET 0x2b +#define QUERY_DEV_CAP_MAX_GSO_OFFSET 0x2d #define QUERY_DEV_CAP_MAX_RDMA_OFFSET 0x2f #define QUERY_DEV_CAP_RSZ_SRQ_OFFSET 0x33 #define QUERY_DEV_CAP_ACK_DELAY_OFFSET 0x35 @@ -215,6 +216,13 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_requester_per_qp = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RES_QP_OFFSET); dev_cap->max_responder_per_qp = 1 << (field & 0x3f); + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_GSO_OFFSET); + field &= 0x1f; + if (!field) + dev_cap->max_gso_sz = 0; + else + dev_cap->max_gso_sz = 1 << field; + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RDMA_OFFSET); dev_cap->max_rdma_global = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_ACK_DELAY_OFFSET); @@ -377,6 +385,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_sq_desc_sz, dev_cap->max_sq_sg); mlx4_dbg(dev, "Max RQ desc size: %d, max RQ S/G: %d\n", dev_cap->max_rq_desc_sz, dev_cap->max_rq_sg); + mlx4_dbg(dev, "Max GSO size: %d\n", dev_cap->max_gso_sz); dump_dev_cap_flags(dev, dev_cap->flags); diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index e16dec8..306cb9b 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -96,6 +96,7 @@ struct mlx4_dev_cap { u8 bmme_flags; u32 reserved_lkey; u64 max_icm_sz; + int max_gso_sz; }; struct mlx4_adapter { diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index 08bfc13..7cfbe75 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -159,6 +159,7 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.page_size_cap = ~(u32) (dev_cap->min_page_sz - 1); dev->caps.flags = dev_cap->flags; dev->caps.stat_rate_support = dev_cap->stat_rate_support; + dev->caps.max_gso_sz = dev_cap->max_gso_sz; return 0; } diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 6cdf813..ff7df1a 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -186,6 +186,7 @@ struct mlx4_caps { u32 flags; u16 stat_rate_support; u8 port_width_cap[MLX4_MAX_PORTS + 1]; + int max_gso_sz; }; struct mlx4_buf_list { diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 31f9eb3..cf0bf4e 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -219,6 +219,11 @@ struct mlx4_wqe_datagram_seg { __be32 reservd[2]; }; +struct mlx4_lso_seg { + __be32 mss_hdr_size; + __be32 header[0]; +}; + struct mlx4_wqe_bind_seg { __be32 flags1; __be32 flags2; -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:08 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:08 +0200 Subject: [ofa-general] [PATCH 7/10] IB/ipoib: Add ethtool support Message-ID: <1205767448.25950.142.camel@mtls03> >From 30082b152206333dc90f866473f43b4ab727017b Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 16 Jan 2008 09:50:18 +0200 Subject: [PATCH] IB/ipoib: Add ethtool support Just add the infrastructure so we can add functionality later. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/Makefile | 3 +- drivers/infiniband/ulp/ipoib/ipoib.h | 2 + drivers/infiniband/ulp/ipoib/ipoib_etool.c | 55 ++++++++++++++++++++++++++++ drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 + 4 files changed, 61 insertions(+), 1 deletions(-) create mode 100644 drivers/infiniband/ulp/ipoib/ipoib_etool.c diff --git a/drivers/infiniband/ulp/ipoib/Makefile b/drivers/infiniband/ulp/ipoib/Makefile index 98ee38e..83488ee 100644 --- a/drivers/infiniband/ulp/ipoib/Makefile +++ b/drivers/infiniband/ulp/ipoib/Makefile @@ -4,7 +4,8 @@ ib_ipoib-y := ipoib_main.o \ ipoib_ib.o \ ipoib_multicast.o \ ipoib_verbs.o \ - ipoib_vlan.o + ipoib_vlan.o \ + ipoib_etool.o ib_ipoib-$(CONFIG_INFINIBAND_IPOIB_CM) += ipoib_cm.o ib_ipoib-$(CONFIG_INFINIBAND_IPOIB_DEBUG) += ipoib_fs.o diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 19a41ff..3524d65 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -460,6 +460,8 @@ void ipoib_pkey_poll(struct work_struct *work); int ipoib_pkey_dev_delay_open(struct net_device *dev); void ipoib_drain_cq(struct net_device *dev); +void ipoib_set_ethtool_ops(struct net_device *dev); + #ifdef CONFIG_INFINIBAND_IPOIB_CM #define IPOIB_FLAGS_RC 0x80 diff --git a/drivers/infiniband/ulp/ipoib/ipoib_etool.c b/drivers/infiniband/ulp/ipoib/ipoib_etool.c new file mode 100644 index 0000000..913aea0 --- /dev/null +++ b/drivers/infiniband/ulp/ipoib/ipoib_etool.c @@ -0,0 +1,55 @@ +/* + * Copyright (c) 2007 Mellanox Technologies. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + * $Id: ipoib_etool.c $ + */ + +#include +#include +#include + +#include "ipoib.h" + +static void ipoib_get_drvinfo(struct net_device *netdev, + struct ethtool_drvinfo *drvinfo) +{ + strncpy(drvinfo->driver, "ipoib", sizeof(drvinfo->driver) - 1); +} + +static const struct ethtool_ops ipoib_ethtool_ops = { + .get_drvinfo = ipoib_get_drvinfo, + .get_tso = ethtool_op_get_tso, +}; + +void ipoib_set_ethtool_ops(struct net_device *dev) +{ + SET_ETHTOOL_OPS(dev, &ipoib_ethtool_ops); +} diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index bba286c..92b1513 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -952,6 +952,8 @@ static void ipoib_setup(struct net_device *dev) dev->set_multicast_list = ipoib_set_mcast_list; dev->neigh_setup = ipoib_neigh_setup_dev; + ipoib_set_ethtool_ops(dev); + netif_napi_add(dev, &priv->napi, ipoib_poll, 100); dev->watchdog_timeo = HZ; -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:08 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:08 +0200 Subject: [ofa-general] [PATCH 7/10] IB/ipoib: Add ethtool support Message-ID: <1205767448.25950.142.camel@mtls03> >From 30082b152206333dc90f866473f43b4ab727017b Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 16 Jan 2008 09:50:18 +0200 Subject: [PATCH] IB/ipoib: Add ethtool support Just add the infrastructure so we can add functionality later. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/Makefile | 3 +- drivers/infiniband/ulp/ipoib/ipoib.h | 2 + drivers/infiniband/ulp/ipoib/ipoib_etool.c | 55 ++++++++++++++++++++++++++++ drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 + 4 files changed, 61 insertions(+), 1 deletions(-) create mode 100644 drivers/infiniband/ulp/ipoib/ipoib_etool.c diff --git a/drivers/infiniband/ulp/ipoib/Makefile b/drivers/infiniband/ulp/ipoib/Makefile index 98ee38e..83488ee 100644 --- a/drivers/infiniband/ulp/ipoib/Makefile +++ b/drivers/infiniband/ulp/ipoib/Makefile @@ -4,7 +4,8 @@ ib_ipoib-y := ipoib_main.o \ ipoib_ib.o \ ipoib_multicast.o \ ipoib_verbs.o \ - ipoib_vlan.o + ipoib_vlan.o \ + ipoib_etool.o ib_ipoib-$(CONFIG_INFINIBAND_IPOIB_CM) += ipoib_cm.o ib_ipoib-$(CONFIG_INFINIBAND_IPOIB_DEBUG) += ipoib_fs.o diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 19a41ff..3524d65 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -460,6 +460,8 @@ void ipoib_pkey_poll(struct work_struct *work); int ipoib_pkey_dev_delay_open(struct net_device *dev); void ipoib_drain_cq(struct net_device *dev); +void ipoib_set_ethtool_ops(struct net_device *dev); + #ifdef CONFIG_INFINIBAND_IPOIB_CM #define IPOIB_FLAGS_RC 0x80 diff --git a/drivers/infiniband/ulp/ipoib/ipoib_etool.c b/drivers/infiniband/ulp/ipoib/ipoib_etool.c new file mode 100644 index 0000000..913aea0 --- /dev/null +++ b/drivers/infiniband/ulp/ipoib/ipoib_etool.c @@ -0,0 +1,55 @@ +/* + * Copyright (c) 2007 Mellanox Technologies. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + * $Id: ipoib_etool.c $ + */ + +#include +#include +#include + +#include "ipoib.h" + +static void ipoib_get_drvinfo(struct net_device *netdev, + struct ethtool_drvinfo *drvinfo) +{ + strncpy(drvinfo->driver, "ipoib", sizeof(drvinfo->driver) - 1); +} + +static const struct ethtool_ops ipoib_ethtool_ops = { + .get_drvinfo = ipoib_get_drvinfo, + .get_tso = ethtool_op_get_tso, +}; + +void ipoib_set_ethtool_ops(struct net_device *dev) +{ + SET_ETHTOOL_OPS(dev, &ipoib_ethtool_ops); +} diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index bba286c..92b1513 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -952,6 +952,8 @@ static void ipoib_setup(struct net_device *dev) dev->set_multicast_list = ipoib_set_mcast_list; dev->neigh_setup = ipoib_neigh_setup_dev; + ipoib_set_ethtool_ops(dev); + netif_napi_add(dev, &priv->napi, ipoib_poll, 100); dev->watchdog_timeo = HZ; -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:11 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:11 +0200 Subject: [ofa-general] [PATCH 8/10] IB/core: Add support for modify CQ Message-ID: <1205767451.25950.143.camel@mtls03> >From aec43b1ea5b4702bcbdb6b6b53677014cc2d5fb8 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 16 Jan 2008 09:59:19 +0200 Subject: [PATCH] IB/core: Add support for modify CQ Add support for modifying CQ parameters for controlling event generation moderation. Signed-off-by: Eli Cohen --- drivers/infiniband/core/verbs.c | 7 +++++++ include/rdma/ib_verbs.h | 11 +++++++++++ 2 files changed, 18 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 86ed8af..84709ed 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -628,6 +628,13 @@ struct ib_cq *ib_create_cq(struct ib_device *device, } EXPORT_SYMBOL(ib_create_cq); +int ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period) +{ + return cq->device->modify_cq ? + cq->device->modify_cq(cq, cq_count, cq_period) : -ENOSYS; +} +EXPORT_SYMBOL(ib_modify_cq); + int ib_destroy_cq(struct ib_cq *cq) { if (atomic_read(&cq->usecnt)) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 9557d10..e35c914 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -983,6 +983,8 @@ struct ib_device { int comp_vector, struct ib_ucontext *context, struct ib_udata *udata); + int (*modify_cq)(struct ib_cq *cq, u16 cq_count, + u16 cq_period); int (*destroy_cq)(struct ib_cq *cq); int (*resize_cq)(struct ib_cq *cq, int cqe, struct ib_udata *udata); @@ -1388,6 +1390,15 @@ struct ib_cq *ib_create_cq(struct ib_device *device, int ib_resize_cq(struct ib_cq *cq, int cqe); /** + * ib_modify_cq - Modifies moderation params of the CQ + * @cq: The CQ to modify. + * @cq_count: number of CQEs that will trigger an event + * @cq_period: max period of time in usec before triggering an event + * + */ +int ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); + +/** * ib_destroy_cq - Destroys the specified CQ. * @cq: The CQ to destroy. */ -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:11 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:11 +0200 Subject: [ofa-general] [PATCH 8/10] IB/core: Add support for modify CQ Message-ID: <1205767451.25950.143.camel@mtls03> >From aec43b1ea5b4702bcbdb6b6b53677014cc2d5fb8 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 16 Jan 2008 09:59:19 +0200 Subject: [PATCH] IB/core: Add support for modify CQ Add support for modifying CQ parameters for controlling event generation moderation. Signed-off-by: Eli Cohen --- drivers/infiniband/core/verbs.c | 7 +++++++ include/rdma/ib_verbs.h | 11 +++++++++++ 2 files changed, 18 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 86ed8af..84709ed 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -628,6 +628,13 @@ struct ib_cq *ib_create_cq(struct ib_device *device, } EXPORT_SYMBOL(ib_create_cq); +int ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period) +{ + return cq->device->modify_cq ? + cq->device->modify_cq(cq, cq_count, cq_period) : -ENOSYS; +} +EXPORT_SYMBOL(ib_modify_cq); + int ib_destroy_cq(struct ib_cq *cq) { if (atomic_read(&cq->usecnt)) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 9557d10..e35c914 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -983,6 +983,8 @@ struct ib_device { int comp_vector, struct ib_ucontext *context, struct ib_udata *udata); + int (*modify_cq)(struct ib_cq *cq, u16 cq_count, + u16 cq_period); int (*destroy_cq)(struct ib_cq *cq); int (*resize_cq)(struct ib_cq *cq, int cqe, struct ib_udata *udata); @@ -1388,6 +1390,15 @@ struct ib_cq *ib_create_cq(struct ib_device *device, int ib_resize_cq(struct ib_cq *cq, int cqe); /** + * ib_modify_cq - Modifies moderation params of the CQ + * @cq: The CQ to modify. + * @cq_count: number of CQEs that will trigger an event + * @cq_period: max period of time in usec before triggering an event + * + */ +int ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); + +/** * ib_destroy_cq - Destroys the specified CQ. * @cq: The CQ to destroy. */ -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:25 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:25 +0200 Subject: [ofa-general] [PATCH 10/10] IB/mlx4: add support for modifying CQ parameters Message-ID: <1205767465.25950.144.camel@mtls03> >From 20b311c1beb7da8b3294635ab6fc1023aab0b3b3 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 16 Jan 2008 10:18:54 +0200 Subject: [PATCH] IB/mlx4: add support for modifying CQ parameters This patch adds support for modifying the count and period moderation parameters of a CQ. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mlx4/cq.c | 19 +++++++++++++ drivers/infiniband/hw/mlx4/main.c | 1 + drivers/infiniband/hw/mlx4/mlx4_ib.h | 1 + drivers/net/mlx4/cq.c | 49 ++++++++++++++++++---------------- include/linux/mlx4/cmd.h | 2 +- include/linux/mlx4/cq.h | 25 +++++++++++++++++ 6 files changed, 73 insertions(+), 24 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 7d70af7..e71186b 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -85,6 +85,25 @@ static struct mlx4_cqe *next_cqe_sw(struct mlx4_ib_cq *cq) return get_sw_cqe(cq, cq->mcq.cons_index); } +int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period) +{ + struct mlx4_ib_cq *mcq = to_mcq(cq); + struct mlx4_ib_dev *dev = to_mdev(cq->device); + struct mlx4_cq_context *context; + int err; + + context = kzalloc(sizeof *context, GFP_KERNEL); + if (!context) + return -ENOMEM; + + context->cq_period = cpu_to_be16(cq_period); + context->cq_max_count = cpu_to_be16(cq_count); + err = mlx4_cq_modify(dev->dev, &mcq->mcq, context, 1); + + kfree(context); + return err; +} + struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector, struct ib_ucontext *context, struct ib_udata *udata) diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 4bc2ca4..3487e9c 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -610,6 +610,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.post_send = mlx4_ib_post_send; ibdev->ib_dev.post_recv = mlx4_ib_post_recv; ibdev->ib_dev.create_cq = mlx4_ib_create_cq; + ibdev->ib_dev.modify_cq = mlx4_ib_modify_cq; ibdev->ib_dev.destroy_cq = mlx4_ib_destroy_cq; ibdev->ib_dev.poll_cq = mlx4_ib_poll_cq; ibdev->ib_dev.req_notify_cq = mlx4_ib_arm_cq; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 0d7ccbc..ccb95c2 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -254,6 +254,7 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, struct ib_udata *udata); int mlx4_ib_dereg_mr(struct ib_mr *mr); +int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector, struct ib_ucontext *context, struct ib_udata *udata); diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index d4441fe..39004c5 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -38,33 +38,11 @@ #include #include +#include #include "mlx4.h" #include "icm.h" -struct mlx4_cq_context { - __be32 flags; - u16 reserved1[3]; - __be16 page_offset; - __be32 logsize_usrpage; - u8 reserved2; - u8 cq_period; - u8 reserved3; - u8 cq_max_count; - u8 reserved4[3]; - u8 comp_eqn; - u8 log_page_size; - u8 reserved5[2]; - u8 mtt_base_addr_h; - __be32 mtt_base_addr_l; - __be32 last_notified_index; - __be32 solicit_producer_index; - __be32 consumer_index; - __be32 producer_index; - u32 reserved6[2]; - __be64 db_rec_addr; -}; - #define MLX4_CQ_STATUS_OK ( 0 << 28) #define MLX4_CQ_STATUS_OVERFLOW ( 9 << 28) #define MLX4_CQ_STATUS_WRITE_FAIL (10 << 28) @@ -121,6 +99,13 @@ static int mlx4_SW2HW_CQ(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox, MLX4_CMD_TIME_CLASS_A); } +static int mlx4_MODIFY_CQ(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox, + int cq_num, u32 opmod) +{ + return mlx4_cmd(dev, mailbox->dma, cq_num, opmod, MLX4_CMD_MODIFY_CQ, + MLX4_CMD_TIME_CLASS_A); +} + static int mlx4_HW2SW_CQ(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox, int cq_num) { @@ -206,6 +191,24 @@ err_out: } EXPORT_SYMBOL_GPL(mlx4_cq_alloc); +int mlx4_cq_modify(struct mlx4_dev *dev, struct mlx4_cq *cq, + struct mlx4_cq_context *context, int modify) +{ + struct mlx4_cmd_mailbox *mailbox; + int err; + + mailbox = mlx4_alloc_cmd_mailbox(dev); + if (IS_ERR(mailbox)) + return PTR_ERR(mailbox); + + memcpy(mailbox->buf, context, sizeof *context); + err = mlx4_MODIFY_CQ(dev, mailbox, cq->cqn, modify); + + mlx4_free_cmd_mailbox(dev, mailbox); + return err; +} +EXPORT_SYMBOL_GPL(mlx4_cq_modify); + void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq) { struct mlx4_priv *priv = mlx4_priv(dev); diff --git a/include/linux/mlx4/cmd.h b/include/linux/mlx4/cmd.h index 7d1eaa9..77323a7 100644 --- a/include/linux/mlx4/cmd.h +++ b/include/linux/mlx4/cmd.h @@ -81,7 +81,7 @@ enum { MLX4_CMD_SW2HW_CQ = 0x16, MLX4_CMD_HW2SW_CQ = 0x17, MLX4_CMD_QUERY_CQ = 0x18, - MLX4_CMD_RESIZE_CQ = 0x2c, + MLX4_CMD_MODIFY_CQ = 0x2c, /* SRQ commands */ MLX4_CMD_SW2HW_SRQ = 0x35, diff --git a/include/linux/mlx4/cq.h b/include/linux/mlx4/cq.h index 1243eba..9e6ed5b 100644 --- a/include/linux/mlx4/cq.h +++ b/include/linux/mlx4/cq.h @@ -38,6 +38,27 @@ #include #include +struct mlx4_cq_context { + __be32 flags; + u16 reserved1[3]; + __be16 page_offset; + __be32 logsize_usrpage; + u16 cq_period; + u16 cq_max_count; + u8 reserved4[3]; + u8 comp_eqn; + u8 log_page_size; + u8 reserved5[2]; + u8 mtt_base_addr_h; + __be32 mtt_base_addr_l; + __be32 last_notified_index; + __be32 solicit_producer_index; + __be32 consumer_index; + __be32 producer_index; + u32 reserved6[2]; + __be64 db_rec_addr; +}; + struct mlx4_cqe { __be32 my_qpn; __be32 immed_rss_invalid; @@ -130,4 +151,8 @@ enum { MLX4_CQ_DB_REQ_NOT = 2 << 24 }; + +int mlx4_cq_modify(struct mlx4_dev *dev, struct mlx4_cq *cq, + struct mlx4_cq_context *context, int resize); + #endif /* MLX4_CQ_H */ -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:24:25 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:24:25 +0200 Subject: [ofa-general] [PATCH 10/10] IB/mlx4: add support for modifying CQ parameters Message-ID: <1205767465.25950.144.camel@mtls03> >From 20b311c1beb7da8b3294635ab6fc1023aab0b3b3 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Wed, 16 Jan 2008 10:18:54 +0200 Subject: [PATCH] IB/mlx4: add support for modifying CQ parameters This patch adds support for modifying the count and period moderation parameters of a CQ. Signed-off-by: Eli Cohen --- drivers/infiniband/hw/mlx4/cq.c | 19 +++++++++++++ drivers/infiniband/hw/mlx4/main.c | 1 + drivers/infiniband/hw/mlx4/mlx4_ib.h | 1 + drivers/net/mlx4/cq.c | 49 ++++++++++++++++++---------------- include/linux/mlx4/cmd.h | 2 +- include/linux/mlx4/cq.h | 25 +++++++++++++++++ 6 files changed, 73 insertions(+), 24 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 7d70af7..e71186b 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -85,6 +85,25 @@ static struct mlx4_cqe *next_cqe_sw(struct mlx4_ib_cq *cq) return get_sw_cqe(cq, cq->mcq.cons_index); } +int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period) +{ + struct mlx4_ib_cq *mcq = to_mcq(cq); + struct mlx4_ib_dev *dev = to_mdev(cq->device); + struct mlx4_cq_context *context; + int err; + + context = kzalloc(sizeof *context, GFP_KERNEL); + if (!context) + return -ENOMEM; + + context->cq_period = cpu_to_be16(cq_period); + context->cq_max_count = cpu_to_be16(cq_count); + err = mlx4_cq_modify(dev->dev, &mcq->mcq, context, 1); + + kfree(context); + return err; +} + struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector, struct ib_ucontext *context, struct ib_udata *udata) diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 4bc2ca4..3487e9c 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -610,6 +610,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.post_send = mlx4_ib_post_send; ibdev->ib_dev.post_recv = mlx4_ib_post_recv; ibdev->ib_dev.create_cq = mlx4_ib_create_cq; + ibdev->ib_dev.modify_cq = mlx4_ib_modify_cq; ibdev->ib_dev.destroy_cq = mlx4_ib_destroy_cq; ibdev->ib_dev.poll_cq = mlx4_ib_poll_cq; ibdev->ib_dev.req_notify_cq = mlx4_ib_arm_cq; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 0d7ccbc..ccb95c2 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -254,6 +254,7 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, struct ib_udata *udata); int mlx4_ib_dereg_mr(struct ib_mr *mr); +int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector, struct ib_ucontext *context, struct ib_udata *udata); diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index d4441fe..39004c5 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -38,33 +38,11 @@ #include #include +#include #include "mlx4.h" #include "icm.h" -struct mlx4_cq_context { - __be32 flags; - u16 reserved1[3]; - __be16 page_offset; - __be32 logsize_usrpage; - u8 reserved2; - u8 cq_period; - u8 reserved3; - u8 cq_max_count; - u8 reserved4[3]; - u8 comp_eqn; - u8 log_page_size; - u8 reserved5[2]; - u8 mtt_base_addr_h; - __be32 mtt_base_addr_l; - __be32 last_notified_index; - __be32 solicit_producer_index; - __be32 consumer_index; - __be32 producer_index; - u32 reserved6[2]; - __be64 db_rec_addr; -}; - #define MLX4_CQ_STATUS_OK ( 0 << 28) #define MLX4_CQ_STATUS_OVERFLOW ( 9 << 28) #define MLX4_CQ_STATUS_WRITE_FAIL (10 << 28) @@ -121,6 +99,13 @@ static int mlx4_SW2HW_CQ(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox, MLX4_CMD_TIME_CLASS_A); } +static int mlx4_MODIFY_CQ(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox, + int cq_num, u32 opmod) +{ + return mlx4_cmd(dev, mailbox->dma, cq_num, opmod, MLX4_CMD_MODIFY_CQ, + MLX4_CMD_TIME_CLASS_A); +} + static int mlx4_HW2SW_CQ(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox, int cq_num) { @@ -206,6 +191,24 @@ err_out: } EXPORT_SYMBOL_GPL(mlx4_cq_alloc); +int mlx4_cq_modify(struct mlx4_dev *dev, struct mlx4_cq *cq, + struct mlx4_cq_context *context, int modify) +{ + struct mlx4_cmd_mailbox *mailbox; + int err; + + mailbox = mlx4_alloc_cmd_mailbox(dev); + if (IS_ERR(mailbox)) + return PTR_ERR(mailbox); + + memcpy(mailbox->buf, context, sizeof *context); + err = mlx4_MODIFY_CQ(dev, mailbox, cq->cqn, modify); + + mlx4_free_cmd_mailbox(dev, mailbox); + return err; +} +EXPORT_SYMBOL_GPL(mlx4_cq_modify); + void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq) { struct mlx4_priv *priv = mlx4_priv(dev); diff --git a/include/linux/mlx4/cmd.h b/include/linux/mlx4/cmd.h index 7d1eaa9..77323a7 100644 --- a/include/linux/mlx4/cmd.h +++ b/include/linux/mlx4/cmd.h @@ -81,7 +81,7 @@ enum { MLX4_CMD_SW2HW_CQ = 0x16, MLX4_CMD_HW2SW_CQ = 0x17, MLX4_CMD_QUERY_CQ = 0x18, - MLX4_CMD_RESIZE_CQ = 0x2c, + MLX4_CMD_MODIFY_CQ = 0x2c, /* SRQ commands */ MLX4_CMD_SW2HW_SRQ = 0x35, diff --git a/include/linux/mlx4/cq.h b/include/linux/mlx4/cq.h index 1243eba..9e6ed5b 100644 --- a/include/linux/mlx4/cq.h +++ b/include/linux/mlx4/cq.h @@ -38,6 +38,27 @@ #include #include +struct mlx4_cq_context { + __be32 flags; + u16 reserved1[3]; + __be16 page_offset; + __be32 logsize_usrpage; + u16 cq_period; + u16 cq_max_count; + u8 reserved4[3]; + u8 comp_eqn; + u8 log_page_size; + u8 reserved5[2]; + u8 mtt_base_addr_h; + __be32 mtt_base_addr_l; + __be32 last_notified_index; + __be32 solicit_producer_index; + __be32 consumer_index; + __be32 producer_index; + u32 reserved6[2]; + __be64 db_rec_addr; +}; + struct mlx4_cqe { __be32 my_qpn; __be32 immed_rss_invalid; @@ -130,4 +151,8 @@ enum { MLX4_CQ_DB_REQ_NOT = 2 << 24 }; + +int mlx4_cq_modify(struct mlx4_dev *dev, struct mlx4_cq *cq, + struct mlx4_cq_context *context, int resize); + #endif /* MLX4_CQ_H */ -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:28:09 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:28:09 +0200 Subject: [ofa-general] [PATCH 9/10] IB/ipoib: Support modifying IPOIB CQ moderation params Message-ID: <1205767689.25950.146.camel@mtls03> >From cdece401306cc86c5eb581b4ebc1c4efe05e1b48 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Mon, 17 Mar 2008 16:01:33 +0200 Subject: [PATCH] IB/ipoib: Support modifying IPOIB CQ moderation params This can be used to tune at run time the paramters controlling the event (interrupt) generation rate and thus reduce the overhead incurred by handling interrupts resulting in better throughput. Since IPOIB uses a single CQ for both rx and tx, rx is chosen to dictate configuration for both rx and tx. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 6 ++++ drivers/infiniband/ulp/ipoib/ipoib_etool.c | 46 ++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 3524d65..43feffc 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -242,6 +242,11 @@ struct ipoib_cm_dev_priv { int num_frags; }; +struct ipoib_ethtool_st { + u16 coalesce_usecs; + u16 max_coalesced_frames; +}; + /* * Device private locking: tx_lock protects members used in TX fast * path (and we use LLTX so upper layers don't do extra locking). @@ -320,6 +325,7 @@ struct ipoib_dev_priv { struct dentry *path_dentry; #endif int hca_caps; + struct ipoib_ethtool_st etool; }; struct ipoib_ah { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_etool.c b/drivers/infiniband/ulp/ipoib/ipoib_etool.c index 913aea0..a3ac4cf 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_etool.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_etool.c @@ -44,9 +44,55 @@ static void ipoib_get_drvinfo(struct net_device *netdev, strncpy(drvinfo->driver, "ipoib", sizeof(drvinfo->driver) - 1); } +static int ipoib_get_coalesce(struct net_device *dev, + struct ethtool_coalesce *coal) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + + coal->rx_coalesce_usecs = priv->etool.coalesce_usecs; + coal->tx_coalesce_usecs = priv->etool.coalesce_usecs; + coal->rx_max_coalesced_frames = priv->etool.max_coalesced_frames; + coal->tx_max_coalesced_frames = priv->etool.max_coalesced_frames; + + return 0; +} + +static int ipoib_set_coalesce(struct net_device *dev, + struct ethtool_coalesce *coal) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int ret; + + /* + * since ipoib uses a single CQ for both rx and tx, + * we assume that rx params dictate the configuration. + * These values are saved in the private data and returned + * when ipoib_get_coalesce is called + */ + if (coal->rx_coalesce_usecs > 0xffff || + coal->rx_max_coalesced_frames > 0xffff) + return -EINVAL; + + ret = ib_modify_cq(priv->cq, coal->rx_max_coalesced_frames, + coal->rx_coalesce_usecs); + if (ret) { + ipoib_dbg(priv, "failed modifying CQ\n"); + return ret; + } + + coal->tx_coalesce_usecs = coal->rx_coalesce_usecs; + priv->etool.coalesce_usecs = coal->rx_coalesce_usecs; + coal->tx_max_coalesced_frames = coal->rx_max_coalesced_frames; + priv->etool.max_coalesced_frames = coal->rx_max_coalesced_frames; + + return 0; +} + static const struct ethtool_ops ipoib_ethtool_ops = { .get_drvinfo = ipoib_get_drvinfo, .get_tso = ethtool_op_get_tso, + .get_coalesce = ipoib_get_coalesce, + .set_coalesce = ipoib_set_coalesce, }; void ipoib_set_ethtool_ops(struct net_device *dev) -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 17 08:28:09 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 17 Mar 2008 17:28:09 +0200 Subject: [ofa-general] [PATCH 9/10] IB/ipoib: Support modifying IPOIB CQ moderation params Message-ID: <1205767689.25950.146.camel@mtls03> >From cdece401306cc86c5eb581b4ebc1c4efe05e1b48 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Mon, 17 Mar 2008 16:01:33 +0200 Subject: [PATCH] IB/ipoib: Support modifying IPOIB CQ moderation params This can be used to tune at run time the paramters controlling the event (interrupt) generation rate and thus reduce the overhead incurred by handling interrupts resulting in better throughput. Since IPOIB uses a single CQ for both rx and tx, rx is chosen to dictate configuration for both rx and tx. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 6 ++++ drivers/infiniband/ulp/ipoib/ipoib_etool.c | 46 ++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 3524d65..43feffc 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -242,6 +242,11 @@ struct ipoib_cm_dev_priv { int num_frags; }; +struct ipoib_ethtool_st { + u16 coalesce_usecs; + u16 max_coalesced_frames; +}; + /* * Device private locking: tx_lock protects members used in TX fast * path (and we use LLTX so upper layers don't do extra locking). @@ -320,6 +325,7 @@ struct ipoib_dev_priv { struct dentry *path_dentry; #endif int hca_caps; + struct ipoib_ethtool_st etool; }; struct ipoib_ah { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_etool.c b/drivers/infiniband/ulp/ipoib/ipoib_etool.c index 913aea0..a3ac4cf 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_etool.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_etool.c @@ -44,9 +44,55 @@ static void ipoib_get_drvinfo(struct net_device *netdev, strncpy(drvinfo->driver, "ipoib", sizeof(drvinfo->driver) - 1); } +static int ipoib_get_coalesce(struct net_device *dev, + struct ethtool_coalesce *coal) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + + coal->rx_coalesce_usecs = priv->etool.coalesce_usecs; + coal->tx_coalesce_usecs = priv->etool.coalesce_usecs; + coal->rx_max_coalesced_frames = priv->etool.max_coalesced_frames; + coal->tx_max_coalesced_frames = priv->etool.max_coalesced_frames; + + return 0; +} + +static int ipoib_set_coalesce(struct net_device *dev, + struct ethtool_coalesce *coal) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int ret; + + /* + * since ipoib uses a single CQ for both rx and tx, + * we assume that rx params dictate the configuration. + * These values are saved in the private data and returned + * when ipoib_get_coalesce is called + */ + if (coal->rx_coalesce_usecs > 0xffff || + coal->rx_max_coalesced_frames > 0xffff) + return -EINVAL; + + ret = ib_modify_cq(priv->cq, coal->rx_max_coalesced_frames, + coal->rx_coalesce_usecs); + if (ret) { + ipoib_dbg(priv, "failed modifying CQ\n"); + return ret; + } + + coal->tx_coalesce_usecs = coal->rx_coalesce_usecs; + priv->etool.coalesce_usecs = coal->rx_coalesce_usecs; + coal->tx_max_coalesced_frames = coal->rx_max_coalesced_frames; + priv->etool.max_coalesced_frames = coal->rx_max_coalesced_frames; + + return 0; +} + static const struct ethtool_ops ipoib_ethtool_ops = { .get_drvinfo = ipoib_get_drvinfo, .get_tso = ethtool_op_get_tso, + .get_coalesce = ipoib_get_coalesce, + .set_coalesce = ipoib_set_coalesce, }; void ipoib_set_ethtool_ops(struct net_device *dev) -- 1.5.4.4 From teixeira_henrique at yahoo.com.br Mon Mar 17 08:48:34 2008 From: teixeira_henrique at yahoo.com.br (Kerry Joyner) Date: Tue, 18 Mar 2008 00:48:34 +0900 Subject: [ofa-general] Smart in bed games Message-ID: <01c88891$cb269d00$2147493d@teixeira_henrique> Be like sex machine Look attached details or find more on http://psilofadcored.com -------------- next part -------------- A non-text attachment was scrubbed... Name: vs.zip Type: application/zip Size: 635 bytes Desc: not available URL: From _c at addld.com Mon Mar 17 08:02:59 2008 From: _c at addld.com (hayes kacy) Date: Mon, 17 Mar 2008 15:02:59 +0000 Subject: [ofa-general] 75% discount. Coupon #zWf6 Message-ID: <36615.avery@thuy> Hi openib-general, be intelligent, buy your drugs from the most reliable supplier since 1997. http://www.google.com/pagead/iclk?sa=l&ai=WqmBWU&num=49389&adurl=http://weOA.sugaronly.com?aaGzM emmy sandy From rajouri.jammu at gmail.com Mon Mar 17 10:03:59 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Mon, 17 Mar 2008 10:03:59 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <200803171212.35173.jackm@dev.mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> <200803171212.35173.jackm@dev.mellanox.co.il> Message-ID: <3307cdf90803171003nccd1846qe824cd42292b7075@mail.gmail.com> Thanks for all the responses. If I have to register large amounts of memory from user space, it's still unclear to me what's the best (predictable) way to do it today? Since there is a limit, how do I figure out the exact limit? As I was able to register almost 20GB in 10MB chunks I'm not sure why I didn't hit the 2^20*4K limit on the MTTs. Any clarifications would be most helpful. thanks! On Mon, Mar 17, 2008 at 3:12 AM, Jack Morgenstein wrote: > On Monday 17 March 2008 09:22, Or Gerlitz wrote: > > Rajouri Jammu wrote: > > > Does that mean that no matter how I size my memory regions the maximum > > > amount of (total) memory I can register is = 2^20 * 4K = 4GB? > > > I.e., Am I limited by the total number of MTTs? > > Generally speaking, yes, the IB network MMU (MTT this case) is limited > > in the number of slots it has and so in the amount of memory it can map > > at once, but I assume this is typical eg for I/O MMUs and makes sense. > > > > However, each slot can map --more-- then 4K, so one should be able to > > use huge-pages etc. I am not sure what is the actual status of > > registering huge-pages by the Linux IB stack, maybe Roland can comment > > on that. > > > > Or. > > In Mellanox HCAs, each memory region (i.e., mpt) can specify how many > pages > each mtt entry it uses represents. This would allow a much larger number > of > large regions to be mapped (essentially eliminating the current > limitation). > > Currently, this feature is not supported by the driver code. > > - Jack > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajouri.jammu at gmail.com Mon Mar 17 10:03:59 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Mon, 17 Mar 2008 10:03:59 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <200803171212.35173.jackm@dev.mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <3307cdf90803170008v1c6d61b8me403929472f0f089@mail.gmail.com> <47DE1C41.9080507@voltaire.com> <200803171212.35173.jackm@dev.mellanox.co.il> Message-ID: <3307cdf90803171003nccd1846qe824cd42292b7075@mail.gmail.com> Thanks for all the responses. If I have to register large amounts of memory from user space, it's still unclear to me what's the best (predictable) way to do it today? Since there is a limit, how do I figure out the exact limit? As I was able to register almost 20GB in 10MB chunks I'm not sure why I didn't hit the 2^20*4K limit on the MTTs. Any clarifications would be most helpful. thanks! On Mon, Mar 17, 2008 at 3:12 AM, Jack Morgenstein wrote: > On Monday 17 March 2008 09:22, Or Gerlitz wrote: > > Rajouri Jammu wrote: > > > Does that mean that no matter how I size my memory regions the maximum > > > amount of (total) memory I can register is = 2^20 * 4K = 4GB? > > > I.e., Am I limited by the total number of MTTs? > > Generally speaking, yes, the IB network MMU (MTT this case) is limited > > in the number of slots it has and so in the amount of memory it can map > > at once, but I assume this is typical eg for I/O MMUs and makes sense. > > > > However, each slot can map --more-- then 4K, so one should be able to > > use huge-pages etc. I am not sure what is the actual status of > > registering huge-pages by the Linux IB stack, maybe Roland can comment > > on that. > > > > Or. > > In Mellanox HCAs, each memory region (i.e., mpt) can specify how many > pages > each mtt entry it uses represents. This would allow a much larger number > of > large regions to be mapped (essentially eliminating the current > limitation). > > Currently, this feature is not supported by the driver code. > > - Jack > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris at mellanox.com Mon Mar 17 10:07:41 2008 From: boris at mellanox.com (Boris Shpolyansky) Date: Mon, 17 Mar 2008 10:07:41 -0700 Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface In-Reply-To: Message-ID: <1E3DCD1C63492545881FACB6063A57C102221AD0@mtiexch01.mti.com> Check : /sys/class/infiniband/[mlx4_*|mthca*]/ports/*/counters/* Boris Shpolyansky Sr. Member of Technical Staff Applications Mellanox Technologies Inc. 2900 Stender Way Santa Clara, CA 95054 Tel.: (408) 916 0014 Fax: (408) 970 3403 Cell: (408) 834 9365 www.mellanox.com -----Original Message----- From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Bart Van Assche Sent: Monday, March 17, 2008 8:09 AM To: Openib-General Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface Hello, Is it possible to obtain statistics about the amount of RDMA traffic that passed over an InfiniBand interface ? While it is easy to obtain the number of bytes that has been communicated via IPoIB (e.g. via the command head /sys/class/net/ib*/statistics/*), I do not know of any way to observe the non-IPoIB traffic that passes over an InfiniBand interface. This would be convenient to observe iSER / SRP traffic. Bart. _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From orangutanea at bgfleming.com Mon Mar 17 10:43:36 2008 From: orangutanea at bgfleming.com (Nora Barajas) Date: Mon, 17 Mar 2008 10:43:36 -0700 Subject: [ofa-general] Vitamin E 20 IU. Message-ID: <01c8881b$c1415d00$72d57697@orangutanea> There are stories and myths of penis size from around the world. http://nadinevausee290.blogspot.com From maxymus at safe-mail.net Mon Mar 17 10:47:18 2008 From: maxymus at safe-mail.net (Maxie Sharp) Date: Mon, 17 Mar 2008 12:47:18 -0500 Subject: [ofa-general] {good_canada_subject} Message-ID: <01c8882d$08a50f00$722990be@maxymus> The list of products prescribed by your therapist is big enough to make you go bankrupt? Visit CanadianPharmacy - #1 discount Canadian pharmacy and purchase all the products you need at a price much lower than that in any licensed USA pharmacy. CanadianPharmacy - real generic quality at best prices.http://geocities.com/bean.marshall/ - your #1 source for cheap generic drugs from Canada.With best regards, Maxie Sharp -------------- next part -------------- An HTML attachment was scrubbed... URL: From nirgaskickerjyl at gaskicker.com Mon Mar 17 10:52:58 2008 From: nirgaskickerjyl at gaskicker.com (Brooke Thomson) Date: Tue, 18 Mar 2008 00:52:58 +0700 Subject: [ofa-general] Legal software sales Message-ID: <220774232.90897055715309@gaskicker.com> Our aim is to render low price PC and Mac legal soft and computer solutions for any budget. Whether you're a corporate customer, a small business holder, or shopping for your own personal computer, we believe that we will assist you. COME TO US JUST NOW! http://isabellaechevarrianu757.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From orenk at dev.mellanox.co.il Mon Mar 17 11:00:44 2008 From: orenk at dev.mellanox.co.il (Oren Kladnitsky) Date: Mon, 17 Mar 2008 20:00:44 +0200 Subject: [ofa-general] Re: ibutils binaries configuration and installation questions In-Reply-To: <1205350506.28072.95.camel@hrosenstock-ws.xsigo.com> References: <1205350506.28072.95.camel@hrosenstock-ws.xsigo.com> Message-ID: <47DEB1CC.6090201@dev.mellanox.co.il> Hal Rosenstock wrote: > Hi Oren, > > Are the ibutils binaries meant to be run by root (like the other > management utilities) ? If so, by default, they should be in sbin rather > than bin. Also, is there a configure option for where the binaries go ? > > Thanks. > > -- Hal > Hi. 1. Yes - The diag apps should be run as root. The "real" restriction is done by the umad interface permissions. I can move these apps to sbin dir. 2. Yes - Autotools configure "--bindir" flag can be set to install the binaries to /usr/sbin for example. From dillowda at ornl.gov Mon Mar 17 11:18:00 2008 From: dillowda at ornl.gov (David Dillow) Date: Mon, 17 Mar 2008 14:18:00 -0400 Subject: [ofa-general] [PATCH] IB/srp: enforce protocol limit on srp_sg_tablesize Message-ID: <1205777880.10010.26.camel@lap75545.ornl.gov> The current SRP initiator will allow unlimited s/g entries in the indirect descriptors lists, but the entry count field in the SRP_CMD request is 8 bits, so setting srp_sg_tablesize too large will open the possibility of wrapping the count and generating invalid requests. Enforce the protocol limits to prevent surprises. Reported-by: Martin W. Schlining III Signed-off-by: David Dillow --- ib_srp.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index fd4a49f..da6fd6d 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -68,7 +68,7 @@ static int srp_max_iu_len; module_param(srp_sg_tablesize, int, 0444); MODULE_PARM_DESC(srp_sg_tablesize, - "Max number of gather/scatter entries per I/O (default is 12)"); + "Max number of gather/scatter entries per I/O (default is 12, max 255)"); static int topspin_workarounds = 1; @@ -2138,6 +2138,11 @@ static int __init srp_init_module(void) { int ret; + if (srp_sg_tablesize > 255) { + printk(KERN_ERR PFX "srp_sg_tablesize too large\n"); + return -EINVAL; + } + ib_srp_transport_template = srp_attach_transport(&ib_srp_transport_functions); if (!ib_srp_transport_template) From boliver at salts.navy.mil Mon Mar 17 11:54:26 2008 From: boliver at salts.navy.mil (Kim Stokes) Date: Mon, 17 Mar 2008 19:54:26 +0100 Subject: [ofa-general] Adobe Acrobat 8, Photoshop CS3, MS Office XP Message-ID: <720712983.69038357403342@salts.navy.mil> Cut-price OEM software e-shopWir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren.http://geocities.com/castanedaalvin23/* Windows XP Professional With SP2 Full Version: $59.95 * Office Enterprise 2007: $79.95 * Adobe Photoshop CS3 Extended: $79.95 http://geocities.com/castanedaalvin23/ Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From Sean at redshift.com Mon Mar 17 11:56:44 2008 From: Sean at redshift.com (Sean Evans) Date: Mon, 17 Mar 2008 21:56:44 +0300 Subject: [ofa-general] I wanted to break up a rival's relationship by having sex with his/her partner. Message-ID: <510301c88860$a502a0f0$20687a5b@Sean> Without the risk of surgery, just 2 capsules a day for visible gains within 1 month http://calburla.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Mon Mar 17 12:34:21 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 17 Mar 2008 12:34:21 -0700 Subject: [ofa-general] Re: [PATCH] IB/srp: enforce protocol limit on srp_sg_tablesize In-Reply-To: <1205777880.10010.26.camel@lap75545.ornl.gov> (David Dillow's message of "Mon, 17 Mar 2008 14:18:00 -0400") References: <1205777880.10010.26.camel@lap75545.ornl.gov> Message-ID: > The current SRP initiator will allow unlimited s/g entries in the > indirect descriptors lists, but the entry count field in the SRP_CMD > request is 8 bits, so setting srp_sg_tablesize too large will open the > possibility of wrapping the count and generating invalid requests. makes sense, but... > + if (srp_sg_tablesize > 255) { > + printk(KERN_ERR PFX "srp_sg_tablesize too large\n"); > + return -EINVAL; > + } friendlier to clamp it to 255 and print a warning? what do you think? From dillowda at ornl.gov Mon Mar 17 12:42:45 2008 From: dillowda at ornl.gov (David Dillow) Date: Mon, 17 Mar 2008 15:42:45 -0400 Subject: [ofa-general] [PATCH] IB/srp: enforce protocol limit on srp_sg_tablesize In-Reply-To: References: <1205777880.10010.26.camel@lap75545.ornl.gov> Message-ID: <1205782965.10010.30.camel@lap75545.ornl.gov> The current SRP initiator will allow unlimited s/g entries in the indirect descriptors lists, but the entry count field in the SRP_CMD request is 8 bits, so setting srp_sg_tablesize too large will open the possibility of wrapping the count and generating invalid requests. Clamp srp_sg_tablesize to the protocol limits to prevent surprises. Reported-by: Martin W. Schlining III Signed-off-by: David Dillow --- On Mon, 2008-03-17 at 12:34 -0700, Roland Dreier wrote: > > The current SRP initiator will allow unlimited s/g entries in the > > indirect descriptors lists, but the entry count field in the SRP_CMD > > request is 8 bits, so setting srp_sg_tablesize too large will open the > > possibility of wrapping the count and generating invalid requests. > > makes sense, but... > > > + if (srp_sg_tablesize > 255) { > > + printk(KERN_ERR PFX "srp_sg_tablesize too large\n"); > > + return -EINVAL; > > + } > > friendlier to clamp it to 255 and print a warning? what do you think? Sure, here's an updated patch. diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index fd4a49f..b536841 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -68,7 +68,7 @@ static int srp_max_iu_len; module_param(srp_sg_tablesize, int, 0444); MODULE_PARM_DESC(srp_sg_tablesize, - "Max number of gather/scatter entries per I/O (default is 12)"); + "Max number of gather/scatter entries per I/O (default is 12, max 255)"); static int topspin_workarounds = 1; @@ -2138,6 +2138,11 @@ static int __init srp_init_module(void) { int ret; + if (srp_sg_tablesize > 255) { + printk(KERN_WARNING PFX "clamping srp_sg_tablesize to 255\n"); + srp_sg_tablesize = 255; + } + ib_srp_transport_template = srp_attach_transport(&ib_srp_transport_functions); if (!ib_srp_transport_template) From abebffe1 at ferronal.com.mx Mon Mar 17 13:12:54 2008 From: abebffe1 at ferronal.com.mx (Vito Clifford) Date: Mon, 17 Mar 2008 21:12:54 +0100 Subject: [ofa-general] Die Software Mit Maximum Qualitat zu Minimalpreis Message-ID: <01c88873$aa4f4f00$ea87df54@abebffe1> Start using your software immediately after purchase.Wir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren.http://geocities.com/santos.irma/* Windows XP Professional With SP2 Full Version: $59.95 * Adobe Acrobat 8.0 Professional: $69.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/santos.irma/ Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From Yg_c at adultconnect.com Mon Mar 17 11:29:47 2008 From: Yg_c at adultconnect.com (cordie lan) Date: Mon, 17 Mar 2008 18:29:47 +0000 Subject: [ofa-general] it will be great to read your reply ASAP Message-ID: <000601c8886b$066025d8$ae613cb2@mbkokek> From: Zhang Zu Beijing china . Date: March/17/2008 Dear, I want to respectfully solicit your attention to receive funds on my behalf. Please send me a reply with your consent to proceed and I will send you the full details and more information about myself and the funds. I assure you that it is a legal and safe proposal. My personal email is: zhangzu at yahoo.com.sg and zhangzuzu at gmail.com Thank you Zhang Zu Reply Email: zhangzu at yahoo.com.sg or zhangzuzu at gmail.com From rdreier at cisco.com Mon Mar 17 13:44:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 17 Mar 2008 13:44:05 -0700 Subject: [ofa-general] [PATCH/RFC] mlx4_core: Fix confusion between mlx4_event and mlx4_dev_event enums Message-ID: The struct mlx4_interface.event() method was supposed to get an enum mlx4_dev_event, but the driver code was actually passing in the hardware enum mlx4_event values. Fix up the callers of mlx4_dispatch_event() so that they pass in the right type of value, and fix up the event method in mlx4_ib so that it can handle the enum mlx4_dev_event values. This eliminates the need for the subtype parameter to the event method, so remove it. This also fixes the sparse warning drivers/net/mlx4/intf.c:127:48: warning: mixing different enum types drivers/net/mlx4/intf.c:127:48: int enum mlx4_event versus drivers/net/mlx4/intf.c:127:48: int enum mlx4_dev_event Signed-off-by: Roland Dreier --- drivers/infiniband/hw/mlx4/main.c | 14 ++++++++------ drivers/net/mlx4/catas.c | 2 +- drivers/net/mlx4/eq.c | 5 ++++- drivers/net/mlx4/intf.c | 8 ++------ drivers/net/mlx4/mlx4.h | 4 ++-- include/linux/mlx4/driver.h | 3 +-- 6 files changed, 18 insertions(+), 18 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index ef5e9db..6ea4746 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -677,18 +677,20 @@ static void mlx4_ib_remove(struct mlx4_dev *dev, void *ibdev_ptr) } static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr, - enum mlx4_dev_event event, int subtype, - int port) + enum mlx4_dev_event event, int port) { struct ib_event ibev; switch (event) { - case MLX4_EVENT_TYPE_PORT_CHANGE: - ibev.event = subtype == MLX4_PORT_CHANGE_SUBTYPE_ACTIVE ? - IB_EVENT_PORT_ACTIVE : IB_EVENT_PORT_ERR; + case MLX4_DEV_EVENT_PORT_UP: + ibev.event = IB_EVENT_PORT_ACTIVE; break; - case MLX4_EVENT_TYPE_LOCAL_CATAS_ERROR: + case MLX4_DEV_EVENT_PORT_DOWN: + ibev.event = IB_EVENT_PORT_ERR; + break; + + case MLX4_DEV_EVENT_CATASTROPHIC_ERROR: ibev.event = IB_EVENT_DEVICE_FATAL; break; diff --git a/drivers/net/mlx4/catas.c b/drivers/net/mlx4/catas.c index 6b32ec9..aa95287 100644 --- a/drivers/net/mlx4/catas.c +++ b/drivers/net/mlx4/catas.c @@ -69,7 +69,7 @@ static void poll_catas(unsigned long dev_ptr) if (readl(priv->catas_err.map)) { dump_err_buf(dev); - mlx4_dispatch_event(dev, MLX4_EVENT_TYPE_LOCAL_CATAS_ERROR, 0, 0); + mlx4_dispatch_event(dev, MLX4_DEV_EVENT_CATASTROPHIC_ERROR, 0); if (internal_err_reset) { spin_lock(&catas_lock); diff --git a/drivers/net/mlx4/eq.c b/drivers/net/mlx4/eq.c index 9c36c20..e141a15 100644 --- a/drivers/net/mlx4/eq.c +++ b/drivers/net/mlx4/eq.c @@ -202,7 +202,10 @@ static int mlx4_eq_int(struct mlx4_dev *dev, struct mlx4_eq *eq) break; case MLX4_EVENT_TYPE_PORT_CHANGE: - mlx4_dispatch_event(dev, eqe->type, eqe->subtype, + mlx4_dispatch_event(dev, + eqe->subtype == MLX4_PORT_CHANGE_SUBTYPE_ACTIVE ? + MLX4_DEV_EVENT_PORT_UP : + MLX4_DEV_EVENT_PORT_DOWN, be32_to_cpu(eqe->event.port_change.port) >> 28); break; diff --git a/drivers/net/mlx4/intf.c b/drivers/net/mlx4/intf.c index be5d9e9..4a6c4d5 100644 --- a/drivers/net/mlx4/intf.c +++ b/drivers/net/mlx4/intf.c @@ -30,8 +30,6 @@ * SOFTWARE. */ -#include - #include "mlx4.h" struct mlx4_device_context { @@ -113,8 +111,7 @@ void mlx4_unregister_interface(struct mlx4_interface *intf) } EXPORT_SYMBOL_GPL(mlx4_unregister_interface); -void mlx4_dispatch_event(struct mlx4_dev *dev, enum mlx4_event type, - int subtype, int port) +void mlx4_dispatch_event(struct mlx4_dev *dev, enum mlx4_dev_event type, int port) { struct mlx4_priv *priv = mlx4_priv(dev); struct mlx4_device_context *dev_ctx; @@ -124,8 +121,7 @@ void mlx4_dispatch_event(struct mlx4_dev *dev, enum mlx4_event type, list_for_each_entry(dev_ctx, &priv->ctx_list, list) if (dev_ctx->intf->event) - dev_ctx->intf->event(dev, dev_ctx->context, type, - subtype, port); + dev_ctx->intf->event(dev, dev_ctx->context, type, port); spin_unlock_irqrestore(&priv->ctx_lock, flags); } diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 53a1cdd..7333681 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -42,6 +42,7 @@ #include #include +#include #include #define DRV_NAME "mlx4_core" @@ -313,8 +314,7 @@ void mlx4_catas_cleanup(void); int mlx4_restart_one(struct pci_dev *pdev); int mlx4_register_device(struct mlx4_dev *dev); void mlx4_unregister_device(struct mlx4_dev *dev); -void mlx4_dispatch_event(struct mlx4_dev *dev, enum mlx4_event type, - int subtype, int port); +void mlx4_dispatch_event(struct mlx4_dev *dev, enum mlx4_dev_event type, int port); struct mlx4_dev_cap; struct mlx4_init_hca_param; diff --git a/include/linux/mlx4/driver.h b/include/linux/mlx4/driver.h index 1b835ca..53c5fdb 100644 --- a/include/linux/mlx4/driver.h +++ b/include/linux/mlx4/driver.h @@ -48,8 +48,7 @@ struct mlx4_interface { void * (*add) (struct mlx4_dev *dev); void (*remove)(struct mlx4_dev *dev, void *context); void (*event) (struct mlx4_dev *dev, void *context, - enum mlx4_dev_event event, int subtype, - int port); + enum mlx4_dev_event event, int port); struct list_head list; }; -- 1.5.4.3 From rdreier at cisco.com Mon Mar 17 13:56:14 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 17 Mar 2008 13:56:14 -0700 Subject: [ofa-general] Re: [PATCH 1/3] Infiniband: make ehca_pd use struct pid pointer rather than pid_t In-Reply-To: <47DE683E.3050005@openvz.org> (Pavel Emelyanov's message of "Mon, 17 Mar 2008 15:46:54 +0300") References: <47DE683E.3050005@openvz.org> Message-ID: > The task_struct->tgid field is about to become deprecated, due to > pid namespaces make tasks have many pids, not one. The infiniband > driver is one of the code, that still uses it in some places. Looks fine in terms of the changes it makes, but actually it seems that the ehca use of this is completely bogus and the ownership checking should be removed. The core ib_uverbs module has checks that make sure that objects can only be accessed through the file that they were created through; of course there are tricky ways a file can be passed from one process to another, but I don't think we want to disallow userspace processes from trying to do interesting stuff as long as it doesn't hurt anything. In other words-- ehca shouldn't be looking at tgids or anything like that at all. If there are missing checks then they should be in the core userspace verbs stuff; but I think what we have is actually OK. ehca guys, what do you think? - R. From or.gerlitz at gmail.com Mon Mar 17 16:09:40 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 18 Mar 2008 01:09:40 +0200 Subject: [ofa-general] [PATCH 0/10] stateless offload patches In-Reply-To: <1205767416.25950.135.camel@mtls03> References: <1205767416.25950.135.camel@mtls03> Message-ID: <15ddcffd0803171609g6a23ab1esb2bdc7dbf82708be@mail.gmail.com> On 3/17/08, Eli Cohen wrote: > > They have been reviewed by Or Gerlitz To be precise, I have reviewed patches 1,7,8,9 in their for-2.6.25 form Or. 0001-IB-mthca-Add-checksum-offload-support.patch > 0007-IB-ipoib-Add-ethtool-support.patch > 0008-IB-core-Add-support-for-modify-CQ.patch > 0009-IB-ipoib-Support-modifying-IPOIB-CQ-moderation-para.patch > -------------- next part -------------- An HTML attachment was scrubbed... URL: From or.gerlitz at gmail.com Mon Mar 17 16:09:40 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 18 Mar 2008 01:09:40 +0200 Subject: [ofa-general] [PATCH 0/10] stateless offload patches In-Reply-To: <1205767416.25950.135.camel@mtls03> References: <1205767416.25950.135.camel@mtls03> Message-ID: <15ddcffd0803171609g6a23ab1esb2bdc7dbf82708be@mail.gmail.com> On 3/17/08, Eli Cohen wrote: > > They have been reviewed by Or Gerlitz To be precise, I have reviewed patches 1,7,8,9 in their for-2.6.25 form Or. 0001-IB-mthca-Add-checksum-offload-support.patch > 0007-IB-ipoib-Add-ethtool-support.patch > 0008-IB-core-Add-support-for-modify-CQ.patch > 0009-IB-ipoib-Support-modifying-IPOIB-CQ-moderation-para.patch > -------------- next part -------------- An HTML attachment was scrubbed... URL: From subbukl at gmail.com Mon Mar 17 17:06:57 2008 From: subbukl at gmail.com (subbu kl) Date: Mon, 17 Mar 2008 19:06:57 -0500 Subject: [ofa-general] ib_create_fmr_pool do INIT_HLIST_NODE only if cache is enabled In-Reply-To: References: Message-ID: I agree Roland. Only thing is it may save very few machine cycles :) ~subbu --- fmr_pool.orig.c 2008-03-05 11:29:20.000000000 -0600 +++ fmr_pool.mod.c 2008-03-17 17:05:04.000000000 -0500 @@ -321,7 +321,9 @@ fmr->pool = pool; fmr->remap_count = 0; fmr->ref_count = 0; - INIT_HLIST_NODE(&fmr->cache_node); + + if (params->cache) + INIT_HLIST_NODE(&fmr->cache_node); fmr->fmr = ib_alloc_fmr(pd, params->access, &fmr_attr); if (IS_ERR(fmr->fmr)) { On Sun, Mar 9, 2008 at 3:51 PM, Roland Dreier wrote: > > - INIT_HLIST_NODE(&fmr->cache_node); > > > + if (params->cache) { > > + INIT_HLIST_NODE(&fmr->cache_node); > > + } > > I guess we could do that but in practice it doesn't make any > difference, does it? Adding the if statement makes the code bigger > (and it's not a fast path anyway). > > (BTW the kernel style is to leave out the { } around a single line block) > > - R. > -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -------------- next part -------------- An HTML attachment was scrubbed... URL: From -dx at yahoogroups.de Mon Mar 17 18:11:51 2008 From: -dx at yahoogroups.de (Barbra Presley) Date: Tue, 18 Mar 2008 10:11:51 +0900 Subject: [ofa-general] {Viagra_onli2_de} Message-ID: <665670172.87278235074267@yahoogroups.de> Online Apotheke - original Qualitaet - 100% wirksam Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro - Kein peinlicher Arztbesuch erforderlich - keine versteckte Kosten - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - Bequem und diskret online bestellen. - Diskrete Verpackung und Zahlung - Kostenlose, arztliche Telefon-Beratung Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen Jetzt bestellen - und vier Pillen umsonst erhalten Potenzschwache? wir haben die Losung (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From Golden-01 at Mail.ru Mon Mar 17 20:22:34 2008 From: Golden-01 at Mail.ru () Date: Tue, 18 Mar 2008 06:22:34 +0300 Subject: [ofa-general] =?iso-8859-1?q?=C2=E0=F8_=E4=EE=EF=EE=EB=ED=E8=F2?= =?iso-8859-1?q?=E5=EB=FC=ED=FB=E9_=E7=E0=F0=E0=E1=EE=F2=EE=EA=2C?= =?iso-8859-1?q?=E0_=E2=EE=E7=EC=EE=E6=ED=EE_=E8_=EE=E1=E5=F1=EF=E5?= =?iso-8859-1?q?=F7=E5=ED=ED=EE=E5_=E1=F3=E4=F3=F9=E5=E5=2E?= Message-ID: <200803180622.ZTFMYWSSPQ@82.138.52.149>                                                                                Добрый день !      Извините,что,возможно ,отнимаю у Вас время,но если Вы имеете  доступ к Интернету,умеете пользоваться  мышкой,   имеете 2-3 часа свободного времени,и Вас интересуют деньги,то всего один щелчок мышкой может отделять Вас от состояния !!! Еще раз простите за вторжение.Желаю Вам удачного дня . Жду Вашего ответа : Moskvich-V at Mail.ru  или  Vladimir11682 at Yandex.ru Удачи Вам !!! Информация находиться в прикрепленном   файле  .... Не беспокойтесь, это не вирус,но если Вы ,тем не менее,  опасаетесь  его  открыть  ,не  спешите  его  удалять ,вовсе не сложно проверить почту при помощи антивируса. Это не реклама, а действительно выгодное предложение !!! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Golden Stream.doc Type: application/octet-stream Size: 160256 bytes Desc: not available URL: From bjh at bluewateryachting.com Mon Mar 17 22:24:00 2008 From: bjh at bluewateryachting.com (Imelda Sams) Date: Tue, 18 Mar 2008 14:24:00 +0900 Subject: [ofa-general] Feel and smell more sexy to women Message-ID: <01c88903$b5743c20$45877e74@bjh> Don't get left behind, get it Find more and link to our ONLINE STORE in attached details. -------------- next part -------------- A non-text attachment was scrubbed... Name: p.zip Type: application/zip Size: 270 bytes Desc: not available URL: From sfr at canb.auug.org.au Mon Mar 17 22:32:16 2008 From: sfr at canb.auug.org.au (Stephen Rothwell) Date: Tue, 18 Mar 2008 16:32:16 +1100 Subject: [ofa-general] linux-next: infiniband merge conflict Message-ID: <20080318163216.f5cb148e.sfr@canb.auug.org.au> Hi Roland, Greg, Yesterday and today, I got a merge conflict between the infiniband tree and driver-core patch ib-convert-struct-class_device-to-struct-device.patch. The resolution is pretty tirivial, but maybe a better solution is for that patch to be added to the infinband tree. The following files get conflicts: drivers/infiniband/hw/amso1100/c2_provider.c drivers/infiniband/hw/cxgb3/iwch_provider.c drivers/infiniband/hw/nes/nes_verbs.c -- Cheers, Stephen Rothwell sfr at canb.auug.org.au http://www.canb.auug.org.au/~sfr/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From trekpott at WILMINK-INT.COM Tue Mar 18 00:21:24 2008 From: trekpott at WILMINK-INT.COM (Jaimes Hustles) Date: Tue, 18 Mar 2008 16:21:24 +0900 Subject: [ofa-general] You'll never believe the difference this makes Message-ID: <000801c888c8$ac6e5dd0$b10178de@QC> Watch how women can come multiple times in one session once you have a 9 inch pounder. http://www.poginova.com/ Why Look Further For the Best Deals Online -------------- next part -------------- An HTML attachment was scrubbed... URL: From bart.vanassche at gmail.com Tue Mar 18 01:17:52 2008 From: bart.vanassche at gmail.com (Bart Van Assche) Date: Tue, 18 Mar 2008 09:17:52 +0100 Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface In-Reply-To: <1E3DCD1C63492545881FACB6063A57C102221AD0@mtiexch01.mti.com> References: <1E3DCD1C63492545881FACB6063A57C102221AD0@mtiexch01.mti.com> Message-ID: On Mon, Mar 17, 2008 at 6:07 PM, Boris Shpolyansky wrote: > Check : > > /sys/class/infiniband/[mlx4_*|mthca*]/ports/*/counters/* Thanks, these are interesting counters. Unfortunately these counters are 32-bit counters and already overflowed during the test I ran (less than one day of SRP communication): $ uname -r 2.6.24 $ uname -m x86_64 $ head /sys/class/infiniband/mthca0/ports/1/counters/*_{data,packets} ==> /sys/class/infiniband/mthca0/ports/1/counters/port_rcv_data <== 4294967295 ==> /sys/class/infiniband/mthca0/ports/1/counters/port_xmit_data <== 4294967295 ==> /sys/class/infiniband/mthca0/ports/1/counters/port_rcv_packets <== 4294967295 ==> /sys/class/infiniband/mthca0/ports/1/counters/port_xmit_packets <== 4294967295 Bart. From invite at goodtree.com Tue Mar 18 02:41:40 2008 From: invite at goodtree.com (Charina Bael via GoodTree) Date: Tue, 18 Mar 2008 02:41:40 -0700 (PDT) Subject: [ofa-general] I gave your name... Message-ID: <20080318094140.B0D7B450F82@web04.goodtree.com> http://goodtree.com/invitations/52311925?version=16&a=85367 Please accept my invitation to the website that is connecting all the good people around us. Be Good, Charina --- This email sent by charinacentury at yahoo.com through GoodTree LLC., 703 Market St. #470, San Francisco, CA 94103. No more invites, please: http://goodtree.com/opt_out?i=52311925&a=85367 -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-arnetb at bellsouth.net Tue Mar 18 04:22:34 2008 From: a-arnetb at bellsouth.net (Genevieve Cordero) Date: Tue, 18 Mar 2008 19:22:34 +0800 Subject: [ofa-general] You told me we can chat Message-ID: <01c8892d$6ae53900$c854a33c@a-arnetb> Hello! I am tired tonight. I am nice girl that would like to chat with you. Email me at Ashley at BestGolova.com only, because I am using my friend's email to write this. Hope you wanna see my pics. From hrosenstock at xsigo.com Tue Mar 18 05:49:59 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 18 Mar 2008 05:49:59 -0700 Subject: [ofa-general] Re: ibutils binaries configuration and installation questions In-Reply-To: <47DEB1CC.6090201@dev.mellanox.co.il> References: <1205350506.28072.95.camel@hrosenstock-ws.xsigo.com> <47DEB1CC.6090201@dev.mellanox.co.il> Message-ID: <1205844599.11393.117.camel@hrosenstock-ws.xsigo.com> Hi again Oren, On Mon, 2008-03-17 at 20:00 +0200, Oren Kladnitsky wrote: > Hal Rosenstock wrote: > > Hi Oren, > > > > Are the ibutils binaries meant to be run by root (like the other > > management utilities) ? If so, by default, they should be in sbin rather > > than bin. Also, is there a configure option for where the binaries go ? > > > > Thanks. > > > > -- Hal > > > Hi. > > 1. Yes - The diag apps should be run as root. The "real" restriction is > done by the umad interface permissions. Right; that's the default rule. > I can move these apps to sbin dir. as the default; Thanks. > 2. Yes - Autotools configure "--bindir" flag can be set to install the > binaries to /usr/sbin for example. Thanks. -- Hal From hrosenstock at xsigo.com Tue Mar 18 05:53:06 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 18 Mar 2008 05:53:06 -0700 Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface In-Reply-To: References: <1E3DCD1C63492545881FACB6063A57C102221AD0@mtiexch01.mti.com> Message-ID: <1205844786.11393.121.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-18 at 09:17 +0100, Bart Van Assche wrote: > On Mon, Mar 17, 2008 at 6:07 PM, Boris Shpolyansky wrote: > > Check : > > > > /sys/class/infiniband/[mlx4_*|mthca*]/ports/*/counters/* > > Thanks, these are interesting counters. Unfortunately these counters > are 32-bit counters and already overflowed during the test I ran (less > than one day of SRP communication): > > $ uname -r > 2.6.24 > $ uname -m > x86_64 > $ head /sys/class/infiniband/mthca0/ports/1/counters/*_{data,packets} > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_rcv_data <== > 4294967295 > > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_xmit_data <== > 4294967295 > > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_rcv_packets <== > 4294967295 > > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_xmit_packets <== > 4294967295 Depending on which fabric management (usually bundled with the SM) is being used, you may be able to obtain this information via the Performance Manager (and not be limited to 32 bit counters). -- Hal > Bart. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From eli at dev.mellanox.co.il Tue Mar 18 06:06:58 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 18 Mar 2008 15:06:58 +0200 Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface In-Reply-To: <1205844786.11393.121.camel@hrosenstock-ws.xsigo.com> References: <1E3DCD1C63492545881FACB6063A57C102221AD0@mtiexch01.mti.com> <1205844786.11393.121.camel@hrosenstock-ws.xsigo.com> Message-ID: <1205845618.25950.167.camel@mtls03> On Tue, 2008-03-18 at 05:53 -0700, Hal Rosenstock wrote: > > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_xmit_packets <== > > 4294967295 > > Depending on which fabric management (usually bundled with the SM) is > being used, you may be able to obtain this information via the > Performance Manager (and not be limited to 32 bit counters). > Hal, can you explain the mechanism that can be used to get counter values larger than 32 bits? I see that the IB spec defines the counters as 32 bits. From hrosenstock at xsigo.com Tue Mar 18 06:13:11 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 18 Mar 2008 06:13:11 -0700 Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface In-Reply-To: <1205845618.25950.167.camel@mtls03> References: <1E3DCD1C63492545881FACB6063A57C102221AD0@mtiexch01.mti.com> <1205844786.11393.121.camel@hrosenstock-ws.xsigo.com> <1205845618.25950.167.camel@mtls03> Message-ID: <1205845991.11393.137.camel@hrosenstock-ws.xsigo.com> Eli, On Tue, 2008-03-18 at 15:06 +0200, Eli Cohen wrote: > Hal, can you explain the mechanism that can be used to get counter > values larger than 32 bits? I see that the IB spec defines the counters > as 32 bits. Polling (at a configurable rate), clearing above some threshold, and aggregation into larger counters on the manager side. For OpenSM, see perf-manager-arch.txt or the current implementation: osm_perfmgr.c and osm_perfmgr_db.c -- Hal From hrosenstock at xsigo.com Tue Mar 18 06:17:37 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 18 Mar 2008 06:17:37 -0700 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <47D59199.1050301@mellanox.co.il> References: <1204654931.6469.423.camel@hrosenstock-ws.xsigo.com> <20080305111045.GC27256@sashak.voltaire.com> <47CEC012.5040203@mellanox.co.il> <20080305161431.GO30723@sashak.voltaire.com> <47D55BE0.6060503@mellanox.co.il> <1205165551.6469.933.camel@hrosenstock-ws.xsigo.com> <47D59199.1050301@mellanox.co.il> Message-ID: <1205846257.11393.141.camel@hrosenstock-ws.xsigo.com> Tziporet, On Mon, 2008-03-10 at 21:52 +0200, Tziporet Koren wrote: > Only ConnectX supports QoS > > Any firmware release ? The OpenSM release notes mention 0.5.0 or later. > > > I think any but will find out tomorrow Were you able to find this out ? Thanks. -- Hal From bart.vanassche at gmail.com Tue Mar 18 06:22:16 2008 From: bart.vanassche at gmail.com (Bart Van Assche) Date: Tue, 18 Mar 2008 14:22:16 +0100 Subject: [ofa-general] Socket Direct Protocol Message-ID: Hello, I am considering to use the SDP protocol for some legacy applications. So I tried to find a standards document defining SDP. I could only find a draft standard on the RDMA consortium website (last updated in 2003), not a final standards document. On the same website there is an SDP FAQ. This FAQ mentions that the SDP specification will be submitted to the IETF. Does anyone know whether this ever happened ? See also: http://www.rdmaconsortium.org/home/draft-pinkerton-iwarp-sdp-v1.0.pdf http://www.rdmaconsortium.org/home/SDP_FAQ.htm Bart. From sri1998d at orca1.com Tue Mar 18 06:33:32 2008 From: sri1998d at orca1.com (Roy Bautista) Date: Tue, 18 Mar 2008 05:33:32 -0800 Subject: [ofa-general] Rechtliche Vertrieb von Software Message-ID: <027902314.90662550650899@orca1.com> Die Software auf allen europaischen Sprachen, fur Windows und Macintosh vorherbestimmt. Die konnen Sie momentan bekommen. Nur bezahlen und auslasten. Hier prasentiert sind nicht teuere, aber echte und vollige Produkte der Software.http://geocities.com/rodriquezmolly/Die professionelle Konsultation des Anwenderdienstes hilft Ihnen jedes Programm leicht aufstellen. Schnelle Antwort ist garantiert. Die Ruckzahlung ist moglich. Sie kaufen nur die vollkommen funktionierende Software -------------- next part -------------- An HTML attachment was scrubbed... URL: From municipally4 at ibsinc-usa.com Tue Mar 18 06:49:41 2008 From: municipally4 at ibsinc-usa.com (Hans Holden) Date: Tue, 18 Mar 2008 14:49:41 +0100 Subject: [ofa-general] buy now Viagra 100mg x 90 pills US $ 159.95 Message-ID: <01c88907$4bd3b080$4e04114f@municipally4> Viagra (Sildenafil) 100mg x 30 pills $99.95 price http://ritamcmonigleut525.blogspot.com From tscott at winningtime.com Tue Mar 18 06:53:59 2008 From: tscott at winningtime.com (Lester Leary) Date: Tue, 18 Mar 2008 16:53:59 +0300 Subject: [ofa-general] Nie mehr zu frueh kommen? Message-ID: <01c88918$a9242d80$9b46cc59@tscott> Online anonym bestellen - original Qualitat - 100% wirksam Fakten von unseren Kunden: - Sex ist befriedigender denn je. Stress und Leistungsdruck verschwinden. Sie ist nie wieder frustriert, ich habe keine Angst mehr zu versagen. Es ist ein wundervolles korperliches Erlebnis, dem ein genauso tiefes Gefuhl folgt. - Die Nebenwirkungen sind minimal: manchmal eine verstopfte Nase, kurzzeitig ein roter Kopf - kein Kopfschmerz, sondern das Gefuhl, als wurde man eine Flasche eiskalte Cola in einem Zug trinken. - Interessanterweise macht eine Vi. allein noch keinen Stander. Man(n) muss wenigstens ein bisschen Lust auf Sex mit der Frau haben. Gegen eine Eiserne Jungfrau im Bett hilft auch die grosste Dosis nichts. Wer aber das erste Kribbeln in den Lenden spurt, wird einen stahlharten Stander haben, und das fur wenigstens vier Stunden. - Eine volle 100-mg-Dosis macht den Schwanz zum Schwert. Wer es ubertreibt, ist Schuld, wenn die Herzallerliebste am Ende einen Y-formigen Sarg braucht. Fur die meisten von uns sind 50 mg mehr als genug, wenn man das gute Stuck zwischen den Hohepunkten auch mal hangen lassen will ... zur Not hilft es da vielleicht, sich ein nacktes Grossmutterchen vorzustellen. - Wer noch Zeit und Lust fur eine schnelle Nummer am nachsten Morgen hat, sollte dafur sorgen, dann noch genug Viagra im Blut zu haben - damit es noch fur ein oder zwei "Stehaufmannchen" reicht. - Das Beste an Vi. ist die Sicherheit, dass man "mit Autopilot fliegt", dass man entspannt und ohne Sorgen zur Sache kommen kann, dass der Stander auch halt, auch wenn man unterbrochen wird (die Kinder klopfen an die Schlafzimmertur, der Hund bellt, das Kondom sitzt schlecht). Wenn man Vi. bewusst anwendet, kann es auch der Partnerin gegenuber ein grosses Geschenk sein. Nur ein Rat: Sagen Sie ihr nicht, dass Sie es verwenden, das weibliche Selbstwertgefuhl ist genauso verletzlich wie das unsere. Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro {$konditions_de} {$konditions_de} {$konditions_de} {$konditions_de} {$konditions_de} {$konditions_de} {$konditions_de} Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen Klicken Sie HIER und Sie erhalten vier Dosen umsonst - Kein peinlicher Arztbesuch erforderlich -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Tue Mar 18 06:54:02 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 18 Mar 2008 15:54:02 +0200 Subject: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support In-Reply-To: <1205846257.11393.141.camel@hrosenstock-ws.xsigo.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C90390C507@mtlexch01.mtl.com> Hi Hal, >> Any firmware release ? The OpenSM release >> notes mention 0.5.0 or later. HCAs: QoS supported by ConnectX. The latest FW release doesn't support QoS. QoS-enabled FW release (2_5_000) is planned for May. If someone wishes to get QoS-enabled FW before the official release, he should contact Mellanox FAE. Switches: QoS supported by InfiniScale III Any InfiniScale III FW that is supported by OpenSM supports QoS. Regards, Yevgeny Kliteynik Mellanox Technologies LTD Tel: +972-4-909-7200 ext: 394 Fax: +972-4-959-3245 P.O. Box 586 Yokneam 20692 ISRAEL -----Original Message----- From: Hal Rosenstock [mailto:hrosenstock at xsigo.com] Sent: Tuesday, March 18, 2008 15:18 To: tziporet at dev.mellanox.co.il Cc: Yevgeny Kliteynik; general at lists.openfabrics.org Subject: Re: [ofa-general] Re: OFED 1.3 OpenSM and QoS Annex Support Tziporet, On Mon, 2008-03-10 at 21:52 +0200, Tziporet Koren wrote: > Only ConnectX supports QoS > > Any firmware release ? The OpenSM release notes mention 0.5.0 or later. > > > I think any but will find out tomorrow Were you able to find this out ? Thanks. -- Hal From hrosenstock at xsigo.com Tue Mar 18 07:19:24 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 18 Mar 2008 07:19:24 -0700 Subject: [ofa-general] [PATCH] OpenSM release notes: Clarify QoS firmware support Message-ID: <1205849964.11393.152.camel@hrosenstock-ws.xsigo.com> OpenSM release notes: Clarify Mellanox firmware support requirements for QoS Also, some other cosmetic changes Please apply to master and OFED 1.3. Signed-off-by: Hal Rosenstock diff --git a/opensm/doc/opensm_release_notes-3.1.10.txt b/opensm/doc/opensm_release_notes-3.1.10.txt index 825d758..34d19d7 100644 --- a/opensm/doc/opensm_release_notes-3.1.10.txt +++ b/opensm/doc/opensm_release_notes-3.1.10.txt @@ -35,7 +35,7 @@ This document includes the following sections: When enabled it collects a fabric port counters and able to log it or to pass to external program via event plugin interface. It handles counters overflow, supports LID/QP redirection and is able to work - when OpenSM is in master. standby and inactive states. + when OpenSM is in master, standby, and inactive states. * Dimension Order routing (DOR) algorithm DOR Unicast routing algorithm - based on the Min Hop algorithm, but @@ -61,7 +61,7 @@ This document includes the following sections: * PKey index support Proper support for PKey index in GSI queries. -* Incremental LFTs. PKey, SL2VL, VLarbitration table update +* Incremental LFTs, PKey, SL2VL, and VLarbitration table updates OpenSM will only fetch those tables in first heavy sweep and then will maintain this internally. @@ -70,7 +70,7 @@ This document includes the following sections: doesn't find this device as disconnected OpenSM will detect this by changed port states and handle accordingly. -* Duplicated GUIDs/port moving detector. +* Duplicated GUIDs/port moving detector OpenSM will be able to detect port moving during a fabric discovery and will not report duplicated GUIDs in this case. @@ -475,5 +475,16 @@ iPath | QHT6040 (PathScale InfiniPath HT-460) iPath | QHT6140 (PathScale InfiniPath HT-465) iPath | QLE6140 (PathScale InfiniPath PE-880) -Note: OpenSM does not run on an IBM Galaxy (eHCA) as it does not expose +Note 1: OpenSM does not run on an IBM Galaxy (eHCA) as it does not expose QP0 and QP1. However, it does support it as a device on the subnet. + +Note 2: QoS firmware and Mellanox devices + +HCAs: QoS supported by ConnectX. The current FW release +doesn't support QoS. QoS-enabled FW release (2_5_000) is +planned for May. If someone wishes to get QoS-enabled FW +before the official release, they should contact Mellanox FAE. + +Switches: QoS supported by InfiniScale III +Any InfiniScale III FW that is supported by OpenSM supports QoS. + From rob_farquharson at bmjlaw.com Tue Mar 18 08:52:01 2008 From: rob_farquharson at bmjlaw.com (Josie Jarrett) Date: Tue, 18 Mar 2008 21:22:01 +0530 Subject: [ofa-general] Die Software, von der Ihr Computer Traumt Message-ID: <01c8893e$1ac2a280$86c25bdb@rob_farquharson> Reliable software onlyWir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren.http://geocities.com/cameronbeulah/* Adobe Acrobat 8.0 Professional: $69.95 * Office Enterprise 2007: $79.95 * Office System Professional 2003 (5 Cds): $59.95 http://geocities.com/cameronbeulah/ Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From efany at sourlemon.com Tue Mar 18 08:52:16 2008 From: efany at sourlemon.com (Cecile Ferrell) Date: Tue, 18 Mar 2008 15:52:16 +0000 Subject: [ofa-general] Energy fur ihren Schwanz, kaufen und 85% sparen Message-ID: <01c88910$09fb3800$a799027a@efany> Online anonym bestellen - original Qualitat - 100% wirksam Fakten von unseren Kunden: - Sex ist befriedigender denn je. Stress und Leistungsdruck verschwinden. Sie ist nie wieder frustriert, ich habe keine Angst mehr zu versagen. Es ist ein wundervolles korperliches Erlebnis, dem ein genauso tiefes Gefuhl folgt. - Die Nebenwirkungen sind minimal: manchmal eine verstopfte Nase, kurzzeitig ein roter Kopf - kein Kopfschmerz, sondern das Gefuhl, als wurde man eine Flasche eiskalte Cola in einem Zug trinken. - Interessanterweise macht eine Vi. allein noch keinen Stander. Man(n) muss wenigstens ein bisschen Lust auf Sex mit der Frau haben. Gegen eine Eiserne Jungfrau im Bett hilft auch die grosste Dosis nichts. Wer aber das erste Kribbeln in den Lenden spurt, wird einen stahlharten Stander haben, und das fur wenigstens vier Stunden. - Eine volle 100-mg-Dosis macht den Schwanz zum Schwert. Wer es ubertreibt, ist Schuld, wenn die Herzallerliebste am Ende einen Y-formigen Sarg braucht. Fur die meisten von uns sind 50 mg mehr als genug, wenn man das gute Stuck zwischen den Hohepunkten auch mal hangen lassen will ... zur Not hilft es da vielleicht, sich ein nacktes Grossmutterchen vorzustellen. - Wer noch Zeit und Lust fur eine schnelle Nummer am nachsten Morgen hat, sollte dafur sorgen, dann noch genug Viagra im Blut zu haben - damit es noch fur ein oder zwei "Stehaufmannchen" reicht. - Das Beste an Vi. ist die Sicherheit, dass man "mit Autopilot fliegt", dass man entspannt und ohne Sorgen zur Sache kommen kann, dass der Stander auch halt, auch wenn man unterbrochen wird (die Kinder klopfen an die Schlafzimmertur, der Hund bellt, das Kondom sitzt schlecht). Wenn man Vi. bewusst anwendet, kann es auch der Partnerin gegenuber ein grosses Geschenk sein. Nur ein Rat: Sagen Sie ihr nicht, dass Sie es verwenden, das weibliche Selbstwertgefuhl ist genauso verletzlich wie das unsere. Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro {$konditions_de} {$konditions_de} {$konditions_de} {$konditions_de} {$konditions_de} {$konditions_de} {$konditions_de} Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen Jetzt bestellen - und vier Pillen umsonst erhalten = 1konditions_de -------------- next part -------------- An HTML attachment was scrubbed... URL: From HNGUYEN at de.ibm.com Tue Mar 18 08:52:27 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Tue, 18 Mar 2008 16:52:27 +0100 Subject: [ofa-general] Re: [PATCH 1/3] Infiniband: make ehca_pd use struct pid pointer rather than pid_t In-Reply-To: Message-ID: Hi Roland! > > The task_struct->tgid field is about to become deprecated, due to > > pid namespaces make tasks have many pids, not one. The infiniband > > driver is one of the code, that still uses it in some places. > > Looks fine in terms of the changes it makes, but actually it seems > that the ehca use of this is completely bogus and the ownership > checking should be removed. > > The core ib_uverbs module has checks that make sure that objects can > only be accessed through the file that they were created through; of > course there are tricky ways a file can be passed from one process to > another, but I don't think we want to disallow userspace processes > from trying to do interesting stuff as long as it doesn't hurt anything. > > In other words-- ehca shouldn't be looking at tgids or anything like > that at all. If there are missing checks then they should be in the > core userspace verbs stuff; but I think what we have is actually OK. > > ehca guys, what do you think? Reason for above checking is to prevent a child process releasing a resource that the parent process has created and still wants to use. Do you think that's something we can generalize into ib_core? Regards Nam From romanyu at iliketohurtpeople.com Tue Mar 18 08:55:06 2008 From: romanyu at iliketohurtpeople.com (Chester Fountain) Date: Tue, 18 Mar 2008 16:55:06 +0100 Subject: [ofa-general] Publisher 2007 Message-ID: <01c88918$d1139100$e2bd494d@romanyu> Microsoft Office Enterprise 2007 includes: • Access 2007 • Communicator 2007 • Excel 2007 • Groove 2007 • InfoPath 2007 • OneNote 2007 • Outlook 2007 • PowerPoint 2007 • Publisher 2007 • Word 2007 http://marianawinklemanxh742.blogspot.com System Requirements • Intel® Pentium® or AMD® 500 MHz processor • Microsoft Windows® XP Professional or Home Edition with Service Pack 2, Windows Server® 2003 with SP1 , Microsoft Windows Vista. • 256 Mb of RAM • 2GB of available hard-disk space. • 1024x768 or higher resolution monitor • Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From chas at cmf.nrl.navy.mil Tue Mar 18 08:58:44 2008 From: chas at cmf.nrl.navy.mil (chas williams - CONTRACTOR) Date: Tue, 18 Mar 2008 11:58:44 -0400 Subject: [ofa-general] ib_create_fmr_pool do INIT_HLIST_NODE only if cache is enabled In-Reply-To: Message-ID: <200803181558.m2IFwiVQ012497@cmf.nrl.navy.mil> perhaps not. conditional branches are slow. In message ,"subbu kl" writes: >Only thing is it may save very few machine cycles :) > >~subbu > >--- fmr_pool.orig.c 2008-03-05 11:29:20.000000000 -0600 >+++ fmr_pool.mod.c 2008-03-17 17:05:04.000000000 -0500 >@@ -321,7 +321,9 @@ > fmr->pool = pool; > fmr->remap_count = 0; > fmr->ref_count = 0; >- INIT_HLIST_NODE(&fmr->cache_node); >+ >+ if (params->cache) >+ INIT_HLIST_NODE(&fmr->cache_node); > > fmr->fmr = ib_alloc_fmr(pd, params->access, &fmr_attr); > if (IS_ERR(fmr->fmr)) { From rdreier at cisco.com Tue Mar 18 09:10:55 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 09:10:55 -0700 Subject: [ofa-general] Re: [PATCH 1/3] Infiniband: make ehca_pd use struct pid pointer rather than pid_t In-Reply-To: (Hoang-Nam Nguyen's message of "Tue, 18 Mar 2008 16:52:27 +0100") References: Message-ID: > Reason for above checking is to prevent a child process releasing > a resource that the parent process has created and still wants to use. > Do you think that's something we can generalize into ib_core? Clearly if we want that check then it should be in the core uverbs module. But I'm not sure why we would want that check -- I don't see any realistic scenario where that would cause problems, and it seems at least as likely that the check would break an app that intentionally does something clever. - R. From weiny2 at llnl.gov Tue Mar 18 09:26:03 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 18 Mar 2008 09:26:03 -0700 Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface In-Reply-To: <1205845991.11393.137.camel@hrosenstock-ws.xsigo.com> References: <1E3DCD1C63492545881FACB6063A57C102221AD0@mtiexch01.mti.com> <1205844786.11393.121.camel@hrosenstock-ws.xsigo.com> <1205845618.25950.167.camel@mtls03> <1205845991.11393.137.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080318092603.3be2fd28.weiny2@llnl.gov> On Tue, 18 Mar 2008 06:13:11 -0700 Hal Rosenstock wrote: > Eli, > > On Tue, 2008-03-18 at 15:06 +0200, Eli Cohen wrote: > > Hal, can you explain the mechanism that can be used to get counter > > values larger than 32 bits? I see that the IB spec defines the counters > > as 32 bits. > > Polling (at a configurable rate), clearing above some threshold, and > aggregation into larger counters on the manager side. For OpenSM, see > perf-manager-arch.txt or the current implementation: osm_perfmgr.c and > osm_perfmgr_db.c > ./configure --help ... --enable-perf-mgr Enable the performance manager (default no) ... ./opensm --help ... --perfmgr Start with PerfMgr enabled. --perfmgr_sweep_time_s PerfMgr sweep interval in seconds. ... Run opensm with the console for more control of the perfmgr and to dump the counters. OpenSM $ help perfmgr perfmgr [enable|disable|clear_counters|dump_counters|sweep_time[seconds]] perfmgr -- print the performance manager state [enable|disable] -- change the perfmgr state [sweep_time] -- change the perfmgr sweep time (requires [seconds] option) [clear_counters] -- clear the counters stored [dump_counters [mach]] -- dump the counters (optionally in [mach]ine readable format) [print_counters ] -- print the counters for the specified node OpenSM $ perfmgr Performance Manager status: state : Enabled sweep state : Sleeping sweep time : 180s outstanding queries/max : 0/500 loaded event plugin : opensmskummeeplugin Dump counters dumps to the file configured in opensm.opts. ... # # Event DB Options # # Dump file to dump the events to event_db_dump_file /var/log/opensm_port_counters.log ... There is no guarantee against losing counts, however, if you set the sweep time fast enough it should be pretty accurate. Hope this helps, Ira From sean.hefty at intel.com Tue Mar 18 09:37:51 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 18 Mar 2008 09:37:51 -0700 Subject: [ofa-general] Socket Direct Protocol In-Reply-To: References: Message-ID: <000501c88916$6853d730$b137170a@amr.corp.intel.com> >I am considering to use the SDP protocol for some legacy applications. >So I tried to find a standards document defining SDP. For InfiniBand, SDP is defined in Annex A4 of the InfiniBand Architecture Spec Vol 1. - Sean From eli at dev.mellanox.co.il Tue Mar 18 09:40:18 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 18 Mar 2008 18:40:18 +0200 Subject: [ofa-general] IPoIB enhancements Message-ID: <1205858418.25950.184.camel@mtls03> Hi Roland, I have two patches that do (1) use two CQs for IPoIB - one for the send side and one for the receive side and (2) use unsignaled QP. Both patches apply for UD mode. In addition they never arm the CQ used for send packets - instead they do polling after sending a packet. the patches are already in used in OFED 1.3 and they provide significant improvement of message rate when sending small UDP packets. For example, on may machine throughput increased from ~300 Mbps to ~500 Mbps for 128 bytes UDP packets. I have the patches ready for ofed and I can generate them for your for-2.6.26 branch. Would you like me to send them for review in the next couple of days or do you prefer me to send them a bit later? From eli at dev.mellanox.co.il Tue Mar 18 09:40:18 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 18 Mar 2008 18:40:18 +0200 Subject: [ofa-general] IPoIB enhancements Message-ID: <1205858418.25950.184.camel@mtls03> Hi Roland, I have two patches that do (1) use two CQs for IPoIB - one for the send side and one for the receive side and (2) use unsignaled QP. Both patches apply for UD mode. In addition they never arm the CQ used for send packets - instead they do polling after sending a packet. the patches are already in used in OFED 1.3 and they provide significant improvement of message rate when sending small UDP packets. For example, on may machine throughput increased from ~300 Mbps to ~500 Mbps for 128 bytes UDP packets. I have the patches ready for ofed and I can generate them for your for-2.6.26 branch. Would you like me to send them for review in the next couple of days or do you prefer me to send them a bit later? From HNGUYEN at de.ibm.com Tue Mar 18 10:09:22 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Tue, 18 Mar 2008 18:09:22 +0100 Subject: [ofa-general] Re: [PATCH 1/3] Infiniband: make ehca_pd use struct pid pointer rather than pid_t In-Reply-To: Message-ID: > > Reason for above checking is to prevent a child process releasing > > a resource that the parent process has created and still wants to use. > > Do you think that's something we can generalize into ib_core? > Clearly if we want that check then it should be in the core uverbs > module. But I'm not sure why we would want that check -- I don't see > any realistic scenario where that would cause problems, and it seems > at least as likely that the check would break an app that > intentionally does something clever. Right, above checking is based on a very simple policy "creator of a resource is also the owner in term of releasing it" and will not cover other customized patterns. We had a case - believe on sles9, in which a child process manipulated/released resources from parent one, and it was not easy to find the bug. Wrt/ your actual question: we can remove the tgid stuff from ehca kernel code. When do you expect me to send a patch at latest? Nam From losborne at hf-law.com Tue Mar 18 10:14:49 2008 From: losborne at hf-law.com (Julio Tidwell) Date: Tue, 18 Mar 2008 18:14:49 +0100 Subject: [ofa-general] Gro?er Softwareverkauf! Niedrigste Preise! Message-ID: <210270424.46213463597154@hf-law.com> Industry standard software for less than cheapSie konnen unsere Software sofort runterladen. Zahlen Sie und die Downloads werden Ihnen sofort zur Verfugung stehen. Unsere Programme wurden fuer mehrere OS (Windows und Macintosh) programmiert und sind in allen europaeischen Sprachen verfuegbar. Wir verkaufen nur originale Vollversionen, haben aber die guenstigsten Preise. http://geocities.com/jackiebarry79/* Windows XP Professional With SP2 Full Version: $59.95 * Office Enterprise 2007: $79.95 * Adobe Creative Suite 3 Design Premium: $229.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/jackiebarry79/Unsere Kundenberater sind immer bereit Ihnen bei der Installation zu helfen. Wir antworten sehr schnell und geben Ihnen auch Geld-Zurueck-Garantie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ksk123ksk at yahoo.co.in Tue Mar 18 10:15:10 2008 From: ksk123ksk at yahoo.co.in (Kristoffe Irwin) Date: Tue, 18 Mar 2008 12:15:10 -0500 Subject: [ofa-general] Software fuer die guenstigste Preise Message-ID: <608506638.89703077076026@yahoo.co.in> Start using your software immediately after purchase.Sie bekommen unsere Software ohne Wartezeiten. Sofort nach der Bezahlung koennen Sie diese runterladen. Unsere Programme wurden nicht nur fuer Windows, sondern auch fuer Macintosh entwickelt und sind in allen europaeischen Sprachen verfuegbar. Wir haben sehr gunstige Preise und es handelt sich dabei nur um vollstandige Originalversionen. http://geocities.com/hudsonbrenton/* Office Enterprise 2007: $79.95 * Windows XP Professional With SP2 Full Version: $59.95 * AutoCAD 2008: $129.95 * Office System Professional 2003 (5 Cds): $59.95 http://geocities.com/hudsonbrenton/Bestellen Sie bei uns ohne Sorgen. Wir haben kompetente Supportmitarbeiter, die Ihnen bei der Installation weiterhelfen, wenn Sie Hilfe brauchen. Schnell und unverzueglich werden von uns Ihre Fragen beantwortet. Bei uns gibt es natuerlich auch Geld-Zurueck-Garantie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From schizophreniaab30 at infra-redonline.com Tue Mar 18 10:36:52 2008 From: schizophreniaab30 at infra-redonline.com (Christa Huffman) Date: Tue, 18 Mar 2008 12:36:52 -0500 Subject: [ofa-general] US $ 119.95 Price for Viagra 50mg x 60 pills Message-ID: <01c888f4$bdeeca00$8153e6c9@schizophreniaab30> Viagra (Sildenafil) 100mg x 10 pills US $ 7.00 Per Pill price http://tamikalantert987.blogspot.com From tequila99 at zeelandnet.nl Tue Mar 18 10:44:44 2008 From: tequila99 at zeelandnet.nl (Dominick Fisher) Date: Tue, 18 Mar 2008 18:44:44 +0100 Subject: [ofa-general] Re: Message-ID: <01c88928$224a2200$4838b05a@tequila99> Satisfy you girlfriend anytime she wants! Buy and take Viagra and Cialis medications. High-quality meds with huge discounts. http://motujertook.com Perfect service, instant delivery, friendly support! -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Tue Mar 18 10:58:13 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 18 Mar 2008 10:58:13 -0700 Subject: [ofa-general] Obtaining RDMA statistics for an InfiniBand interface In-Reply-To: <20080318092603.3be2fd28.weiny2@llnl.gov> References: <1E3DCD1C63492545881FACB6063A57C102221AD0@mtiexch01.mti.com> <1205844786.11393.121.camel@hrosenstock-ws.xsigo.com> <1205845618.25950.167.camel@mtls03> <1205845991.11393.137.camel@hrosenstock-ws.xsigo.com> <20080318092603.3be2fd28.weiny2@llnl.gov> Message-ID: <1205863093.11393.213.camel@hrosenstock-ws.xsigo.com> On Tue, 2008-03-18 at 09:26 -0700, Ira Weiny wrote: > There is no guarantee against losing counts, however, if you set the sweep time > fast enough it should be pretty accurate. Regardless of how fast the polling is, it can never be 100% accurate as there is no atomic read and reset so there is always that window of change which is lost in the case of rapidly changing counters. It can be worse than this if the counters reach their sticky maximums and more counting is lost. -- Hal > Hope this helps, > Ira From eports at usga.org Tue Mar 18 11:03:41 2008 From: eports at usga.org (Lon Bean) Date: Tue, 18 Mar 2008 19:03:41 +0100 Subject: [ofa-general] Leich zu installierende Software Message-ID: <249978933.72128525666176@usga.org> An HTML attachment was scrubbed... URL: From rdreier at cisco.com Tue Mar 18 11:27:00 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 11:27:00 -0700 Subject: [ofa-general] Re: [PATCH 1/3] Infiniband: make ehca_pd use struct pid pointer rather than pid_t In-Reply-To: (Hoang-Nam Nguyen's message of "Tue, 18 Mar 2008 18:09:22 +0100") References: Message-ID: > Right, above checking is based on a very simple policy "creator of a > resource is also the owner in term of releasing it" and will not cover > other customized patterns. We had a case - believe on sles9, in which > a child process manipulated/released resources from parent one, and > it was not easy to find the bug. Hmm, I can see how that might be ugly. On the other hand it doesn't seem like the unix way for a process not to be able to do something with a file if it has a valid fd. > Wrt/ your actual question: we can remove the tgid stuff from ehca kernel > code. When do you expect me to send a patch at latest? I don't think it's super-urgent. If you can't do it, say, by the end of this week, then I'll apply Pavel's patch so we don't block his progress on namespace stuff. But I would still like to get a patch to move this out of ehca at some point, so please don't drop it. If you guys think there is value in having the checks, then please send a patch to add the ownership stuff to the uverbs core and we can argue about it then. Thanks, Roland From rdreier at cisco.com Tue Mar 18 11:29:44 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 11:29:44 -0700 Subject: [ofa-general] linux-next: infiniband merge conflict In-Reply-To: <20080318163216.f5cb148e.sfr@canb.auug.org.au> (Stephen Rothwell's message of "Tue, 18 Mar 2008 16:32:16 +1100") References: <20080318163216.f5cb148e.sfr@canb.auug.org.au> Message-ID: > Yesterday and today, I got a merge conflict between the infiniband tree > and driver-core patch > ib-convert-struct-class_device-to-struct-device.patch. The resolution is > pretty tirivial, but maybe a better solution is for that patch to be > added to the infinband tree. I assume it's the __FUNCTION__ -> __func__ conversion causing the clash? Greg, I'm fine with merging that patch through my tree. The only potential problem I see is if you also have the final class_device removal patch in your tree too, in which case your tree then depends on my tree in an unfortunate way. Anyway, I'm flexible, let me know what works best for you. - R. From rdreier at cisco.com Tue Mar 18 11:30:00 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 11:30:00 -0700 Subject: [ofa-general] Re: IPoIB enhancements In-Reply-To: <1205858418.25950.184.camel@mtls03> (Eli Cohen's message of "Tue, 18 Mar 2008 18:40:18 +0200") References: <1205858418.25950.184.camel@mtls03> Message-ID: > I have two patches that do (1) use two CQs for IPoIB - one for the send > side and one for the receive side and (2) use unsignaled QP. Both > patches apply for UD mode. In addition they never arm the CQ used for > send packets - instead they do polling after sending a packet. the > patches are already in used in OFED 1.3 and they provide significant > improvement of message rate when sending small UDP packets. For example, > on may machine throughput increased from ~300 Mbps to ~500 Mbps for 128 > bytes UDP packets. I have the patches ready for ofed and I can generate > them for your for-2.6.26 branch. Would you like me to send them for > review in the next couple of days or do you prefer me to send them a bit > later? Please send them whenever they are ready. From rdreier at cisco.com Tue Mar 18 11:30:00 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 11:30:00 -0700 Subject: [ofa-general] Re: IPoIB enhancements In-Reply-To: <1205858418.25950.184.camel@mtls03> (Eli Cohen's message of "Tue, 18 Mar 2008 18:40:18 +0200") References: <1205858418.25950.184.camel@mtls03> Message-ID: > I have two patches that do (1) use two CQs for IPoIB - one for the send > side and one for the receive side and (2) use unsignaled QP. Both > patches apply for UD mode. In addition they never arm the CQ used for > send packets - instead they do polling after sending a packet. the > patches are already in used in OFED 1.3 and they provide significant > improvement of message rate when sending small UDP packets. For example, > on may machine throughput increased from ~300 Mbps to ~500 Mbps for 128 > bytes UDP packets. I have the patches ready for ofed and I can generate > them for your for-2.6.26 branch. Would you like me to send them for > review in the next couple of days or do you prefer me to send them a bit > later? Please send them whenever they are ready. From rdreier at cisco.com Tue Mar 18 11:36:27 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 11:36:27 -0700 Subject: [ofa-general] Re: [PATCH 1/10] IB/mthca: Add checksum offload support In-Reply-To: <1205767422.25950.136.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:42 +0200") References: <1205767422.25950.136.camel@mtls03> Message-ID: thanks, applied From rdreier at cisco.com Tue Mar 18 11:36:27 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 11:36:27 -0700 Subject: [ofa-general] Re: [PATCH 1/10] IB/mthca: Add checksum offload support In-Reply-To: <1205767422.25950.136.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:42 +0200") References: <1205767422.25950.136.camel@mtls03> Message-ID: thanks, applied From hrosenstock at xsigo.com Tue Mar 18 11:54:18 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 18 Mar 2008 11:54:18 -0700 Subject: [ofa-general] mthca IRQ sharing Message-ID: <1205866458.11393.244.camel@hrosenstock-ws.xsigo.com> Hi, Can mthca share an IRQ with some other device (not operating in MSI or MSI-X mode if mthca is configured this way) ? Are there any known issues in doing this ? I seem to vaguely recall some discussion of this quite a long time ago but couldn't dig out the email threads on this. Thanks in advance for any insights on this. -- Hal From interminglesc8 at scottegreen.com Tue Mar 18 11:55:20 2008 From: interminglesc8 at scottegreen.com (Dave Joyce) Date: Tue, 18 Mar 2008 20:55:20 +0200 Subject: [ofa-general] Your erection can be much harder. Message-ID: <01c8893a$607d6c00$42b6ea58@interminglesc8> And if you one day you see that your penis size is satisfied... http://ogfgcburss21.blogspot.com From rdreier at cisco.com Tue Mar 18 11:58:47 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 11:58:47 -0700 Subject: [ofa-general] mthca IRQ sharing In-Reply-To: <1205866458.11393.244.camel@hrosenstock-ws.xsigo.com> (Hal Rosenstock's message of "Tue, 18 Mar 2008 11:54:18 -0700") References: <1205866458.11393.244.camel@hrosenstock-ws.xsigo.com> Message-ID: > Can mthca share an IRQ with some other device (not operating in MSI or > MSI-X mode if mthca is configured this way) ? Are there any known issues > in doing this ? I seem to vaguely recall some discussion of this quite a > long time ago but couldn't dig out the email threads on this. Thanks in > advance for any insights on this. As far as I know there should be no problem if an mthca device is sharing an interrupt line with another device in INTx mode. - R. From tegoo at qq.com Tue Mar 18 12:22:35 2008 From: tegoo at qq.com (Renee Sheppard) Date: Tue, 18 Mar 2008 19:22:35 +0000 Subject: [ofa-general] Re: Viagra Message-ID: <01c8892d$6b7dcf80$ae89654d@tegoo> Forget about sexual problems now! Viagra and Cialis pills are what you need. Simple way to enhance your sexual life. Forget about problems with Canadian pharmacy! http://galowefol.com We are verified by Verisign and VISA From rdreier at cisco.com Tue Mar 18 13:27:58 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 13:27:58 -0700 Subject: [ofa-general] [PATCH/RFC] IB/ehca: Make symbols used only in a single source file static Message-ID: Allow the compiler to optimize better and generate smaller code: add/remove: 0/6 grow/shrink: 2/0 up/down: 1528/-1864 (-336) function old new delta .ehca_set_pagebuf 1344 2172 +828 .ehca_probe 2312 3012 +700 ehca_set_pagebuf_phys 24 - -24 ehca_set_pagebuf_fmr 24 - -24 ehca_init_device 24 - -24 .ehca_set_pagebuf_fmr 480 - -480 .ehca_set_pagebuf_phys 512 - -512 .ehca_init_device 800 - -800 Also this fixes warnings like: drivers/infiniband/hw/ehca/ehca_mrmw.c:2015:5: warning: symbol 'ehca_set_pagebuf_fmr' was not declared. Should it be static? Signed-off-by: Roland Dreier --- Will merge for 2.6.26 unless someone objects... drivers/infiniband/hw/ehca/ehca_hca.c | 2 +- drivers/infiniband/hw/ehca/ehca_main.c | 19 ++++++++++--------- drivers/infiniband/hw/ehca/ehca_mrmw.c | 10 ++++------ 3 files changed, 15 insertions(+), 16 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c index 5bd7b59..8832123 100644 --- a/drivers/infiniband/hw/ehca/ehca_hca.c +++ b/drivers/infiniband/hw/ehca/ehca_hca.c @@ -314,7 +314,7 @@ query_gid1: return ret; } -const u32 allowed_port_caps = ( +static const u32 allowed_port_caps = ( IB_PORT_SM | IB_PORT_LED_INFO_SUP | IB_PORT_CM_SUP | IB_PORT_SNMP_TUNNEL_SUP | IB_PORT_DEVICE_MGMT_SUP | IB_PORT_VENDOR_CLASS_SUP); diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c index a86ebcc..65b3362 100644 --- a/drivers/infiniband/hw/ehca/ehca_main.c +++ b/drivers/infiniband/hw/ehca/ehca_main.c @@ -57,16 +57,17 @@ MODULE_AUTHOR("Christoph Raisch "); MODULE_DESCRIPTION("IBM eServer HCA InfiniBand Device Driver"); MODULE_VERSION(HCAD_VERSION); -int ehca_open_aqp1 = 0; +static int ehca_open_aqp1 = 0; +static int ehca_hw_level = 0; +static int ehca_poll_all_eqs = 1; +static int ehca_mr_largepage = 1; + int ehca_debug_level = 0; -int ehca_hw_level = 0; int ehca_nr_ports = 2; int ehca_use_hp_mr = 0; int ehca_port_act_time = 30; -int ehca_poll_all_eqs = 1; int ehca_static_rate = -1; int ehca_scaling_code = 0; -int ehca_mr_largepage = 1; int ehca_lock_hcalls = -1; module_param_named(open_aqp1, ehca_open_aqp1, int, S_IRUGO); @@ -396,7 +397,7 @@ init_node_guid1: return ret; } -int ehca_init_device(struct ehca_shca *shca) +static int ehca_init_device(struct ehca_shca *shca) { int ret; @@ -579,8 +580,8 @@ static ssize_t ehca_store_debug_level(struct device_driver *ddp, return 1; } -DRIVER_ATTR(debug_level, S_IRUSR | S_IWUSR, - ehca_show_debug_level, ehca_store_debug_level); +static DRIVER_ATTR(debug_level, S_IRUSR | S_IWUSR, + ehca_show_debug_level, ehca_store_debug_level); static struct attribute *ehca_drv_attrs[] = { &driver_attr_debug_level.attr, @@ -941,7 +942,7 @@ void ehca_poll_eqs(unsigned long data) spin_unlock(&shca_list_lock); } -int __init ehca_module_init(void) +static int __init ehca_module_init(void) { int ret; @@ -988,7 +989,7 @@ module_init1: return ret; }; -void __exit ehca_module_exit(void) +static void __exit ehca_module_exit(void) { if (ehca_poll_all_eqs == 1) del_timer_sync(&poll_eqs_timer); diff --git a/drivers/infiniband/hw/ehca/ehca_mrmw.c b/drivers/infiniband/hw/ehca/ehca_mrmw.c index e239bbf..5e99c45 100644 --- a/drivers/infiniband/hw/ehca/ehca_mrmw.c +++ b/drivers/infiniband/hw/ehca/ehca_mrmw.c @@ -1952,9 +1952,8 @@ next_kpage: return ret; } -int ehca_set_pagebuf_phys(struct ehca_mr_pginfo *pginfo, - u32 number, - u64 *kpage) +static int ehca_set_pagebuf_phys(struct ehca_mr_pginfo *pginfo, + u32 number, u64 *kpage) { int ret = 0; struct ib_phys_buf *pbuf; @@ -2012,9 +2011,8 @@ int ehca_set_pagebuf_phys(struct ehca_mr_pginfo *pginfo, return ret; } -int ehca_set_pagebuf_fmr(struct ehca_mr_pginfo *pginfo, - u32 number, - u64 *kpage) +static int ehca_set_pagebuf_fmr(struct ehca_mr_pginfo *pginfo, + u32 number, u64 *kpage) { int ret = 0; u64 *fmrlist; -- 1.5.4.3 From kiyomi at jcom.home.ne.jp Tue Mar 18 14:07:59 2008 From: kiyomi at jcom.home.ne.jp (University Degree) Date: Tue, 18 Mar 2008 16:07:59 -0500 Subject: [ofa-general] Obtain University Degree qQickly Message-ID: <248433075.43698640052684@jcom.home.ne.jp> 1-410-230-1849 University Degree These are real, genuine degrees that include Bachelors, Masters, MBA and Doctorate Degrees. Confidentiality Assured 1-410-230-1849 24 hours a day, 7 days a week including Sundays and Holidays -------------- next part -------------- An HTML attachment was scrubbed... URL: From Golden-01 at Mail.ru Tue Mar 18 14:43:32 2008 From: Golden-01 at Mail.ru () Date: Wed, 19 Mar 2008 00:43:32 +0300 Subject: [ofa-general] =?iso-8859-1?q?=C2=E0=F8_=E4=EE=EF=EE=EB=ED=E8=F2?= =?iso-8859-1?q?=E5=EB=FC=ED=FB=E9_=E7=E0=F0=E0=E1=EE=F2=EE=EA=2C?= =?iso-8859-1?q?=E0_=E2=EE=E7=EC=EE=E6=ED=EE_=E8_=EE=E1=E5=F1=EF=E5?= =?iso-8859-1?q?=F7=E5=ED=ED=EE=E5_=E1=F3=E4=F3=F9=E5=E5=2E?= Message-ID: <200803190043.ZCFNCDNBLE@82.138.52.149>                                                                                Добрый день !      Извините,что,возможно ,отнимаю у Вас время,но если Вы имеете  доступ к Интернету,умеете пользоваться  мышкой,   имеете 2-3 часа свободного времени,и Вас интересуют деньги,то всего один щелчок мышкой может отделять Вас от состояния !!! Еще раз простите за вторжение.Желаю Вам удачного дня . Жду Вашего ответа : Moskvich-V at Mail.ru  или  Golden-V at Yandex.ru Удачи Вам !!! Информация находиться в прикрепленном   файле  .... Не беспокойтесь, это не вирус,но если Вы ,тем не менее,  опасаетесь  его  открыть  ,не  спешите  его  удалять ,вовсе не сложно проверить почту при помощи антивируса. Это не реклама, а действительно выгодное предложение !!! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Golden Stream.doc Type: application/octet-stream Size: 160256 bytes Desc: not available URL: From ardavis at ichips.intel.com Tue Mar 18 15:37:06 2008 From: ardavis at ichips.intel.com (Arlin Davis) Date: Tue, 18 Mar 2008 15:37:06 -0700 Subject: [ofa-general] Re: [PATCH 1/2] [uDAPL v1] fix provider-specific compiler warnings In-Reply-To: <20080314213015.12201.6949.stgit@b64-10.mv.qlogic.com> References: <20080314213015.12201.6949.stgit@b64-10.mv.qlogic.com> Message-ID: <47E04412.9030105@ichips.intel.com> Patrick Marchand Latifi wrote: > Initialize ds_array_start_p otherwise the compiler would claim > that this variable could be used with an uninitialized value. > > Makes the uDAPL providers now build successfully when using make > VERBS= (the -Werror flag was causing the build failure) > > Signed-off-by: Patrick Marchand Latifi Thanks. applied v1 and v2 patches. From sfr at canb.auug.org.au Tue Mar 18 15:54:08 2008 From: sfr at canb.auug.org.au (Stephen Rothwell) Date: Wed, 19 Mar 2008 09:54:08 +1100 Subject: [ofa-general] linux-next: infiniband merge conflict In-Reply-To: References: <20080318163216.f5cb148e.sfr@canb.auug.org.au> Message-ID: <20080319095408.aec45505.sfr@canb.auug.org.au> On Tue, 18 Mar 2008 11:29:44 -0700 Roland Dreier wrote: > > > Yesterday and today, I got a merge conflict between the infiniband tree > > and driver-core patch > > ib-convert-struct-class_device-to-struct-device.patch. The resolution is > > pretty tirivial, but maybe a better solution is for that patch to be > > added to the infinband tree. > > I assume it's the __FUNCTION__ -> __func__ conversion causing the clash? Yes, that and "RDMA/nes: Remove redundant NULL check in nes_unregister_ofa_device()" removes some of the context of Greg's patch. > Greg, I'm fine with merging that patch through my tree. The only > potential problem I see is if you also have the final class_device > removal patch in your tree too, in which case your tree then depends > on my tree in an unfortunate way. I keep reverting that patch (and hope Greg will remove it from his linux-next series (NEXT_PATCHES_START/END hint, hint :-)) -- Cheers, Stephen Rothwell sfr at canb.auug.org.au http://www.canb.auug.org.au/~sfr/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From rdreier at cisco.com Tue Mar 18 16:04:09 2008 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 18 Mar 2008 16:04:09 -0700 Subject: [ofa-general] linux-next: infiniband merge conflict In-Reply-To: <20080319095408.aec45505.sfr@canb.auug.org.au> (Stephen Rothwell's message of "Wed, 19 Mar 2008 09:54:08 +1100") References: <20080318163216.f5cb148e.sfr@canb.auug.org.au> <20080319095408.aec45505.sfr@canb.auug.org.au> Message-ID: > > Greg, I'm fine with merging that patch through my tree. The only > > potential problem I see is if you also have the final class_device > > removal patch in your tree too, in which case your tree then depends > > on my tree in an unfortunate way. > > I keep reverting that patch (and hope Greg will remove it from his > linux-next series (NEXT_PATCHES_START/END hint, hint :-)) Sorry, I'm not able to parse the antecedent of "that" in your sentence. Which patch are you hoping about? - R. From sfr at canb.auug.org.au Tue Mar 18 16:07:55 2008 From: sfr at canb.auug.org.au (Stephen Rothwell) Date: Wed, 19 Mar 2008 10:07:55 +1100 Subject: [ofa-general] linux-next: infiniband merge conflict In-Reply-To: References: <20080318163216.f5cb148e.sfr@canb.auug.org.au> <20080319095408.aec45505.sfr@canb.auug.org.au> Message-ID: <20080319100755.9bbfd089.sfr@canb.auug.org.au> On Tue, 18 Mar 2008 16:04:09 -0700 Roland Dreier wrote: > > > > Greg, I'm fine with merging that patch through my tree. The only > > > potential problem I see is if you also have the final class_device > > > removal patch in your tree too, in which case your tree then depends > > > on my tree in an unfortunate way. > > > > I keep reverting that patch (and hope Greg will remove it from his > > linux-next series (NEXT_PATCHES_START/END hint, hint :-)) > > Sorry, I'm not able to parse the antecedent of "that" in your > sentence. Which patch are you hoping about? The one that removes struct class_device (driver-core-remove-no-longer-used-struct-class_device.patch). Greg has agreed that that patch needs to be merged late. -- Cheers, Stephen Rothwell sfr at canb.auug.org.au http://www.canb.auug.org.au/~sfr/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From arlin.r.davis at intel.com Tue Mar 18 16:31:01 2008 From: arlin.r.davis at intel.com (Arlin Davis) Date: Tue, 18 Mar 2008 16:31:01 -0700 Subject: [ofa-general] [PATCH 1/1] [v1] remove unnecessary assert from dapl_ep_free. Message-ID: <000001c88950$20c01f70$9f97070a@amr.corp.intel.com> dat_ep_free must handle the case where a consumer calls free in CONNECTED or DISCONNECT_PENDING states. After free calls disconnect, there may be a pending event, in which case the providers dapls_ib_qp_free will block accordingly and handle pending events. Signed-off by: Arlin Davis ardavis at ichips.intel.com --- dapl/common/dapl_ep_free.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-) diff --git a/dapl/common/dapl_ep_free.c b/dapl/common/dapl_ep_free.c index 3bf41ab..4326825 100644 --- a/dapl/common/dapl_ep_free.c +++ b/dapl/common/dapl_ep_free.c @@ -110,14 +110,22 @@ dapl_ep_free ( * Invoke ep_disconnect to clean up outstanding connections */ (void) dapl_ep_disconnect (ep_ptr, DAT_CLOSE_ABRUPT_FLAG); - dapl_os_assert (ep_ptr->param.ep_state == DAT_EP_STATE_DISCONNECTED || - ep_ptr->param.ep_state == DAT_EP_STATE_UNCONNECTED); /* * Do verification of parameters and the state change atomically. */ dapl_os_lock ( &ep_ptr->header.lock ); +#ifdef DAPL_DBG + /* check if event pending and warn, don't assert, state is valid */ + if (ep_ptr->param.ep_state == DAT_EP_STATE_DISCONNECT_PENDING) { + dapl_dbg_log (DAPL_DBG_TYPE_WARN, " dat_ep_free WARNING: " + "EVENT PENDING on ep %p, disconnect " + "and wait before calling dat_ep_free\n", + ep_ptr); + } +#endif + if (ep_ptr->cxn_timer != NULL) { dapls_timer_cancel ( ep_ptr->cxn_timer ); -- 1.5.2.5 From arlin.r.davis at intel.com Tue Mar 18 16:32:15 2008 From: arlin.r.davis at intel.com (Arlin Davis) Date: Tue, 18 Mar 2008 16:32:15 -0700 Subject: [ofa-general] [PATCH 1/1] [v2] remove unnecessary assert from dapl_ep_free. Message-ID: <000101c88950$4c921450$9f97070a@amr.corp.intel.com> dat_ep_free must handle the case where a consumer calls free in CONNECTED or DISCONNECT_PENDING states. After free calls disconnect, there may be a pending event, in which case the providers dapls_ib_qp_free will block accordingly and handle pending events. Signed-off by: Arlin Davis ardavis at ichips.intel.com --- dapl/common/dapl_ep_free.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-) diff --git a/dapl/common/dapl_ep_free.c b/dapl/common/dapl_ep_free.c index e4e6944..62d9644 100644 --- a/dapl/common/dapl_ep_free.c +++ b/dapl/common/dapl_ep_free.c @@ -111,14 +111,22 @@ dapl_ep_free ( * Invoke ep_disconnect to clean up outstanding connections */ (void) dapl_ep_disconnect (ep_ptr, DAT_CLOSE_ABRUPT_FLAG); - dapl_os_assert (ep_ptr->param.ep_state == DAT_EP_STATE_DISCONNECTED || - ep_ptr->param.ep_state == DAT_EP_STATE_UNCONNECTED); /* * Do verification of parameters and the state change atomically. */ dapl_os_lock ( &ep_ptr->header.lock ); +#ifdef DAPL_DBG + /* check if event pending and warn, don't assert, state is valid */ + if (ep_ptr->param.ep_state == DAT_EP_STATE_DISCONNECT_PENDING) { + dapl_dbg_log (DAPL_DBG_TYPE_WARN, " dat_ep_free WARNING: " + "EVENT PENDING on ep %p, disconnect " + "and wait before calling dat_ep_free\n", + ep_ptr); + } +#endif + if (ep_ptr->cxn_timer != NULL) { dapls_timer_cancel ( ep_ptr->cxn_timer ); -- 1.5.2.5 From ralph.campbell at qlogic.com Tue Mar 18 17:23:40 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 18 Mar 2008 17:23:40 -0700 Subject: [ofa-general] [PATCH 0/6] IB/ipath -- patches in for-roland for 2.6.26 Message-ID: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> The following patches fix some sparse errors in the ib_ipath driver and other low priority changes so I think they are appropriate for 2.6.26. These can also be pulled into Roland's infiniband.git repo using: git pull git://git.qlogic.com/ipath-linux-2.6 for-roland From ralph.campbell at qlogic.com Tue Mar 18 17:23:46 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 18 Mar 2008 17:23:46 -0700 Subject: [ofa-general] [PATCH 1/6] IB/ipath - misc sparse cleanup In-Reply-To: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> Message-ID: <20080319002346.19088.3336.stgit@eng-46.mv.qlogic.com> From: Arthur Jones Recent sparse versions knock down the false positive rate of the ipath driver code to a point where having it be sparse clean is worthwhile. Here we fixup the sparse warnings. Some of these warnings (and the impetus to run sparse again) are due to work by Roland Dreier. Signed-off-by: Arthur Jones --- drivers/infiniband/hw/ipath/ipath_intr.c | 8 +++++--- drivers/infiniband/hw/ipath/ipath_srq.c | 3 ++- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_intr.c b/drivers/infiniband/hw/ipath/ipath_intr.c index 92e58c9..3b89952 100644 --- a/drivers/infiniband/hw/ipath/ipath_intr.c +++ b/drivers/infiniband/hw/ipath/ipath_intr.c @@ -59,9 +59,11 @@ static void ipath_clrpiobuf(struct ipath_devdata *dd, u32 pnum) dev_info(&dd->pcidev->dev, "Rewrite PIO buffer %u, to recover from parity error\n", pnum); - *pbuf = dwcnt+1; /* no flush required, since already in freeze */ - while(--dwcnt) - *pbuf++ = 0; + + /* no flush required, since already in freeze */ + writel(dwcnt + 1, pbuf); + while (--dwcnt) + writel(0, pbuf++); } /* diff --git a/drivers/infiniband/hw/ipath/ipath_srq.c b/drivers/infiniband/hw/ipath/ipath_srq.c index f772102..3366d66 100644 --- a/drivers/infiniband/hw/ipath/ipath_srq.c +++ b/drivers/infiniband/hw/ipath/ipath_srq.c @@ -245,7 +245,8 @@ int ipath_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr, sizeof(offset_addr)); if (ret) goto bail_free; - udata->outbuf = (void __user *) offset_addr; + udata->outbuf = + (void __user *) (unsigned long) offset_addr; ret = ib_copy_to_udata(udata, &offset, sizeof(offset)); if (ret) From ralph.campbell at qlogic.com Tue Mar 18 17:23:51 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 18 Mar 2008 17:23:51 -0700 Subject: [ofa-general] [PATCH 2/6] IB/ipath - make some constants chip-specific, related cleanup In-Reply-To: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> Message-ID: <20080319002351.19088.78025.stgit@eng-46.mv.qlogic.com> From: Dave Olson This patch makes some constants chip-specific, and makes some related changes to prepare for supporting another HCA. Signed-off-by: Dave Olson #include #include +#include #include "ipath_kernel.h" #include "ipath_registers.h" @@ -476,7 +477,13 @@ static const struct ipath_hwerror_msgs ipath_6110_hwerror_msgs[] = { #define RXE_EAGER_PARITY (INFINIPATH_HWE_RXEMEMPARITYERR_EAGERTID \ << INFINIPATH_HWE_RXEMEMPARITYERR_SHIFT) -static int ipath_ht_txe_recover(struct ipath_devdata *); +static void ipath_ht_txe_recover(struct ipath_devdata *dd) +{ + ++ipath_stats.sps_txeparity; + dev_info(&dd->pcidev->dev, + "Recovering from TXE PIO parity error\n"); +} + /** * ipath_ht_handle_hwerrors - display hardware errors. @@ -557,11 +564,11 @@ static void ipath_ht_handle_hwerrors(struct ipath_devdata *dd, char *msg, * occur if a processor speculative read is done to the PIO * buffer while we are sending a packet, for example. */ - if ((hwerrs & TXE_PIO_PARITY) && ipath_ht_txe_recover(dd)) + if (hwerrs & TXE_PIO_PARITY) { + ipath_ht_txe_recover(dd); hwerrs &= ~TXE_PIO_PARITY; - if (hwerrs & RXE_EAGER_PARITY) - ipath_dev_err(dd, "RXE parity, Eager TID error is not " - "recoverable\n"); + } + if (!hwerrs) { ipath_dbg("Clearing freezemode on ignored or " "recovered hardware error\n"); @@ -1653,22 +1660,6 @@ static int ipath_ht_early_init(struct ipath_devdata *dd) } -static int ipath_ht_txe_recover(struct ipath_devdata *dd) -{ - int cnt = ++ipath_stats.sps_txeparity; - if (cnt >= IPATH_MAX_PARITY_ATTEMPTS) { - if (cnt == IPATH_MAX_PARITY_ATTEMPTS) - ipath_dev_err(dd, - "Too many attempts to recover from " - "TXE parity, giving up\n"); - return 0; - } - dev_info(&dd->pcidev->dev, - "Recovering from TXE PIO parity error\n"); - return 1; -} - - /** * ipath_init_ht_get_base_info - set chip-specific flags for user code * @dd: the infinipath device diff --git a/drivers/infiniband/hw/ipath/ipath_iba6120.c b/drivers/infiniband/hw/ipath/ipath_iba6120.c index c7a2f50..8423fee 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6120.c +++ b/drivers/infiniband/hw/ipath/ipath_iba6120.c @@ -38,7 +38,7 @@ #include #include #include - +#include #include "ipath_kernel.h" #include "ipath_registers.h" @@ -311,6 +311,9 @@ static const struct ipath_cregs ipath_pe_cregs = { .cr_ibsymbolerrcnt = IPATH_CREG_OFFSET(IBSymbolErrCnt) }; +/* kr_control bits */ +#define INFINIPATH_C_RESET 1U + /* kr_intstatus, kr_intclear, kr_intmask bits */ #define INFINIPATH_I_RCVURG_MASK ((1U<<5)-1) #define INFINIPATH_I_RCVAVAIL_MASK ((1U<<5)-1) @@ -338,6 +341,9 @@ static const struct ipath_cregs ipath_pe_cregs = { #define INFINIPATH_EXTS_MEMBIST_ENDTEST 0x0000000000004000 #define INFINIPATH_EXTS_MEMBIST_FOUND 0x0000000000008000 +/* kr_xgxsconfig bits */ +#define INFINIPATH_XGXS_RESET 0x5ULL + #define _IPATH_GPIO_SDA_NUM 1 #define _IPATH_GPIO_SCL_NUM 0 @@ -346,6 +352,16 @@ static const struct ipath_cregs ipath_pe_cregs = { #define IPATH_GPIO_SCL (1ULL << \ (_IPATH_GPIO_SCL_NUM+INFINIPATH_EXTC_GPIOOE_SHIFT)) +#define INFINIPATH_RT_BUFSIZE_MASK 0xe0000000ULL +#define INFINIPATH_RT_BUFSIZE_SHIFTVAL(tid) \ + ((((tid) & INFINIPATH_RT_BUFSIZE_MASK) >> 29) + 11 - 1) +#define INFINIPATH_RT_BUFSIZE(tid) (1 << INFINIPATH_RT_BUFSIZE_SHIFTVAL(tid)) +#define INFINIPATH_RT_IS_VALID(tid) \ + (((tid) & INFINIPATH_RT_BUFSIZE_MASK) && \ + ((((tid) & INFINIPATH_RT_BUFSIZE_MASK) != INFINIPATH_RT_BUFSIZE_MASK))) +#define INFINIPATH_RT_ADDR_MASK 0x1FFFFFFFULL /* 29 bits valid */ +#define INFINIPATH_RT_ADDR_SHIFT 10 + #define INFINIPATH_R_INTRAVAIL_SHIFT 16 #define INFINIPATH_R_TAILUPD_SHIFT 31 @@ -372,6 +388,8 @@ static const struct ipath_hwerror_msgs ipath_6120_hwerror_msgs[] = { #define TXE_PIO_PARITY ((INFINIPATH_HWE_TXEMEMPARITYERR_PIOBUF | \ INFINIPATH_HWE_TXEMEMPARITYERR_PIOPBC) \ << INFINIPATH_HWE_TXEMEMPARITYERR_SHIFT) +#define RXE_EAGER_PARITY (INFINIPATH_HWE_RXEMEMPARITYERR_EAGERTID \ + << INFINIPATH_HWE_RXEMEMPARITYERR_SHIFT) static void ipath_pe_put_tid_2(struct ipath_devdata *, u64 __iomem *, u32, unsigned long); @@ -450,10 +468,8 @@ static void ipath_pe_handle_hwerrors(struct ipath_devdata *dd, char *msg, * make sure we get this much out, unless told to be quiet, * or it's occurred within the last 5 seconds */ - if ((hwerrs & ~(dd->ipath_lasthwerror | - ((INFINIPATH_HWE_TXEMEMPARITYERR_PIOBUF | - INFINIPATH_HWE_TXEMEMPARITYERR_PIOPBC) - << INFINIPATH_HWE_TXEMEMPARITYERR_SHIFT))) || + if ((hwerrs & ~(dd->ipath_lasthwerror | TXE_PIO_PARITY | + RXE_EAGER_PARITY)) || (ipath_debug & __IPATH_VERBDBG)) dev_info(&dd->pcidev->dev, "Hardware error: hwerr=0x%llx " "(cleared)\n", (unsigned long long) hwerrs); @@ -1218,7 +1234,7 @@ static void ipath_pe_put_tid(struct ipath_devdata *dd, u64 __iomem *tidptr, } pa >>= 11; /* paranoia check */ - if (pa & (7<<29)) + if (pa & ~INFINIPATH_RT_ADDR_MASK) ipath_dev_err(dd, "BUG: Physical page address 0x%lx " "has bits set in 31-29\n", pa); @@ -1270,7 +1286,7 @@ static void ipath_pe_put_tid_2(struct ipath_devdata *dd, u64 __iomem *tidptr, } pa >>= 11; /* paranoia check */ - if (pa & (7<<29)) + if (pa & ~INFINIPATH_RT_ADDR_MASK) ipath_dev_err(dd, "BUG: Physical page address 0x%lx " "has bits set in 31-29\n", pa); diff --git a/drivers/infiniband/hw/ipath/ipath_registers.h b/drivers/infiniband/hw/ipath/ipath_registers.h index 92ad73a..cb19ea2 100644 --- a/drivers/infiniband/hw/ipath/ipath_registers.h +++ b/drivers/infiniband/hw/ipath/ipath_registers.h @@ -63,7 +63,6 @@ /* kr_control bits */ #define INFINIPATH_C_FREEZEMODE 0x00000002 #define INFINIPATH_C_LINKENABLE 0x00000004 -#define INFINIPATH_C_RESET 0x00000001 /* kr_sendctrl bits */ #define INFINIPATH_S_DISARMPIOBUF_SHIFT 16 @@ -287,7 +286,6 @@ #define INFINIPATH_SERDC0_L1PWR_DN 0xF0ULL /* kr_xgxsconfig bits */ -#define INFINIPATH_XGXS_RESET 0x7ULL #define INFINIPATH_XGXS_RX_POL_SHIFT 19 #define INFINIPATH_XGXS_RX_POL_MASK 0xfULL From ralph.campbell at qlogic.com Tue Mar 18 17:23:56 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 18 Mar 2008 17:23:56 -0700 Subject: [ofa-general] [PATCH 3/6] IB/ipath - provide I/O bus speeds for diagnostic purposes In-Reply-To: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> Message-ID: <20080319002356.19088.66871.stgit@eng-46.mv.qlogic.com> From: Arthur Jones Modern I/O buses like PCIe and HT can be configured for multiple speeds and widths similar to Infiniband. When an ipath HCA seems to have lower than expected performance, it is very useful to be able to display what the driver thinks the bus speed is. Signed-off-by: Dave Olson --- drivers/infiniband/hw/ipath/ipath_iba6110.c | 14 +++-- drivers/infiniband/hw/ipath/ipath_iba6120.c | 74 ++++++++++++++++++++++----- drivers/infiniband/hw/ipath/ipath_kernel.h | 9 ++- drivers/infiniband/hw/ipath/ipath_sysfs.c | 11 ++++ 4 files changed, 85 insertions(+), 23 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_iba6110.c b/drivers/infiniband/hw/ipath/ipath_iba6110.c index 684c27e..76c8030 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6110.c +++ b/drivers/infiniband/hw/ipath/ipath_iba6110.c @@ -742,11 +742,10 @@ static int ipath_ht_boardname(struct ipath_devdata *dd, char *name, */ dd->ipath_flags |= IPATH_32BITCOUNTERS; dd->ipath_flags |= IPATH_GPIO_INTR; - if (dd->ipath_htspeed != 800) + if (dd->ipath_lbus_speed != 800) ipath_dev_err(dd, "Incorrectly configured for HT @ %uMHz\n", - dd->ipath_htspeed); - ret = 0; + dd->ipath_lbus_speed); /* * set here, not in ipath_init_*_funcs because we have to do @@ -911,7 +910,7 @@ static void slave_or_pri_blk(struct ipath_devdata *dd, struct pci_dev *pdev, break; } - dd->ipath_htwidth = width; + dd->ipath_lbus_width = width; if (linkwidth != 0x11) { ipath_dev_err(dd, "Not configured for 16 bit HT " @@ -959,8 +958,13 @@ static void slave_or_pri_blk(struct ipath_devdata *dd, struct pci_dev *pdev, speed = 200; break; } - dd->ipath_htspeed = speed; + dd->ipath_lbus_speed = speed; } + snprintf(dd->ipath_lbus_info, sizeof(dd->ipath_lbus_info), + "HyperTransport,%uMHz,x%u\n", + dd->ipath_lbus_speed, + dd->ipath_lbus_width); + } static int ipath_ht_intconfig(struct ipath_devdata *dd) diff --git a/drivers/infiniband/hw/ipath/ipath_iba6120.c b/drivers/infiniband/hw/ipath/ipath_iba6120.c index 8423fee..828066e 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6120.c +++ b/drivers/infiniband/hw/ipath/ipath_iba6120.c @@ -626,7 +626,6 @@ static int ipath_pe_boardname(struct ipath_devdata *dd, char *name, dd->ipath_f_put_tid = ipath_pe_put_tid_2; } - /* * set here, not in ipath_init_*_funcs because we have to do * it after we can read chip registers. @@ -879,6 +878,62 @@ static void ipath_setup_pe_cleanup(struct ipath_devdata *dd) pci_disable_msi(dd->pcidev); } +static void ipath_6120_pcie_params(struct ipath_devdata *dd) +{ + u16 linkstat, speed; + int pos; + + pos = pci_find_capability(dd->pcidev, PCI_CAP_ID_EXP); + if (!pos) { + ipath_dev_err(dd, "Can't find PCI Express capability!\n"); + goto bail; + } + + pci_read_config_word(dd->pcidev, pos + PCI_EXP_LNKSTA, + &linkstat); + /* + * speed is bits 0-4, linkwidth is bits 4-8 + * no defines for them in headers + */ + speed = linkstat & 0xf; + linkstat >>= 4; + linkstat &= 0x1f; + dd->ipath_lbus_width = linkstat; + + switch (speed) { + case 1: + dd->ipath_lbus_speed = 2500; /* Gen1, 2.5GHz */ + break; + case 2: + dd->ipath_lbus_speed = 5000; /* Gen1, 5GHz */ + break; + default: /* not defined, assume gen1 */ + dd->ipath_lbus_speed = 2500; + break; + } + + if (linkstat < 8) + ipath_dev_err(dd, + "PCIe width %u (x8 HCA), performance reduced\n", + linkstat); + else + ipath_cdbg(VERBOSE, "PCIe speed %u width %u (x8 HCA)\n", + dd->ipath_lbus_speed, linkstat); + + if (speed != 1) + ipath_dev_err(dd, + "PCIe linkspeed %u is incorrect; " + "should be 1 (2500)!\n", speed); +bail: + /* fill in string, even on errors */ + snprintf(dd->ipath_lbus_info, sizeof(dd->ipath_lbus_info), + "PCIe,%uMHz,x%u\n", + dd->ipath_lbus_speed, + dd->ipath_lbus_width); + + return; +} + /** * ipath_setup_pe_config - setup PCIe config related stuff * @dd: the infinipath device @@ -936,19 +991,8 @@ static int ipath_setup_pe_config(struct ipath_devdata *dd, } else ipath_dev_err(dd, "Can't find MSI capability, " "can't save MSI settings for reset\n"); - if ((pos = pci_find_capability(dd->pcidev, PCI_CAP_ID_EXP))) { - u16 linkstat; - pci_read_config_word(dd->pcidev, pos + PCI_EXP_LNKSTA, - &linkstat); - linkstat >>= 4; - linkstat &= 0x1f; - if (linkstat != 8) - ipath_dev_err(dd, "PCIe width %u, " - "performance reduced\n", linkstat); - } - else - ipath_dev_err(dd, "Can't find PCI Express " - "capability!\n"); + + ipath_6120_pcie_params(dd); dd->ipath_link_width_supported = IB_WIDTH_1X | IB_WIDTH_4X; dd->ipath_link_speed_supported = IPATH_IB_SDR; @@ -1206,6 +1250,8 @@ static int ipath_setup_pe_reset(struct ipath_devdata *dd) ret = 0; /* failed */ bail: + if (ret) + ipath_6120_pcie_params(dd); return ret; } diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 3d4a254..59dc895 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -546,10 +546,10 @@ struct ipath_devdata { u32 ipath_init_ibmaxlen; /* size of each rcvegrbuffer */ u32 ipath_rcvegrbufsize; - /* width (2,4,8,16,32) from HT config reg */ - u32 ipath_htwidth; - /* HT speed (200,400,800,1000) from HT config */ - u32 ipath_htspeed; + /* localbus width (1, 2,4,8,16,32) from config space */ + u32 ipath_lbus_width; + /* localbus speed (HT: 200,400,800,1000; PCIe 2500) */ + u32 ipath_lbus_speed; /* * number of sequential ibcstatus change for polling active/quiet * (i.e., link not coming up). @@ -574,6 +574,7 @@ struct ipath_devdata { u8 ipath_serial[16]; /* human readable board version */ u8 ipath_boardversion[80]; + u8 ipath_lbus_info[32]; /* human readable localbus info */ /* chip major rev, from ipath_revision */ u8 ipath_majrev; /* chip minor rev, from ipath_revision */ diff --git a/drivers/infiniband/hw/ipath/ipath_sysfs.c b/drivers/infiniband/hw/ipath/ipath_sysfs.c index 56dfc8a..bb41c3f 100644 --- a/drivers/infiniband/hw/ipath/ipath_sysfs.c +++ b/drivers/infiniband/hw/ipath/ipath_sysfs.c @@ -163,6 +163,15 @@ static ssize_t show_boardversion(struct device *dev, return scnprintf(buf, PAGE_SIZE, "%s", dd->ipath_boardversion); } +static ssize_t show_localbus_info(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct ipath_devdata *dd = dev_get_drvdata(dev); + /* The string printed here is already newline-terminated. */ + return scnprintf(buf, PAGE_SIZE, "%s", dd->ipath_lbus_info); +} + static ssize_t show_lmc(struct device *dev, struct device_attribute *attr, char *buf) @@ -1011,6 +1020,7 @@ static DEVICE_ATTR(unit, S_IRUGO, show_unit, NULL); static DEVICE_ATTR(rx_pol_inv, S_IWUSR, NULL, store_rx_pol_inv); static DEVICE_ATTR(led_override, S_IWUSR, NULL, store_led_override); static DEVICE_ATTR(logged_errors, S_IRUGO, show_logged_errs, NULL); +static DEVICE_ATTR(localbus_info, S_IRUGO, show_localbus_info, NULL); static DEVICE_ATTR(jint_max_packets, S_IWUSR | S_IRUGO, show_jint_max_packets, store_jint_max_packets); static DEVICE_ATTR(jint_idle_ticks, S_IWUSR | S_IRUGO, @@ -1034,6 +1044,7 @@ static struct attribute *dev_attributes[] = { &dev_attr_rx_pol_inv.attr, &dev_attr_led_override.attr, &dev_attr_logged_errors.attr, + &dev_attr_localbus_info.attr, NULL }; From ralph.campbell at qlogic.com Tue Mar 18 17:24:01 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 18 Mar 2008 17:24:01 -0700 Subject: [ofa-general] [PATCH 4/6] IB/ipath - Enable use of 4KB MTU via module paramater In-Reply-To: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> Message-ID: <20080319002401.19088.29911.stgit@eng-46.mv.qlogic.com> From: Dave Olson This patch enables use of 4KB MTUs via module paramater. Signed-off-by: Dave Olson --- drivers/infiniband/hw/ipath/ipath_driver.c | 31 ++++++++++++------------- drivers/infiniband/hw/ipath/ipath_file_ops.c | 7 +++++- drivers/infiniband/hw/ipath/ipath_iba6120.c | 10 ++------ drivers/infiniband/hw/ipath/ipath_init_chip.c | 30 ++++++------------------ drivers/infiniband/hw/ipath/ipath_kernel.h | 1 + drivers/infiniband/hw/ipath/ipath_mad.c | 12 ++++------ drivers/infiniband/hw/ipath/ipath_qp.c | 12 +++++----- drivers/infiniband/hw/ipath/ipath_verbs.c | 7 +----- 8 files changed, 45 insertions(+), 65 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index 367f2a3..7121fe8 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -73,6 +73,10 @@ module_param_named(debug, ipath_debug, uint, S_IWUSR | S_IRUGO); MODULE_PARM_DESC(debug, "mask for debug prints"); EXPORT_SYMBOL_GPL(ipath_debug); +unsigned ipath_mtu4096 = 1; /* max 4KB IB mtu by default, if supported */ +module_param_named(mtu4096, ipath_mtu4096, uint, S_IRUGO); +MODULE_PARM_DESC(mtu4096, "enable MTU of 4096 bytes, if supported"); + MODULE_LICENSE("GPL"); MODULE_AUTHOR("QLogic "); MODULE_DESCRIPTION("QLogic InfiniPath driver"); @@ -1800,7 +1804,7 @@ int ipath_set_mtu(struct ipath_devdata *dd, u16 arg) * piosize). We check that it's one of the valid IB sizes. */ if (arg != 256 && arg != 512 && arg != 1024 && arg != 2048 && - arg != 4096) { + (arg != 4096 || !ipath_mtu4096)) { ipath_dbg("Trying to set invalid mtu %u, failing\n", arg); ret = -EINVAL; goto bail; @@ -1816,6 +1820,8 @@ int ipath_set_mtu(struct ipath_devdata *dd, u16 arg) if (arg >= (piosize - IPATH_PIO_MAXIBHDR)) { /* Only if it's not the initial value (or reset to it) */ if (piosize != dd->ipath_init_ibmaxlen) { + if (arg > piosize && arg <= dd->ipath_init_ibmaxlen) + piosize = dd->ipath_init_ibmaxlen; dd->ipath_ibmaxlen = piosize; changed = 1; } @@ -1829,24 +1835,17 @@ int ipath_set_mtu(struct ipath_devdata *dd, u16 arg) } if (changed) { + u64 ibc = dd->ipath_ibcctrl, ibdw; /* - * set the IBC maxpktlength to the size of our pio - * buffers in words + * update our housekeeping variables, and set IBC max + * size, same as init code; max IBC is max we allow in + * buffer, less the qword pbc, plus 1 for ICRC, in dwords */ - u64 ibc = dd->ipath_ibcctrl; + dd->ipath_ibmaxlen = piosize - 2 * sizeof(u32); + ibdw = (dd->ipath_ibmaxlen >> 2) + 1; ibc &= ~(INFINIPATH_IBCC_MAXPKTLEN_MASK << - INFINIPATH_IBCC_MAXPKTLEN_SHIFT); - - piosize = piosize - 2 * sizeof(u32); /* ignore pbc */ - dd->ipath_ibmaxlen = piosize; - piosize /= sizeof(u32); /* in words */ - /* - * for ICRC, which we only send in diag test pkt mode, and - * we don't need to worry about that for mtu - */ - piosize += 1; - - ibc |= piosize << INFINIPATH_IBCC_MAXPKTLEN_SHIFT; + dd->ibcc_mpl_shift); + ibc |= ibdw << dd->ibcc_mpl_shift; dd->ipath_ibcctrl = ibc; ipath_write_kreg(dd, dd->ipath_kregs->kr_ibcctrl, dd->ipath_ibcctrl); diff --git a/drivers/infiniband/hw/ipath/ipath_file_ops.c b/drivers/infiniband/hw/ipath/ipath_file_ops.c index 7e025c8..3d95b70 100644 --- a/drivers/infiniband/hw/ipath/ipath_file_ops.c +++ b/drivers/infiniband/hw/ipath/ipath_file_ops.c @@ -219,7 +219,12 @@ static int ipath_get_base_info(struct file *fp, kinfo->spi_pioalign = dd->ipath_palign; kinfo->spi_qpair = IPATH_KD_QP; - kinfo->spi_piosize = dd->ipath_ibmaxlen; + /* + * user mode PIO buffers are always 2KB, even when 4KB can + * be received, and sent via the kernel; this is ibmaxlen + * for 2K MTU. + */ + kinfo->spi_piosize = dd->ipath_piosize2k - 2 * sizeof(u32); kinfo->spi_mtu = dd->ipath_ibmaxlen; /* maxlen, not ibmtu */ kinfo->spi_port = pd->port_port; kinfo->spi_subport = subport_fp(fp); diff --git a/drivers/infiniband/hw/ipath/ipath_iba6120.c b/drivers/infiniband/hw/ipath/ipath_iba6120.c index 828066e..a9fc804 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6120.c +++ b/drivers/infiniband/hw/ipath/ipath_iba6120.c @@ -1441,17 +1441,13 @@ static int ipath_pe_early_init(struct ipath_devdata *dd) dd->ipath_egrtidbase = (u64 __iomem *) ((char __iomem *) dd->ipath_kregbase + dd->ipath_rcvegrbase); - /* - * To truly support a 4KB MTU (for usermode), we need to - * bump this to a larger value. For now, we use them for - * the kernel only. - */ - dd->ipath_rcvegrbufsize = 2048; + dd->ipath_rcvegrbufsize = ipath_mtu4096 ? 4096 : 2048; /* * the min() check here is currently a nop, but it may not always * be, depending on just how we do ipath_rcvegrbufsize */ - dd->ipath_ibmaxlen = min(dd->ipath_piosize2k, + dd->ipath_ibmaxlen = min(ipath_mtu4096 ? dd->ipath_piosize4k : + dd->ipath_piosize2k, dd->ipath_rcvegrbufsize + (dd->ipath_rcvhdrentsize << 2)); dd->ipath_init_ibmaxlen = dd->ipath_ibmaxlen; diff --git a/drivers/infiniband/hw/ipath/ipath_init_chip.c b/drivers/infiniband/hw/ipath/ipath_init_chip.c index 5428aff..f0d7848 100644 --- a/drivers/infiniband/hw/ipath/ipath_init_chip.c +++ b/drivers/infiniband/hw/ipath/ipath_init_chip.c @@ -155,24 +155,13 @@ static int bringup_link(struct ipath_devdata *dd) dd->ipath_control); /* - * Note that prior to try 14 or 15 of IB, the credit scaling - * wasn't working, because it was swapped for writes with the - * 1 bit default linkstate field + * set initial max size pkt IBC will send, including ICRC; it's the + * PIO buffer size in dwords, less 1; also see ipath_set_mtu() */ + val = (dd->ipath_ibmaxlen >> 2) + 1; + ibc = val << dd->ibcc_mpl_shift; - /* ignore pbc and align word */ - val = dd->ipath_piosize2k - 2 * sizeof(u32); - /* - * for ICRC, which we only send in diag test pkt mode, and we - * don't need to worry about that for mtu - */ - val += 1; - /* - * Set the IBC maxpktlength to the size of our pio buffers the - * maxpktlength is in words. This is *not* the IB data MTU. - */ - ibc = (val / sizeof(u32)) << INFINIPATH_IBCC_MAXPKTLEN_SHIFT; - /* in KB */ + /* flowcontrolwatermark is in units of KBytes */ ibc |= 0x5ULL << INFINIPATH_IBCC_FLOWCTRLWATERMARK_SHIFT; /* * How often flowctrl sent. More or less in usecs; balance against @@ -295,12 +284,9 @@ static int init_chip_first(struct ipath_devdata *dd, val = ipath_read_kreg64(dd, dd->ipath_kregs->kr_sendpiosize); dd->ipath_piosize2k = val & ~0U; dd->ipath_piosize4k = val >> 32; - /* - * Note: the chips support a maximum MTU of 4096, but the driver - * hasn't implemented this feature yet, so set the initial value - * to 2048. - */ - dd->ipath_ibmtu = 2048; + if (dd->ipath_piosize4k == 0 && ipath_mtu4096) + ipath_mtu4096 = 0; /* 4KB not supported by this chip */ + dd->ipath_ibmtu = ipath_mtu4096 ? 4096 : 2048; val = ipath_read_kreg64(dd, dd->ipath_kregs->kr_sendpiobufcnt); dd->ipath_piobcnt2k = val & ~0U; dd->ipath_piobcnt4k = val >> 32; diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 59dc895..70c0a0d 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -1066,6 +1066,7 @@ dma_addr_t ipath_map_single(struct pci_dev *, void *, size_t, int); #endif extern unsigned ipath_debug; /* debugging bit mask */ +extern unsigned ipath_mtu4096; #define IPATH_MAX_PARITY_ATTEMPTS 10000 /* max times to try recovery */ diff --git a/drivers/infiniband/hw/ipath/ipath_mad.c b/drivers/infiniband/hw/ipath/ipath_mad.c index b34b91d..aca876b 100644 --- a/drivers/infiniband/hw/ipath/ipath_mad.c +++ b/drivers/infiniband/hw/ipath/ipath_mad.c @@ -292,13 +292,9 @@ static int recv_subn_get_portinfo(struct ib_smp *smp, /* pip->vl_arb_high_cap; // only one VL */ /* pip->vl_arb_low_cap; // only one VL */ /* InitTypeReply = 0 */ - /* - * Note: the chips support a maximum MTU of 4096, but the driver - * hasn't implemented this feature yet, so set the maximum value - * to 2048. - */ - pip->inittypereply_mtucap = IB_MTU_2048; - // HCAs ignore VLStallCount and HOQLife + /* our mtu cap depends on whether 4K MTU enabled or not */ + pip->inittypereply_mtucap = ipath_mtu4096 ? IB_MTU_4096 : IB_MTU_2048; + /* HCAs ignore VLStallCount and HOQLife */ /* pip->vlstallcnt_hoqlife; */ pip->operationalvl_pei_peo_fpi_fpo = 0x10; /* OVLs = 1 */ pip->mkey_violations = cpu_to_be16(dev->mkey_violations); @@ -491,6 +487,8 @@ static int recv_subn_set_portinfo(struct ib_smp *smp, mtu = 2048; break; case IB_MTU_4096: + if (!ipath_mtu4096) + goto err; mtu = 4096; break; default: diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 087ed31..5c8094a 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -516,13 +516,13 @@ int ipath_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, goto inval; /* - * Note: the chips support a maximum MTU of 4096, but the driver - * hasn't implemented this feature yet, so don't allow Path MTU - * values greater than 2048. + * don't allow invalid Path MTU values or greater than 2048 + * unless we are configured for a 4KB MTU */ - if (attr_mask & IB_QP_PATH_MTU) - if (attr->path_mtu > IB_MTU_2048) - goto inval; + if ((attr_mask & IB_QP_PATH_MTU) && + (ib_mtu_enum_to_int(attr->path_mtu) == -1 || + (attr->path_mtu > IB_MTU_2048 && !ipath_mtu4096))) + goto inval; if (attr_mask & IB_QP_PATH_MIG_STATE) if (attr->path_mig_state != IB_MIG_MIGRATED && diff --git a/drivers/infiniband/hw/ipath/ipath_verbs.c b/drivers/infiniband/hw/ipath/ipath_verbs.c index 32d8f88..012ccb4 100644 --- a/drivers/infiniband/hw/ipath/ipath_verbs.c +++ b/drivers/infiniband/hw/ipath/ipath_verbs.c @@ -1201,12 +1201,7 @@ static int ipath_query_port(struct ib_device *ibdev, props->max_vl_num = 1; /* VLCap = VL0 */ props->init_type_reply = 0; - /* - * Note: the chip supports a maximum MTU of 4096, but the driver - * hasn't implemented this feature yet, so set the maximum value - * to 2048. - */ - props->max_mtu = IB_MTU_2048; + props->max_mtu = ipath_mtu4096 ? IB_MTU_4096 : IB_MTU_2048; switch (dd->ipath_ibmtu) { case 4096: mtu = IB_MTU_4096; From ralph.campbell at qlogic.com Tue Mar 18 17:24:06 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 18 Mar 2008 17:24:06 -0700 Subject: [ofa-general] [PATCH 5/6] IB/ipath - fix a minor code style violation In-Reply-To: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> Message-ID: <20080319002406.19088.25328.stgit@eng-46.mv.qlogic.com> From: Arthur Jones This patch fixes a minor code style violation. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_sysfs.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_sysfs.c b/drivers/infiniband/hw/ipath/ipath_sysfs.c index bb41c3f..58c2805 100644 --- a/drivers/infiniband/hw/ipath/ipath_sysfs.c +++ b/drivers/infiniband/hw/ipath/ipath_sysfs.c @@ -1131,7 +1131,7 @@ int ipath_device_create_group(struct device *dev, struct ipath_devdata *dd) goto bail_max; } - return 0; + goto bail; bail_max: device_remove_file(dev, &dev_attr_jint_max_packets); From ralph.campbell at qlogic.com Tue Mar 18 17:24:11 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 18 Mar 2008 17:24:11 -0700 Subject: [ofa-general] [PATCH 6/6] IB/ipath - shared context code needs to be sure device is usable In-Reply-To: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> Message-ID: <20080319002411.19088.72997.stgit@eng-46.mv.qlogic.com> From: Dave Olson Code was checking for present, but not that present units all were usable (link up, etc.) Signed-off-by: Dave Olson --- drivers/infiniband/hw/ipath/ipath_file_ops.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_file_ops.c b/drivers/infiniband/hw/ipath/ipath_file_ops.c index 3d95b70..cddf29b 100644 --- a/drivers/infiniband/hw/ipath/ipath_file_ops.c +++ b/drivers/infiniband/hw/ipath/ipath_file_ops.c @@ -1765,7 +1765,7 @@ static int find_shared_port(struct file *fp, for (ndev = 0; ndev < devmax; ndev++) { struct ipath_devdata *dd = ipath_lookup(ndev); - if (!dd) + if (!usable(dd)) continue; for (i = 1; i < dd->ipath_cfgports; i++) { struct ipath_portdata *pd = dd->ipath_pd[i]; From haelkrase at gmx.de Tue Mar 18 17:40:46 2008 From: haelkrase at gmx.de (Benny Cope) Date: Wed, 19 Mar 2008 09:40:46 +0900 Subject: [ofa-general] {Viagra_onli2_de} Message-ID: <01c889a5$4e84e300$6d2c8a7d@haelkrase> Online Apotheke - original Qualitaet - 100% wirksam Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro - Kein peinlicher Arztbesuch erforderlich - keine versteckte Kosten - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - Bequem und diskret online bestellen. - Diskrete Verpackung und Zahlung - Kostenlose, arztliche Telefon-Beratung Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen Jetzt bestellen - und vier Pillen umsonst erhalten Jetzt bestellen und ein blaues Wunder erleben (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcella.uliano at giustizia.it Tue Mar 18 17:57:40 2008 From: marcella.uliano at giustizia.it (Antwan Russo) Date: Wed, 19 Mar 2008 08:57:40 +0800 Subject: [ofa-general] Quality Pills, Saving Opportunities Message-ID: <01c8899f$49249a00$18ca7fde@marcella.uliano> TargetPharmacy is your choice when you're looking for the place to buy products in a safe and confidential way. Full range of 100% generic products which are available to order online. Prompt delivery, personal approach, excellent service.12 free bonus pills will be added to any order over $300.Save your money with one mouse click. http://geocities.com/freddywood53/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tem at abenaquicarriers.com Tue Mar 18 19:22:24 2008 From: tem at abenaquicarriers.com (Geneva Hughes) Date: Tue, 18 Mar 2008 21:22:24 -0500 Subject: [ofa-general] Show your loved one you care, help them quit smoking Message-ID: <01c8893e$28782800$8fdde6c9@tem> Stop smoking today At least read it! It is YOUR life, it is important. Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: ls.zip Type: application/zip Size: 560 bytes Desc: not available URL: From dalisha at eburke.com Tue Mar 18 22:35:52 2008 From: dalisha at eburke.com (University Degree) Date: Wed, 19 Mar 2008 11:35:52 +0600 Subject: [ofa-general] Get University Degree in months Message-ID: <767013368.58866808885410@eburke.com> An HTML attachment was scrubbed... URL: From akpm at linux-foundation.org Tue Mar 18 23:02:02 2008 From: akpm at linux-foundation.org (Andrew Morton) Date: Tue, 18 Mar 2008 23:02:02 -0700 Subject: [ofa-general] infiniband sparc64 build Message-ID: <20080318230202.dd47368c.akpm@linux-foundation.org> drivers/infiniband/hw/nes/nes_verbs.c: In function `nes_setup_virt_qp': drivers/infiniband/hw/nes/nes_verbs.c:1047: warning: cast to pointer from integer of different size drivers/infiniband/hw/nes/nes_verbs.c:1078: warning: cast to pointer from integer of different size drivers/infiniband/hw/nes/nes_verbs.c:1078: warning: cast to pointer from integer of different size drivers/infiniband/hw/nes/nes_verbs.c: In function `nes_reg_user_mr': drivers/infiniband/hw/nes/nes_verbs.c:2657: warning: cast to pointer from integer of different size From ogerlitz at voltaire.com Wed Mar 19 01:00:40 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 19 Mar 2008 10:00:40 +0200 Subject: [ofa-general] itt member in iscsi_data? In-Reply-To: References: Message-ID: <47E0C828.8090102@voltaire.com> Roland Dreier wrote: > iser_initiator.c has this code: > > itt = ntohl(hdr->itt); > > which triggers sparse warnings like: > > drivers/infiniband/ulp/iser/iser_initiator.c:419:8: warning: cast to restricted type > drivers/infiniband/ulp/iser/iser_initiator.c:419:8: warning: cast from restricted type > > and indeed the itt field is declared as itt_t, with no particular > endianness, so the sparse warnings seem like they are pointing at > something that really is suspicious. > > It seems the only use of itt in the function that does ntohl() is to > print the value our in debugging statements. What is intended here? > Would it just make sense to for the debug statements to print hdr->itt > out directly with no conversion? > > Roland, Erez is out of the office till the end of the month, so I suggest this will wait for his return, fine? Or. From a-angelj at usa.net Wed Mar 19 01:02:20 2008 From: a-angelj at usa.net (Alfredo Meeks) Date: Wed, 19 Mar 2008 16:02:20 +0800 Subject: [ofa-general] why are you not repying? Message-ID: <01c889da$9c680e00$5341dc3c@a-angelj> Hello! I am tired tonight. I am nice girl that would like to chat with you. Email me at Kristina at BestGolova.com only, because I am using my friend's email to write this. You will see some of my private pics. From k.iten at eckert.ch Wed Mar 19 02:06:31 2008 From: k.iten at eckert.ch (Demi Shelton) Date: Wed, 19 Mar 2008 18:06:31 +0900 Subject: [ofa-general] Grosse Auswahl der Software zum runterladen Message-ID: <01c889eb$f58c6d80$4c040576@k.iten> Brand new software at old priceBekommen Sie Ihre Software unverzueglich. Einfach zahlen und sofort runterladen. Hier sind Programme in allen europaeischen Sprachen verfuegbar, programmiert fuer Windows und Macintosh. Alle Softwaren sind sehr guenstig, es handelt sich dabei garantiert um originale, komplette und voellig funktionale Versionen. http://sartosoft.com* Windows XP Professional With SP2 Full Version: $59.95 * Office Enterprise 2007: $79.95 * Adobe Photoshop CS3 Extended: $79.95 * Office System Professional 2003 (5 Cds): $59.95 http://sartosoft.comBestellen Sie bei uns ohne Sorgen. Wir haben kompetente Supportmitarbeiter, die Ihnen bei der Installation weiterhelfen, wenn Sie Hilfe brauchen. Schnell und unverzueglich werden von uns Ihre Fragen beantwortet. Bei uns gibt es natuerlich auch Geld-Zurueck-Garantie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bart.vanassche at gmail.com Wed Mar 19 04:18:53 2008 From: bart.vanassche at gmail.com (Bart Van Assche) Date: Wed, 19 Mar 2008 12:18:53 +0100 Subject: [ofa-general] Socket Direct Protocol In-Reply-To: <000501c88916$6853d730$b137170a@amr.corp.intel.com> References: <000501c88916$6853d730$b137170a@amr.corp.intel.com> Message-ID: On Tue, Mar 18, 2008 at 5:37 PM, Sean Hefty wrote: > >I am considering to use the SDP protocol for some legacy applications. > >So I tried to find a standards document defining SDP. > > For InfiniBand, SDP is defined in Annex A4 of the InfiniBand Architecture Spec > Vol 1. Thanks for the information. Bart. From bryant at countrybumpkinshow.com Wed Mar 19 05:31:36 2008 From: bryant at countrybumpkinshow.com (Leonor Bergman) Date: Wed, 19 Mar 2008 13:31:36 +0100 Subject: [ofa-general] Kaufen Sie Softwareloesungen billiger Message-ID: <667228793.12405883274948@countrybumpkinshow.com> There is no cheaper source of original and perfectly working software.Unsere Software konnen Sie sofort downloaden. Einfach bezahlen und schon kann das runterladen beginnen. Unsere Programme funktionieren unter Windows und mit Macintosh und sind in allen europaeischen Sprachen verfugbar. Wir haben die guenstigsten Preise, aber es handelt sich natuerlich nur um Originalversionen, die voellig legal und im vollen Umfang sind. http://geocities.com/phillips.latoya/* Office Enterprise 2007: $79.95 * Adobe Acrobat 8.0 Professional: $69.95 * Adobe Photoshop CS3 Extended: $79.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/phillips.latoya/Sie werden nach dem Kauf nicht alleine gelassen. Unsere freundlichen Kundenberater werden Ihnen bei der Installation helfen, falls Sie es benotigen. Schnelle Antworten und Geld-Zurueck-Garantie sind bei uns selbstverstandlich. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjmagicden at aol.com Wed Mar 19 06:40:47 2008 From: bjmagicden at aol.com (Kaye Montgomery) Date: Wed, 19 Mar 2008 22:40:47 +0900 Subject: [ofa-general] {Viagra_onli2_de} Message-ID: <01c88a12$4616c180$72f94f3a@bjmagicden> Online Apotheke - original Qualitaet - 100% wirksam Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro - Kein peinlicher Arztbesuch erforderlich - keine versteckte Kosten - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - Bequem und diskret online bestellen. - Diskrete Verpackung und Zahlung - Kostenlose, arztliche Telefon-Beratung Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen Jetzt bestellen - und vier Pillen umsonst erhalten Man Lebt nur einmal - probiers aus ! (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From huggins at nvsoft.com Wed Mar 19 07:20:40 2008 From: huggins at nvsoft.com (Vance Stiles) Date: Wed, 19 Mar 2008 15:20:40 +0100 Subject: [ofa-general] Wir kuemmern uns fuer Sie um die besten Pillen Message-ID: <040453032.10057277107679@nvsoft.com> EuropeanPharmacy ist einer der wenigen Onlineapotheken, die sich um Ihre Kunden wirklich kuemmert. Diese Apotheke hat feste Positionen in diesem hart umkaempften Markt und sicherlich keine Eintagsfliege.Besuchen Sie unser Onlineshop, um die besten Angebote anzuschauen. http://ngaivv.breadunit.cn/?530753275823Produkte nur mit Lizenz. Vertraulicher und sicherer Kauf. Blitzschnelle Lieferung. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Wed Mar 19 07:35:04 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 19 Mar 2008 07:35:04 -0700 Subject: [ofa-general] [PATCH][TRIVIAL] OpenSM/osm_subnet.c: Cosmetic changes to options file Message-ID: <1205937304.11393.389.camel@hrosenstock-ws.xsigo.com> OpenSM/osm_subnet.c: Cosmetic changes to options file Signed-off-by: Hal Rosenstock diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index 903825b..b65cd74 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -1471,14 +1471,14 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) "# 5: 2.5 or 10.0 Gbps\n" "# 7: 2.5 or 5.0 or 10.0 Gbps\n" "# 2,4,6,8-14 Reserved\n" - "# Default 15: set to PortInfo:LinkSpeedSupported\n\n" + "# Default 15: set to PortInfo:LinkSpeedSupported\n" "force_link_speed %u\n\n" "# The subnet_timeout code that will be set for all the ports\n" "# The actual timeout is 4.096usec * 2^\n" "subnet_timeout %u\n\n" "# Threshold of local phy errors for sending Trap 129\n" "local_phy_errors_threshold 0x%02x\n\n" - "# Threshold of credits overrun errors for sending Trap 130\n" + "# Threshold of credit overrun errors for sending Trap 130\n" "overrun_errors_threshold 0x%02x\n\n", cl_ntoh64(p_opts->guid), cl_ntoh64(p_opts->m_key), @@ -1718,6 +1718,7 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) subn_dump_qos_options(opts_file, "QoS Router ports options", "qos_rtr", &p_opts->qos_rtr_options); + fprintf(opts_file, "\n"); fprintf(opts_file, "# Prefix routes file name\n" From dwsccyom at sccyo.org Wed Mar 19 07:36:39 2008 From: dwsccyom at sccyo.org (Cole Guerra) Date: Wed, 19 Mar 2008 15:36:39 +0100 Subject: [ofa-general] Get any soft you need without delays. Message-ID: <01c889d7$05e61d80$62ff744d@dwsccyom> Great site to purchase more than 270 programs! Even for Macintosh! Software in all European languages available! Cheap prices and original programs only! There are special offers and discounts for you to make even more significant savings. Buy software and be sure our professional customer support team will help to install it. Be also sure, if some problem occurs and your software does not run, we give your money back. You also will be able to do all the updates. http://geocities.com/willardpeck50 Incredible selection of programs and applications! From teesquaredd at btinternet.com Wed Mar 19 07:44:33 2008 From: teesquaredd at btinternet.com (Alyssa Vera) Date: Wed, 19 Mar 2008 15:44:33 +0100 Subject: [ofa-general] Melt away fat easily Message-ID: <01c889d8$206cc680$c37ae854@teesquaredd> Losing weight has never been so easy! You must know this secret! Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: as.zip Type: application/zip Size: 578 bytes Desc: not available URL: From rdreier at cisco.com Wed Mar 19 08:15:33 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 08:15:33 -0700 Subject: [ofa-general] itt member in iscsi_data? In-Reply-To: <47E0C828.8090102@voltaire.com> (Or Gerlitz's message of "Wed, 19 Mar 2008 10:00:40 +0200") References: <47E0C828.8090102@voltaire.com> Message-ID: > Erez is out of the office till the end of the month, so I suggest this > will wait for his return, fine? Sure. From ariston at gmail.com Wed Mar 19 08:17:08 2008 From: ariston at gmail.com (Ramachandra K) Date: Wed, 19 Mar 2008 20:47:08 +0530 Subject: [ofa-general] EQ creation in ConnectX Message-ID: <71d336490803190817lc9ccea0qc332c0456e5f304f@mail.gmail.com> Hi, I had a query related to EQ creation in the Connect X driver. Please correct me if I am wrong, but as I understand it, the driver has to do the following things for creating an EQ: 1. Allocate ICM space for EQ Context table 2. Allocate CMPT space for the EQ 3. Create an MTT entry to describe the physical address of the EQ buffer. 4. Use the SW2HW_EQ command to transfer ownership to hardware and passing the EQ context table entry to HW. I have a few questions related to these steps: 1. Is mtt_base_addr_h/l in the EQC entry used in SW2HW_EQ command, the offset of the MTT entry in the MTT made for this EQ describing the EQ buffer physical address ? 2. The Programmer's reference manual states that "The underlying MTTs are allocated and populated by the driver through the WRITE_MTT command". But I do not see a WRITE_MTT command being used in the Connect X driver. Is the EQ buffer information being directly written to the MTT table by the driver instead of using the WRITE_MTT command ? 3. Is the MTT entry just the physical address of the EQ buffer (and the present bit) or is there more significance to the MTT entry ? 4. Does the driver need do anything with the CMPT entries or are they completely handled by the hardware ? Regards, Ram From HNGUYEN at de.ibm.com Wed Mar 19 08:35:04 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 19 Mar 2008 16:35:04 +0100 Subject: [ofa-general] Re: [PATCH/RFC] IB/ehca: Make symbols used only in a single source file static In-Reply-To: Message-ID: > Allow the compiler to optimize better and generate smaller code: > > add/remove: 0/6 grow/shrink: 2/0 up/down: 1528/-1864 (-336) > function old new delta > .ehca_set_pagebuf 1344 2172 +828 > .ehca_probe 2312 3012 +700 > ehca_set_pagebuf_phys 24 - -24 > ehca_set_pagebuf_fmr 24 - -24 > ehca_init_device 24 - -24 > .ehca_set_pagebuf_fmr 480 - -480 > .ehca_set_pagebuf_phys 512 - -512 > .ehca_init_device 800 - -800 > > Also this fixes warnings like: > > drivers/infiniband/hw/ehca/ehca_mrmw.c:2015:5: warning: symbol > 'ehca_set_pagebuf_fmr' was not declared. Should it be static? > > Signed-off-by: Roland Dreier > --- Looks good to me. Much thanks! Nam From weiny2 at llnl.gov Wed Mar 19 09:36:49 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 19 Mar 2008 09:36:49 -0700 Subject: [ofa-general] [PATCH] Ensure ownership of the /etc/opensm directory Message-ID: <20080319093649.1b868acc.weiny2@llnl.gov> We found that the rpm generated for opensm did not "own" the directory specified by the --with-opensm-conf-sub-dir configure option. Also "ofa" was hard coded into the autoconf stuff. This patch makes sure that the rpm owns whatever directory was configured for the config files as well as putting the config file all together in the configured directory. Ira >From de8246f7e5501a1ad4de12f8d5938443f04e0e56 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 18 Mar 2008 19:08:18 -0700 Subject: [PATCH] Ensure ownership of the /etc/opensm directory. Also change references to ofa to be the option specified via the --with-opensm-conf-sub-dir Signed-off-by: Ira K. Weiny --- opensm/configure.in | 1 + opensm/opensm.spec.in | 7 ++++--- opensm/scripts/opensmd.in | 2 +- opensm/scripts/redhat-opensm.init.in | 4 ++-- 4 files changed, 8 insertions(+), 6 deletions(-) diff --git a/opensm/configure.in b/opensm/configure.in index 184c9f8..5bb87b4 100644 --- a/opensm/configure.in +++ b/opensm/configure.in @@ -98,6 +98,7 @@ AC_ARG_WITH(opensm-conf-sub-dir, esac ] ) AC_MSG_RESULT(${withopensmconfsubdir=no}) +AC_SUBST(OPENSM_CONF_SUB_DIR) dnl Set up /opensm config dir. CONF_DIR_TMP1="`eval echo ${sysconfdir}/$OPENSM_CONF_SUB_DIR`" diff --git a/opensm/opensm.spec.in b/opensm/opensm.spec.in index 6de6333..882e6e4 100644 --- a/opensm/opensm.spec.in +++ b/opensm/opensm.spec.in @@ -94,9 +94,9 @@ if [ -f /etc/redhat-release -o -s /etc/redhat-release ]; then else REDHAT="" fi -mkdir -p $etc/{init.d,ofa,logrotate.d} +mkdir -p $etc/{init.d, at OPENSM_CONF_SUB_DIR@,logrotate.d} install -m 755 scripts/${REDHAT}opensm.init $etc/init.d/opensmd -install -m 644 scripts/opensm.conf $etc/ofa/opensm.conf +install -m 644 scripts/opensm.conf $etc/@OPENSM_CONF_SUB_DIR@/opensm.conf install -m 644 scripts/opensm.logrotate $etc/logrotate.d/opensm install -m 755 scripts/sldd.sh $RPM_BUILD_ROOT%{_sbindir}/sldd.sh @@ -128,9 +128,10 @@ fi %doc AUTHORS COPYING README %{_sysconfdir}/init.d/opensmd %{_sbindir}/sldd.sh -%config(noreplace) %{_sysconfdir}/ofa/opensm.conf +%config(noreplace) %{_sysconfdir}/@OPENSM_CONF_SUB_DIR@/opensm.conf %config(noreplace) %{_sysconfdir}/logrotate.d/opensm %dir /var/cache/opensm +%dir %{_sysconfdir}/@OPENSM_CONF_SUB_DIR@ %files libs %defattr(-,root,root,-) diff --git a/opensm/scripts/opensmd.in b/opensm/scripts/opensmd.in index 0b150f7..23f50fa 100755 --- a/opensm/scripts/opensmd.in +++ b/opensm/scripts/opensmd.in @@ -34,7 +34,7 @@ prefix=@prefix@ exec_prefix=@exec_prefix@ -CONFIG=@sysconfdir@/ofa/opensm.conf +CONFIG=@sysconfdir@/@OPENSM_CONF_SUB_DIR@/opensm.conf if [ ! -f @CONFIG@ ]; then exit 0 diff --git a/opensm/scripts/redhat-opensm.init.in b/opensm/scripts/redhat-opensm.init.in index 4ce6605..689ffa0 100755 --- a/opensm/scripts/redhat-opensm.init.in +++ b/opensm/scripts/redhat-opensm.init.in @@ -38,7 +38,7 @@ # $Id: openib-1.0-opensm.init,v 1.5 2006/08/02 18:18:23 dledford Exp $ # # processname: @sbindir@/opensm -# config: @sysconfdir@/ofa/opensm.conf +# config: @sysconfdir@/@OPENSM_CONF_SUB_DIR@/opensm.conf # pidfile: /var/run/opensm.pid prefix=@prefix@ @@ -46,7 +46,7 @@ exec_prefix=@exec_prefix@ . /etc/rc.d/init.d/functions -CONFIG=@sysconfdir@/ofa/opensm.conf +CONFIG=@sysconfdir@/@OPENSM_CONF_SUB_DIR@/opensm.conf if [ ! -f $CONFIG ]; then exit 0 fi -- 1.5.1 From kdjynawnv at bmaccountants.com Wed Mar 19 10:05:59 2008 From: kdjynawnv at bmaccountants.com (Sally Turner) Date: Wed, 19 Mar 2008 17:05:59 +0000 Subject: [ofa-general] You ever try it? Message-ID: <01c889e3$81021d00$05d43fd5@kdjynawnv> Give yourself the edge over the other guys Find more and link to our ONLINE STORE in attached details. -------------- next part -------------- A non-text attachment was scrubbed... Name: p.zip Type: application/zip Size: 270 bytes Desc: not available URL: From michaelc at cs.wisc.edu Wed Mar 19 10:20:05 2008 From: michaelc at cs.wisc.edu (Mike Christie) Date: Wed, 19 Mar 2008 12:20:05 -0500 Subject: [ofa-general] Re: itt member in iscsi_data? In-Reply-To: References: Message-ID: <47E14B45.9040509@cs.wisc.edu> Roland Dreier wrote: > iser_initiator.c has this code: > > itt = ntohl(hdr->itt); > > which triggers sparse warnings like: > > drivers/infiniband/ulp/iser/iser_initiator.c:419:8: warning: cast to restricted type > drivers/infiniband/ulp/iser/iser_initiator.c:419:8: warning: cast from restricted type > > and indeed the itt field is declared as itt_t, with no particular > endianness, so the sparse warnings seem like they are pointing at > something that really is suspicious. > > It seems the only use of itt in the function that does ntohl() is to > print the value our in debugging statements. What is intended here? > Would it just make sense to for the debug statements to print hdr->itt > out directly with no conversion? > Sorry I missed this mail. I think it will be ok to just print it without the ntohl, because of how libiscsi and ib_iser use the tag. The iscsi RFC defines it as a big endien field, but as you probably saw we have that wierd encoding and Al Viro did that code to define it as a itt_t I can send a patch when Erez gets back. I have some other patches for him to look over for the next window. From rdreier at cisco.com Wed Mar 19 10:28:06 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 10:28:06 -0700 Subject: [ofa-general] infiniband sparc64 build In-Reply-To: <20080318230202.dd47368c.akpm@linux-foundation.org> (Andrew Morton's message of "Tue, 18 Mar 2008 23:02:02 -0700") References: <20080318230202.dd47368c.akpm@linux-foundation.org> Message-ID: > drivers/infiniband/hw/nes/nes_verbs.c:1047: warning: cast to pointer from integer of different size Thanks... an amusing printk problem I hadn't seen before, namely that dma_addr_t and void* don't have the same size sometimes. Should be fixed by the patch below (which will appear in your next for-mm pull): RDMA/nes: Use proper format and cast to print dma_addr_t On some platforms, eg sparc64, dma_addr_t is not the same size as a pointer, so printing dma_addr_t values by casting to void * and using a %p format generates warnings. Fix this by casting to unsigned long and using %lx instead. This fixes the warnings: drivers/infiniband/hw/nes/nes_verbs.c: In function 'nes_setup_virt_qp': drivers/infiniband/hw/nes/nes_verbs.c:1047: warning: cast to pointer from integer of different size drivers/infiniband/hw/nes/nes_verbs.c:1078: warning: cast to pointer from integer of different size drivers/infiniband/hw/nes/nes_verbs.c:1078: warning: cast to pointer from integer of different size drivers/infiniband/hw/nes/nes_verbs.c: In function 'nes_reg_user_mr': drivers/infiniband/hw/nes/nes_verbs.c:2657: warning: cast to pointer from integer of different size Reported by Andrew Morton . Signed-off-by: Roland Dreier --- drivers/infiniband/hw/nes/nes_verbs.c | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c index d755681..46b3b8e 100644 --- a/drivers/infiniband/hw/nes/nes_verbs.c +++ b/drivers/infiniband/hw/nes/nes_verbs.c @@ -1044,10 +1044,10 @@ static int nes_setup_virt_qp(struct nes_qp *nesqp, struct nes_pbl *nespbl, u8 sq_pbl_entries; pbl_entries = nespbl->pbl_size >> 3; - nes_debug(NES_DBG_QP, "Userspace PBL, pbl_size=%u, pbl_entries = %d pbl_vbase=%p, pbl_pbase=%p\n", + nes_debug(NES_DBG_QP, "Userspace PBL, pbl_size=%u, pbl_entries = %d pbl_vbase=%p, pbl_pbase=%lx\n", nespbl->pbl_size, pbl_entries, (void *)nespbl->pbl_vbase, - (void *)nespbl->pbl_pbase); + (unsigned long) nespbl->pbl_pbase); pbl = (__le64 *) nespbl->pbl_vbase; /* points to first pbl entry */ /* now lets set the sq_vbase as well as rq_vbase addrs we will assign */ /* the first pbl to be fro the rq_vbase... */ @@ -1075,9 +1075,9 @@ static int nes_setup_virt_qp(struct nes_qp *nesqp, struct nes_pbl *nespbl, /* nesqp->hwqp.rq_vbase = bus_to_virt(*pbl); */ /*nesqp->hwqp.rq_vbase = phys_to_virt(*pbl); */ - nes_debug(NES_DBG_QP, "QP sq_vbase= %p sq_pbase=%p rq_vbase=%p rq_pbase=%p\n", - nesqp->hwqp.sq_vbase, (void *)nesqp->hwqp.sq_pbase, - nesqp->hwqp.rq_vbase, (void *)nesqp->hwqp.rq_pbase); + nes_debug(NES_DBG_QP, "QP sq_vbase= %p sq_pbase=%lx rq_vbase=%p rq_pbase=%lx\n", + nesqp->hwqp.sq_vbase, (unsigned long) nesqp->hwqp.sq_pbase, + nesqp->hwqp.rq_vbase, (unsigned long) nesqp->hwqp.rq_pbase); spin_lock_irqsave(&nesadapter->pbl_lock, flags); if (!nesadapter->free_256pbl) { pci_free_consistent(nesdev->pcidev, nespbl->pbl_size, nespbl->pbl_vbase, @@ -2654,10 +2654,10 @@ static struct ib_mr *nes_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, nespbl->pbl_vbase = (u64 *)pbl; nespbl->user_base = start; - nes_debug(NES_DBG_MR, "Allocated PBL memory, %u bytes, pbl_pbase=%p," + nes_debug(NES_DBG_MR, "Allocated PBL memory, %u bytes, pbl_pbase=%lx," " pbl_vbase=%p user_base=0x%lx\n", - nespbl->pbl_size, (void *)nespbl->pbl_pbase, - (void*)nespbl->pbl_vbase, nespbl->user_base); + nespbl->pbl_size, (unsigned long) nespbl->pbl_pbase, + (void *) nespbl->pbl_vbase, nespbl->user_base); list_for_each_entry(chunk, ®ion->chunk_list, list) { for (nmap_index = 0; nmap_index < chunk->nmap; ++nmap_index) { -- 1.5.4.3 From rdreier at cisco.com Wed Mar 19 10:29:27 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 10:29:27 -0700 Subject: [ofa-general] Re: itt member in iscsi_data? In-Reply-To: <47E14B45.9040509@cs.wisc.edu> (Mike Christie's message of "Wed, 19 Mar 2008 12:20:05 -0500") References: <47E14B45.9040509@cs.wisc.edu> Message-ID: > I can send a patch when Erez gets back. I have some other patches for > him to look over for the next window. OK, no hurry... just something I noticed in my big warning spring cleaning frenzy. - R. From rdreier at cisco.com Wed Mar 19 10:57:31 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 10:57:31 -0700 Subject: [ofa-general] Re: [PATCH 3/6] IB/ipath - provide I/O bus speeds for diagnostic purposes In-Reply-To: <20080319002356.19088.66871.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Tue, 18 Mar 2008 17:23:56 -0700") References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> <20080319002356.19088.66871.stgit@eng-46.mv.qlogic.com> Message-ID: thanks, applied 1-3 From rdreier at cisco.com Wed Mar 19 11:00:10 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 11:00:10 -0700 Subject: [ofa-general] Re: [PATCH 6/6] IB/ipath - shared context code needs to be sure device is usable In-Reply-To: <20080319002411.19088.72997.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Tue, 18 Mar 2008 17:24:11 -0700") References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> <20080319002411.19088.72997.stgit@eng-46.mv.qlogic.com> Message-ID: thanks, applied From rdreier at cisco.com Wed Mar 19 11:00:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 11:00:05 -0700 Subject: [ofa-general] [PATCH 1/6] IB/ipath - misc sparse cleanup In-Reply-To: <20080319002346.19088.3336.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Tue, 18 Mar 2008 17:23:46 -0700") References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> <20080319002346.19088.3336.stgit@eng-46.mv.qlogic.com> Message-ID: OK... still have endianness problems (build with C=2 CF=-D__CHECK_ENDIAN__ to see the warnings) around line 837 of drivers/infiniband/hw/ipath/ipath_intr.c and also the bug of pci_write_config_byte(pdev, link_off, linkctrl & (0xf << 8)); (&ing wiht 0xf << 8 means writing a byte always writes 0) at line 849 of drivers/infiniband/hw/ipath/ipath_iba6110.c that generates a compiler warning From rdreier at cisco.com Wed Mar 19 11:00:41 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 11:00:41 -0700 Subject: [ofa-general] [PATCH 4/6] IB/ipath - Enable use of 4KB MTU via module paramater In-Reply-To: <20080319002401.19088.29911.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Tue, 18 Mar 2008 17:24:01 -0700") References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> <20080319002401.19088.29911.stgit@eng-46.mv.qlogic.com> Message-ID: > This patch enables use of 4KB MTUs via module paramater. > +module_param_named(mtu4096, ipath_mtu4096, uint, S_IRUGO); > +MODULE_PARM_DESC(mtu4096, "enable MTU of 4096 bytes, if supported"); When would a user ever want to set mtu4096 to 0? - R. From rdreier at cisco.com Wed Mar 19 11:01:18 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 11:01:18 -0700 Subject: [ofa-general] Re: [PATCH 5/6] IB/ipath - fix a minor code style violation In-Reply-To: <20080319002406.19088.25328.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Tue, 18 Mar 2008 17:24:06 -0700") References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> <20080319002406.19088.25328.stgit@eng-46.mv.qlogic.com> Message-ID: > - return 0; > + goto bail; Changing a simple return 0 to a goto where I have to study the code to understand that ret == 0 seems like a step in the wrong direction to me. From educ.empresarial at yahoo.com.br Wed Mar 19 11:01:40 2008 From: educ.empresarial at yahoo.com.br (K.L.A Eventos Empresarias) Date: Wed, 19 Mar 2008 18:01:40 GMT Subject: [ofa-general] 21 DVDs Leve os Melhores para sua Empresa Message-ID: An HTML attachment was scrubbed... URL: From milan.madzia at wing.sk Wed Mar 19 11:02:43 2008 From: milan.madzia at wing.sk (Lizabeth Clay) Date: Wed, 19 Mar 2008 19:02:43 +0100 Subject: [ofa-general] Hier bekommen Sie die besten Pillen Message-ID: <01c889f3$cf6acb80$0f93204f@milan.madzia> Wir sind froh, Ihnen die Apotheke EuropeanPharmacy vorstellen zu koennen. Es ist eine vertrauliche und ganz professionelle Quelle, um die hochwertigsten Medikamente ganz guenstig zu besorgen. Es wird auch sehr viel zur Auswahl angeboten. Das beste, was die moderne Medizin Ihnen anbieten kann. http://ndmujo.duringwing.cn/?355235813087Bringen Sie sich Ihre Gesundheit zurueck. Vollkommen anonymer und sicherer Kauf. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Wed Mar 19 11:24:51 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 19 Mar 2008 18:24:51 +0000 Subject: [ofa-general] Re: [PATCH] OpenSM release notes: Clarify QoS firmware support In-Reply-To: <1205849964.11393.152.camel@hrosenstock-ws.xsigo.com> References: <1205849964.11393.152.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080319182451.GH11085@sashak.voltaire.com> On 07:19 Tue 18 Mar , Hal Rosenstock wrote: > OpenSM release notes: Clarify Mellanox firmware support requirements for > QoS > > Also, some other cosmetic changes > > Please apply to master and OFED 1.3. > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Wed Mar 19 11:26:32 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 19 Mar 2008 18:26:32 +0000 Subject: [ofa-general] Re: [PATCH][TRIVIAL] OpenSM/osm_subnet.c: Cosmetic changes to options file In-Reply-To: <1205937304.11393.389.camel@hrosenstock-ws.xsigo.com> References: <1205937304.11393.389.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080319182632.GI11085@sashak.voltaire.com> On 07:35 Wed 19 Mar , Hal Rosenstock wrote: > OpenSM/osm_subnet.c: Cosmetic changes to options file > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Wed Mar 19 11:29:31 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 19 Mar 2008 18:29:31 +0000 Subject: [ofa-general] Re: [PATCH] Ensure ownership of the /etc/opensm directory In-Reply-To: <20080319093649.1b868acc.weiny2@llnl.gov> References: <20080319093649.1b868acc.weiny2@llnl.gov> Message-ID: <20080319182931.GJ11085@sashak.voltaire.com> On 09:36 Wed 19 Mar , Ira Weiny wrote: > We found that the rpm generated for opensm did not "own" the directory > specified by the --with-opensm-conf-sub-dir configure option. Also "ofa" was > hard coded into the autoconf stuff. This patch makes sure that the rpm owns > whatever directory was configured for the config files as well as putting the > config file all together in the configured directory. > > Ira > > From de8246f7e5501a1ad4de12f8d5938443f04e0e56 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 18 Mar 2008 19:08:18 -0700 > Subject: [PATCH] Ensure ownership of the /etc/opensm directory. > Also change references to ofa to be the option specified via the > --with-opensm-conf-sub-dir Nice thing. > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From tehornq at wsprings.com Wed Mar 19 11:36:19 2008 From: tehornq at wsprings.com (Federico Carlisle) Date: Wed, 19 Mar 2008 19:36:19 +0100 Subject: [ofa-general] On top all night Message-ID: <01c889f8$810bfb80$3e493b52@tehornq> Be like sex machine Please look attached file and know MORE ABOUT THIS! -------------- next part -------------- A non-text attachment was scrubbed... Name: vv.zip Type: application/zip Size: 636 bytes Desc: not available URL: From ralph.campbell at qlogic.com Wed Mar 19 11:42:29 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Wed, 19 Mar 2008 11:42:29 -0700 Subject: [ofa-general] Re: [PATCH 5/6] IB/ipath - fix a minor code style violation In-Reply-To: References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> <20080319002406.19088.25328.stgit@eng-46.mv.qlogic.com> Message-ID: <1205952149.5109.1019.camel@brick.pathscale.com> On Wed, 2008-03-19 at 11:01 -0700, Roland Dreier wrote: > > - return 0; > > + goto bail; > > Changing a simple return 0 to a goto where I have to study the code to > understand that ret == 0 seems like a step in the wrong direction to me. No problem. I withdraw this one. From ralph.campbell at qlogic.com Wed Mar 19 11:52:05 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Wed, 19 Mar 2008 11:52:05 -0700 Subject: [ofa-general] [PATCH 4/6] IB/ipath - Enable use of 4KB MTU via module paramater In-Reply-To: References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> <20080319002401.19088.29911.stgit@eng-46.mv.qlogic.com> Message-ID: <1205952725.5109.1025.camel@brick.pathscale.com> On Wed, 2008-03-19 at 11:00 -0700, Roland Dreier wrote: > > This patch enables use of 4KB MTUs via module paramater. > > > +module_param_named(mtu4096, ipath_mtu4096, uint, S_IRUGO); > > +MODULE_PARM_DESC(mtu4096, "enable MTU of 4096 bytes, if supported"); > > When would a user ever want to set mtu4096 to 0? > > - R. The driver uses more pinned memory for receive buffers when the parameter is set whether or not the switch/fabric supports 4K MTU so it seemed reasonable to allow a smaller memory footprint. From genne at gmx.de Wed Mar 19 11:58:04 2008 From: genne at gmx.de (Tad Caldwell) Date: Wed, 19 Mar 2008 15:58:04 -0300 Subject: [ofa-general] {Viagra_onli2_de} Message-ID: <01c889da$03d18e00$4bc00ebd@genne> Online Apotheke - original Qualitaet - 100% wirksam Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro - Kein peinlicher Arztbesuch erforderlich - keine versteckte Kosten - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - Bequem und diskret online bestellen. - Diskrete Verpackung und Zahlung - Kostenlose, arztliche Telefon-Beratung Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen Jetzt bestellen - und vier Pillen umsonst erhalten http://yearnumeral.com (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From surs at us.ibm.com Wed Mar 19 11:54:33 2008 From: surs at us.ibm.com (Sayantan Sur) Date: Wed, 19 Mar 2008 14:54:33 -0400 Subject: [ofa-general] [p2s2-announce] UPDATED CFP: Workshop on Parallel Programming Models and Systems Software for High-end Computing (P2S2) Message-ID: CALL FOR PAPERS =============== First International Workshop on Parallel Programming Models and Systems Software for High-end Computing (P2S2) (http://www.mcs.anl.gov/events/workshops/p2s2) Sep. 8th, 2008 To be held in conjunction with ICPP-08: The 27th International Conference on Parallel Processing Sep. 8-12, 2008 Portland, Oregon, USA SCOPE ----- The goal of this workshop is to bring together researchers and practitioners in parallel programming models and systems software for high-end computing systems. Please join us in a discussion of new ideas, experiences, and the latest trends in these areas at the workshop. TOPICS OF INTEREST ------------------ The focus areas for this workshop include, but are not limited to: * Programming models and their high-performance implementations o MPI, Sockets, OpenMP, Global Arrays, X10, UPC, Chapel o Other Hybrid Programming Models * Systems software for scientific and enterprise computing o Communication sub-subsystems for high-end computing o High-performance File and storage systems o Fault-tolerance techniques and implementations o Efficient and high-performance virtualization and other management mechanisms * Tools for Management, Maintenance, Coordination and Synchronization o Software for Enterprise Data-centers using Modern Architectures o Job scheduling libraries o Management libraries for large-scale system o Toolkits for process and task coordination on modern platforms * Performance evaluation, analysis and modeling of emerging computing platforms PROCEEDINGS ----------- Proceedings of this workshop will be published by the IEEE Computer Society (together with the ICPP conference proceedings) in CD format only and will be available at the conference. SUBMISSION INSTRUCTIONS ----------------------- Submissions should be in PDF format in U.S. Letter size paper. They should not exceed 8 pages (all inclusive). Submissions will be judged based on relevance, significance, originality, correctness and clarity. DATES AND DEADLINES ------------------- Paper Submission: April 11th, 2008 Author Notification: May 20th, 2008 Camera Ready: June 2nd, 2008 PROGRAM CHAIRS -------------- * Pavan Balaji (Argonne National Laboratory) * Sayantan Sur (IBM Research) STEERING COMMITTEE ------------------ * William D. Gropp (University of Illinois Urbana-Champaign) * Dhabaleswar K. Panda (Ohio State University) * Vijay Saraswat (IBM Research) PROGRAM COMMITTEE ----------------- * David Bernholdt (Oak Ridge National Laboratory) * Ron Brightwell (Sandia National Laboratory) * Wu-chun Feng (Virginia Tech) * Richard Graham (Oak Ridge National Laboratory) * Hyun-wook Jin (Konkuk University, South Korea) * Sameer Kumar (IBM Research) * Doug Lea (State University of New York at Oswego) * Jarek Nieplocha (Pacific Northwest National Laboratory) * Scott Pakin (Los Alamos National Laboratory) * Vivek Sarkar (Rice University) * Rajeev Thakur (Argonne National Laboratory) * Pete Wyckoff (Ohio Supercomputing Center) If you have any questions, please contact us at p2s2-chairs at mcs.anl.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Wed Mar 19 13:49:05 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 13:49:05 -0700 Subject: [ofa-general] [PATCH 4/6] IB/ipath - Enable use of 4KB MTU via module paramater In-Reply-To: <1205952725.5109.1025.camel@brick.pathscale.com> (Ralph Campbell's message of "Wed, 19 Mar 2008 11:52:05 -0700") References: <20080319002340.19088.64620.stgit@eng-46.mv.qlogic.com> <20080319002401.19088.29911.stgit@eng-46.mv.qlogic.com> <1205952725.5109.1025.camel@brick.pathscale.com> Message-ID: > The driver uses more pinned memory for receive buffers when the > parameter is set whether or not the switch/fabric supports 4K > MTU so it seemed reasonable to allow a smaller memory footprint. OK, fair enough. I added that rationale to the changelog and applied the patch. - R. From rdreier at cisco.com Wed Mar 19 13:54:21 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 13:54:21 -0700 Subject: [ofa-general] Re: [PATCH] IB/srp: enforce protocol limit on srp_sg_tablesize In-Reply-To: <1205782965.10010.30.camel@lap75545.ornl.gov> (David Dillow's message of "Mon, 17 Mar 2008 15:42:45 -0400") References: <1205777880.10010.26.camel@lap75545.ornl.gov> <1205782965.10010.30.camel@lap75545.ornl.gov> Message-ID: thanks, applied From rdreier at cisco.com Wed Mar 19 14:05:10 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 14:05:10 -0700 Subject: [ofa-general] Re: [PATCH 3/3] Infiniband: don't use task_struct.tgid in make_cm_node() In-Reply-To: <47DE692F.5010600@openvz.org> (Pavel Emelyanov's message of "Mon, 17 Mar 2008 15:50:55 +0300") References: <47DE692F.5010600@openvz.org> Message-ID: > /* get a unique session ID , add thread_id to an upcounter to handle race */ > atomic_inc(&cm_core->node_cnt); > atomic_inc(&cm_core->session_id); > - cm_node->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); > + cm_node->session_id = (u32)(atomic_read(&cm_core->session_id) + > + task_tgid_nr(current)); Again the change of current->tgid to task_tgid_nr() looks fine as far as cleaning up the existing code, but again I think we should be able to get rid of this completely... First, the use of the tgid seems to be there to protect against a race like the following where the same value gets used twice: atomic_inc() atomic_inc() atomic_read() atomic_read() but of course it does leave another possibility of an unlucky pair of tgid values that differ by one and get a collision anyway. And it seems like this can be handled more cleanly with atomic_inc_return(), and not try to use tgid at all. Second, I don't see anywhere that actually uses the session_id field after it gets set, so can we just delete it completely? The patch below compiles and links fine for me, so is the simplest fix? (I don't think hardware is looking at session_id behind our back, is it?) diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c index c1a85f3..31a28e9 100644 --- a/drivers/infiniband/hw/nes/nes_cm.c +++ b/drivers/infiniband/hw/nes/nes_cm.c @@ -365,7 +365,6 @@ static void print_core(struct nes_cm_core *core) if (!core) return; nes_debug(NES_DBG_CM, "---------------------------------------------\n"); - nes_debug(NES_DBG_CM, "Session ID : %u \n", atomic_read(&core->session_id)); nes_debug(NES_DBG_CM, "State : %u \n", core->state); @@ -1100,8 +1099,6 @@ static struct nes_cm_node *make_cm_node(struct nes_cm_core *cm_core, cm_node->tcp_cntxt.rcv_nxt = 0; /* get a unique session ID , add thread_id to an upcounter to handle race */ atomic_inc(&cm_core->node_cnt); - atomic_inc(&cm_core->session_id); - cm_node->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); cm_node->conn_type = cm_info->conn_type; cm_node->apbvt_set = 0; cm_node->accept_pend = 0; @@ -1628,9 +1625,7 @@ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core, listener->cm_core = cm_core; listener->nesvnic = nesvnic; atomic_inc(&cm_core->node_cnt); - atomic_inc(&cm_core->session_id); - listener->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); listener->conn_type = cm_info->conn_type; listener->backlog = cm_info->backlog; listener->listener_state = NES_CM_LISTENER_ACTIVE_STATE; @@ -1943,7 +1938,6 @@ static struct nes_cm_core *nes_cm_alloc_core(void) cm_core->state = NES_CM_STATE_INITED; cm_core->free_tx_pkt_max = NES_CM_DEFAULT_FREE_PKTS; - atomic_set(&cm_core->session_id, 0); atomic_set(&cm_core->events_posted, 0); /* init the packet lists */ diff --git a/drivers/infiniband/hw/nes/nes_cm.h b/drivers/infiniband/hw/nes/nes_cm.h index 980fb67..7717cb2 100644 --- a/drivers/infiniband/hw/nes/nes_cm.h +++ b/drivers/infiniband/hw/nes/nes_cm.h @@ -225,7 +225,6 @@ enum nes_cm_listener_state { struct nes_cm_listener { struct list_head list; - u64 session_id; struct nes_cm_core *cm_core; u8 loc_mac[ETH_ALEN]; nes_addr_t loc_addr; @@ -242,7 +241,6 @@ struct nes_cm_listener { /* per connection node and node state information */ struct nes_cm_node { - u64 session_id; u32 hashkey; nes_addr_t loc_addr, rem_addr; @@ -327,7 +325,6 @@ struct nes_cm_event { struct nes_cm_core { enum nes_cm_node_state state; - atomic_t session_id; atomic_t listen_node_cnt; struct nes_cm_node listen_list; From clameter at sgi.com Wed Mar 19 14:27:06 2008 From: clameter at sgi.com (Christoph Lameter) Date: Wed, 19 Mar 2008 14:27:06 -0700 (PDT) Subject: [ofa-general] Re: [PATCH] 4/4 i_mmap_lock spinlock2rwsem (#v9 was 1/4) In-Reply-To: <20080307155244.GF24114@v2.random> References: <20080303213707.GA8091@v2.random> <20080303220502.GA5301@v2.random> <47CC9B57.5050402@qumranet.com> <20080304133020.GC5301@v2.random> <20080304222030.GB8951@v2.random> <20080307151722.GD24114@v2.random> <20080307152328.GE24114@v2.random> <20080307155244.GF24114@v2.random> Message-ID: You need this patch to address the issues (that I already mentioned when I sent the patch to you). New EMM notifier patch with sleeping coming soon. From: Christoph Lameter Subject: Move tlb flushing into free_pgtables Move the tlb flushing into free_pgtables. The conversion of the locks taken for reverse map scanning would require taking sleeping locks in free_pgtables. Moving the tlb flushing into free_pgtables allows sleeping in part of free_pgtables(). Signed-off-by: Christoph Lameter --- include/linux/mm.h | 4 ++-- mm/memory.c | 14 ++++++++++---- mm/mmap.c | 6 +++--- 3 files changed, 15 insertions(+), 9 deletions(-) Index: linux-2.6/include/linux/mm.h =================================================================== --- linux-2.6.orig/include/linux/mm.h 2008-03-19 13:30:51.460856986 -0700 +++ linux-2.6/include/linux/mm.h 2008-03-19 13:31:20.809377398 -0700 @@ -751,8 +751,8 @@ int walk_page_range(const struct mm_stru void *private); void free_pgd_range(struct mmu_gather **tlb, unsigned long addr, unsigned long end, unsigned long floor, unsigned long ceiling); -void free_pgtables(struct mmu_gather **tlb, struct vm_area_struct *start_vma, - unsigned long floor, unsigned long ceiling); +void free_pgtables(struct vm_area_struct *start_vma, unsigned long floor, + unsigned long ceiling); int copy_page_range(struct mm_struct *dst, struct mm_struct *src, struct vm_area_struct *vma); void unmap_mapping_range(struct address_space *mapping, Index: linux-2.6/mm/memory.c =================================================================== --- linux-2.6.orig/mm/memory.c 2008-03-19 13:29:06.007351495 -0700 +++ linux-2.6/mm/memory.c 2008-03-19 13:46:31.352774359 -0700 @@ -271,9 +271,11 @@ void free_pgd_range(struct mmu_gather ** } while (pgd++, addr = next, addr != end); } -void free_pgtables(struct mmu_gather **tlb, struct vm_area_struct *vma, - unsigned long floor, unsigned long ceiling) +void free_pgtables(struct vm_area_struct *vma, unsigned long floor, + unsigned long ceiling) { + struct mmu_gather *tlb; + while (vma) { struct vm_area_struct *next = vma->vm_next; unsigned long addr = vma->vm_start; @@ -285,8 +287,10 @@ void free_pgtables(struct mmu_gather **t unlink_file_vma(vma); if (is_vm_hugetlb_page(vma)) { - hugetlb_free_pgd_range(tlb, addr, vma->vm_end, + tlb = tlb_gather_mmu(vma->vm_mm, 0); + hugetlb_free_pgd_range(&tlb, addr, vma->vm_end, floor, next? next->vm_start: ceiling); + tlb_finish_mmu(tlb, addr, vma->vm_end); } else { /* * Optimization: gather nearby vmas into one call down @@ -298,8 +302,10 @@ void free_pgtables(struct mmu_gather **t anon_vma_unlink(vma); unlink_file_vma(vma); } - free_pgd_range(tlb, addr, vma->vm_end, + tlb = tlb_gather_mmu(vma->vm_mm, 0); + free_pgd_range(&tlb, addr, vma->vm_end, floor, next? next->vm_start: ceiling); + tlb_finish_mmu(tlb, addr, vma->vm_end); } vma = next; } Index: linux-2.6/mm/mmap.c =================================================================== --- linux-2.6.orig/mm/mmap.c 2008-03-19 13:29:48.659889667 -0700 +++ linux-2.6/mm/mmap.c 2008-03-19 13:30:36.296604891 -0700 @@ -1750,9 +1750,9 @@ static void unmap_region(struct mm_struc update_hiwater_rss(mm); unmap_vmas(&tlb, vma, start, end, &nr_accounted, NULL); vm_unacct_memory(nr_accounted); - free_pgtables(&tlb, vma, prev? prev->vm_end: FIRST_USER_ADDRESS, - next? next->vm_start: 0); tlb_finish_mmu(tlb, start, end); + free_pgtables(vma, prev? prev->vm_end: FIRST_USER_ADDRESS, + next? next->vm_start: 0); emm_notify(mm, emm_invalidate_end, start, end); } @@ -2049,8 +2049,8 @@ void exit_mmap(struct mm_struct *mm) /* Use -1 here to ensure all VMAs in the mm are unmapped */ end = unmap_vmas(&tlb, vma, 0, -1, &nr_accounted, NULL); vm_unacct_memory(nr_accounted); - free_pgtables(&tlb, vma, FIRST_USER_ADDRESS, 0); tlb_finish_mmu(tlb, 0, end); + free_pgtables(vma, FIRST_USER_ADDRESS, 0); /* * Walk the list again, actually closing and freeing it, From pawel.dziekonski at wcss.pl Wed Mar 19 15:05:31 2008 From: pawel.dziekonski at wcss.pl (Pawel Dziekonski) Date: Wed, 19 Mar 2008 23:05:31 +0100 Subject: [ofa-general] IB/OFED training courses? Message-ID: <20080319220531.GA23061@cefeid.wcss.wroc.pl> hi, I'm looking for good training courses about IB architecture, diagnostics, performance tuning and OFED. anywhere in Europe. my hardware is Mellanox-based. any hints very appreciated! Pawel -- Pawel Dziekonski Wroclaw Centre for Networking & Supercomputing, HPC Department Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl From Golden-01 at Mail.ru Wed Mar 19 15:12:10 2008 From: Golden-01 at Mail.ru () Date: Thu, 20 Mar 2008 01:12:10 +0300 Subject: [ofa-general] =?iso-8859-1?q?=C2=E0=F8_=E4=EE=EF=EE=EB=ED=E8=F2?= =?iso-8859-1?q?=E5=EB=FC=ED=FB=E9_=E7=E0=F0=E0=E1=EE=F2=EE=EA=2C?= =?iso-8859-1?q?=E0_=E2=EE=E7=EC=EE=E6=ED=EE_=E8_=EE=E1=E5=F1=EF=E5?= =?iso-8859-1?q?=F7=E5=ED=ED=EE=E5_=E1=F3=E4=F3=F9=E5=E5=2E?= Message-ID: <200803200112.QDUZVVVJEG@82.138.52.149>                                                                                Добрый день !      Извините,что,возможно ,отнимаю у Вас время,но если Вы имеете  доступ к Интернету,умеете пользоваться  мышкой,   имеете 2-3 часа свободного времени,и Вас интересуют деньги,то всего один щелчок мышкой может отделять Вас от состояния !!! Еще раз простите за вторжение.Желаю Вам удачного дня . Жду Вашего ответа : Moskvich-V at Mail.ru  или  Golden-V at Yandex.ru Удачи Вам !!! Информация находиться в прикрепленном   файле  .... Не беспокойтесь, это не вирус,но если Вы ,тем не менее,  опасаетесь  его  открыть  ,не  спешите  его  удалять ,вовсе не сложно проверить почту при помощи антивируса. Это не реклама, а действительно выгодное предложение !!! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Golden Stream.doc Type: application/octet-stream Size: 160256 bytes Desc: not available URL: From rdreier at cisco.com Wed Mar 19 15:14:28 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 15:14:28 -0700 Subject: [ofa-general] [PATCH] libibverbs/man: add support to the new parameter that control the SL In-Reply-To: <200802251616.39251.dotanb@dev.mellanox.co.il> (Dotan Barak's message of "Mon, 25 Feb 2008 16:16:39 +0200") References: <200802251616.39251.dotanb@dev.mellanox.co.il> Message-ID: thanks, applied along with the actual patch to add SL options. From rdreier at cisco.com Wed Mar 19 16:06:26 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 16:06:26 -0700 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: <1205767427.25950.137.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:47 +0200") References: <1205767427.25950.137.camel@mtls03> Message-ID: > @@ -505,6 +509,7 @@ struct ib_qp_init_attr { > enum ib_sig_type sq_sig_type; > enum ib_qp_type qp_type; > u8 port_num; /* special QP types only */ > + enum qp_create_flags create_flags; > }; I'm dubious about this. It seems to me like everything (including the mlx4 low-level driver changes for LSO) would be simpler to implement and use if we just extend the qp_type to include IB_QPT_UD_LSO. For example it eliminates the need to test if someone tries to create an LSO QP for a non-UD transport and return an error (although I don't see that test in your code anyway). As I recall, the XRC patches want this flags field too. But does it work there if we just add another XRC QP type too? - R. From rdreier at cisco.com Wed Mar 19 16:06:26 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 16:06:26 -0700 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: <1205767427.25950.137.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:47 +0200") References: <1205767427.25950.137.camel@mtls03> Message-ID: > @@ -505,6 +509,7 @@ struct ib_qp_init_attr { > enum ib_sig_type sq_sig_type; > enum ib_qp_type qp_type; > u8 port_num; /* special QP types only */ > + enum qp_create_flags create_flags; > }; I'm dubious about this. It seems to me like everything (including the mlx4 low-level driver changes for LSO) would be simpler to implement and use if we just extend the qp_type to include IB_QPT_UD_LSO. For example it eliminates the need to test if someone tries to create an LSO QP for a non-UD transport and return an error (although I don't see that test in your code anyway). As I recall, the XRC patches want this flags field too. But does it work there if we just add another XRC QP type too? - R. From Ignite at InvestingIdeas.prserv.net Wed Mar 19 18:53:04 2008 From: Ignite at InvestingIdeas.prserv.net (Investing Ideas) Date: Wed, 19 Mar 2008 20:53:04 -0500 Subject: [ofa-general] Gold is Risky - Green is a solid investment Message-ID: An HTML attachment was scrubbed... URL: From rdreier at cisco.com Wed Mar 19 20:33:24 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 20:33:24 -0700 Subject: [ofa-general] [PATCH/RFC 1/2] IB/core: Add support for "send with invalidate" work requests Message-ID: Add an IB_SEND_INVALIDATE send flag that can be used to mark a "send with invalidate" work request as defined in the iWARP verbs and the InfiniBand base memory management extensions. Also add a new "invalidate" structure to the struct ib_send_wr.wr union that can be used to pass in an R_Key/STag to be invalidated. Add this new structure to struct ib_uverbs_send_wr and add code to copy the rkey field in ib_uverbs_post_send(). Also, move the existing IB_DEVICE_SEND_W_INV flag to a new bit, since the iWARP drivers currently in the tree set the bit. The amso1100 driver at least will silently fail to honor the IB_SEND_INVALIDATE bit if passed in as part of userspace send requests (since it does not implement kernel bypass work request queueing). Remove the flag from all existing drivers that set it until we know which ones are OK. The values chosen for the new flags are not consecutive to avoid clashing with flags defined in the IPoIB large send offload and XRC patches, which are not merged yet but which are already in use and are likely to be merged soon. This resurrects a patch sent long ago by Mikkel Hagen . Signed-off-by: Roland Dreier --- I tried to avoid clashes with any flags values defined in OFED 1.3, just to make things a little simpler for users. However a double check that I succeeded in avoiding already-used values would be good. Also I think redefining the IB_DEVICE_SEND_W_INV to a new value avoids all the backwards and forwards compatibility problems that were a sticking point when this idea originally appeared. Anyway if this looks good to people I will merge it for 2.6.26. drivers/infiniband/core/uverbs_cmd.c | 4 ++++ drivers/infiniband/hw/amso1100/c2_rnic.c | 2 +- drivers/infiniband/hw/cxgb3/iwch_provider.c | 3 +-- drivers/infiniband/hw/nes/nes_hw.c | 2 +- include/rdma/ib_user_verbs.h | 4 ++++ include/rdma/ib_verbs.h | 9 +++++++-- 6 files changed, 18 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 495c803..238680c 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1492,6 +1492,10 @@ ssize_t ib_uverbs_post_send(struct ib_uverbs_file *file, next->wr.atomic.swap = user_wr->wr.atomic.swap; next->wr.atomic.rkey = user_wr->wr.atomic.rkey; break; + case IB_WR_SEND: + next->wr.invalidate.rkey = + user_wr->wr.invalidate.rkey; + break; default: break; } diff --git a/drivers/infiniband/hw/amso1100/c2_rnic.c b/drivers/infiniband/hw/amso1100/c2_rnic.c index 7a62552..b1441ae 100644 --- a/drivers/infiniband/hw/amso1100/c2_rnic.c +++ b/drivers/infiniband/hw/amso1100/c2_rnic.c @@ -455,7 +455,7 @@ int __devinit c2_rnic_init(struct c2_dev *c2dev) IB_DEVICE_CURR_QP_STATE_MOD | IB_DEVICE_SYS_IMAGE_GUID | IB_DEVICE_ZERO_STAG | - IB_DEVICE_SEND_W_INV | IB_DEVICE_MEM_WINDOW); + IB_DEVICE_MEM_WINDOW); /* Allocate the qptr_array */ c2dev->qptr_array = vmalloc(C2_MAX_CQS * sizeof(void *)); diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c index 50e1f2a..ca72654 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c @@ -1109,8 +1109,7 @@ int iwch_register_device(struct iwch_dev *dev) memcpy(&dev->ibdev.node_guid, dev->rdev.t3cdev_p->lldev->dev_addr, 6); dev->ibdev.owner = THIS_MODULE; dev->device_cap_flags = - (IB_DEVICE_ZERO_STAG | - IB_DEVICE_SEND_W_INV | IB_DEVICE_MEM_WINDOW); + (IB_DEVICE_ZERO_STAG | IB_DEVICE_MEM_WINDOW); dev->ibdev.uverbs_cmd_mask = (1ull << IB_USER_VERBS_CMD_GET_CONTEXT) | diff --git a/drivers/infiniband/hw/nes/nes_hw.c b/drivers/infiniband/hw/nes/nes_hw.c index 134189d..aa53aab 100644 --- a/drivers/infiniband/hw/nes/nes_hw.c +++ b/drivers/infiniband/hw/nes/nes_hw.c @@ -393,7 +393,7 @@ struct nes_adapter *nes_init_adapter(struct nes_device *nesdev, u8 hw_rev) { nesadapter->base_pd = 1; nesadapter->device_cap_flags = - IB_DEVICE_ZERO_STAG | IB_DEVICE_SEND_W_INV | IB_DEVICE_MEM_WINDOW; + IB_DEVICE_ZERO_STAG | IB_DEVICE_MEM_WINDOW; nesadapter->allocated_qps = (unsigned long *)&(((unsigned char *)nesadapter) [(sizeof(struct nes_adapter)+(sizeof(unsigned long)-1))&(~(sizeof(unsigned long)-1))]); diff --git a/include/rdma/ib_user_verbs.h b/include/rdma/ib_user_verbs.h index 64a721f..7159cb8 100644 --- a/include/rdma/ib_user_verbs.h +++ b/include/rdma/ib_user_verbs.h @@ -553,6 +553,10 @@ struct ib_uverbs_send_wr { __u32 remote_qkey; __u32 reserved; } ud; + struct { + __u32 rkey; + __u32 reserved; + } invalidate; } wr; }; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 40ff512..51fc1f7 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -94,7 +94,7 @@ enum ib_device_cap_flags { IB_DEVICE_SRQ_RESIZE = (1<<13), IB_DEVICE_N_NOTIFY_CQ = (1<<14), IB_DEVICE_ZERO_STAG = (1<<15), - IB_DEVICE_SEND_W_INV = (1<<16), + IB_DEVICE_RESERVED = (1<<16), /* old SEND_W_INV */ IB_DEVICE_MEM_WINDOW = (1<<17), /* * Devices should set IB_DEVICE_UD_IP_SUM if they support @@ -104,6 +104,7 @@ enum ib_device_cap_flags { * IPoIB driver may set NETIF_F_IP_CSUM for datagram mode. */ IB_DEVICE_UD_IP_CSUM = (1<<18), + IB_DEVICE_SEND_W_INV = (1<<21), }; enum ib_atomic_cap { @@ -625,7 +626,8 @@ enum ib_send_flags { IB_SEND_SIGNALED = (1<<1), IB_SEND_SOLICITED = (1<<2), IB_SEND_INLINE = (1<<3), - IB_SEND_IP_CSUM = (1<<4) + IB_SEND_IP_CSUM = (1<<4), + IB_SEND_INVALIDATE = (1<<6) }; struct ib_sge { @@ -660,6 +662,9 @@ struct ib_send_wr { u16 pkey_index; /* valid for GSI only */ u8 port_num; /* valid for DR SMPs on switch only */ } ud; + struct { + u32 rkey; + } invalidate; } wr; }; -- 1.5.4.3 From tugging at anyware-online.de Wed Mar 19 20:34:39 2008 From: tugging at anyware-online.de (Elva Medina) Date: Wed, 19 Mar 2008 19:34:39 -0800 Subject: [ofa-general] save 90%%% on watches! visit store! Message-ID: <01c889f8$45f02580$3a822160@tugging> K qqf ing R ur ep fnk lic ymt a K oo in xw gR cd ep ybi li nx ca was established in early 1999 as a specialist onli wpp ne store selling competitively priced brand name quality re tba pl cp ic pi as. HIGH QUALITY R md EPL wve ICA nar S:R jmj ol tj ex D lw ate rtv jus jp tsR ir ol jg ex S bgx po mu rts M whx ode bt lsAl jjc ai ex n S vhb ilbe rt rst roc einA La pp n wse ge & So fkn hn kg eB lai ell & Ro nl ssBv mzp lg qpc ariCh bcu op wwl ardFr oo an cbp ck M ar ull wb erG pg uc dxv ciPa bs ne jo raiAnd there are a lot of other beautiful things in a gift by New Year!!! All the p euq ric hf es you can find on our si juq te! Elva Medina -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Wed Mar 19 20:35:40 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 20:35:40 -0700 Subject: [ofa-general] [PATCH/RFC 2/2] RDMA/amso1100: Add support for "send with invalidate" work requests In-Reply-To: (Roland Dreier's message of "Wed, 19 Mar 2008 20:33:24 -0700") References: Message-ID: Handle work requests that have the IB_SEND_INVALIDATE flag set. This resurrects a patch sent long ago by Mikkel Hagen . Signed-off-by: Roland Dreier --- Compile-tested only, because I retired my amso1100 systems a while ago. drivers/infiniband/hw/amso1100/c2_qp.c | 16 ++++++++++++---- drivers/infiniband/hw/amso1100/c2_rnic.c | 3 ++- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/hw/amso1100/c2_qp.c b/drivers/infiniband/hw/amso1100/c2_qp.c index 9190bd5..d63edc1 100644 --- a/drivers/infiniband/hw/amso1100/c2_qp.c +++ b/drivers/infiniband/hw/amso1100/c2_qp.c @@ -811,16 +811,24 @@ int c2_post_send(struct ib_qp *ibqp, struct ib_send_wr *ib_wr, switch (ib_wr->opcode) { case IB_WR_SEND: - if (ib_wr->send_flags & IB_SEND_SOLICITED) { + if (ib_wr->send_flags & IB_SEND_SOLICITED && + ib_wr->send_flags & IB_SEND_INVALIDATE) { + c2_wr_set_id(&wr, C2_WR_TYPE_SEND_SE_INV); + wr.sqwr.send.remote_stag = + cpu_to_be32(ib_wr->wr.invalidate.rkey); + } else if (ib_wr->send_flags & IB_SEND_INVALIDATE) { + c2_wr_set_id(&wr, C2_WR_TYPE_SEND_INV); + wr.sqwr.send.remote_stag = + cpu_to_be32(ib_wr->wr.invalidate.rkey); + } else if (ib_wr->send_flags & IB_SEND_SOLICITED) { c2_wr_set_id(&wr, C2_WR_TYPE_SEND_SE); - msg_size = sizeof(struct c2wr_send_req); } else { c2_wr_set_id(&wr, C2_WR_TYPE_SEND); - msg_size = sizeof(struct c2wr_send_req); } wr.sqwr.send.remote_stag = 0; - msg_size += sizeof(struct c2_data_addr) * ib_wr->num_sge; + msg_size = sizeof(struct c2wr_send_req) + + sizeof(struct c2_data_addr) * ib_wr->num_sge; if (ib_wr->num_sge > qp->send_sgl_depth) { err = -EINVAL; break; diff --git a/drivers/infiniband/hw/amso1100/c2_rnic.c b/drivers/infiniband/hw/amso1100/c2_rnic.c index b1441ae..9a054c6 100644 --- a/drivers/infiniband/hw/amso1100/c2_rnic.c +++ b/drivers/infiniband/hw/amso1100/c2_rnic.c @@ -455,7 +455,8 @@ int __devinit c2_rnic_init(struct c2_dev *c2dev) IB_DEVICE_CURR_QP_STATE_MOD | IB_DEVICE_SYS_IMAGE_GUID | IB_DEVICE_ZERO_STAG | - IB_DEVICE_MEM_WINDOW); + IB_DEVICE_MEM_WINDOW | + IB_DEVICE_SEND_W_INV); /* Allocate the qptr_array */ c2dev->qptr_array = vmalloc(C2_MAX_CQS * sizeof(void *)); -- 1.5.4.3 From a-allenm at 4d-konsult.se Wed Mar 19 22:02:03 2008 From: a-allenm at 4d-konsult.se (Lyman Sierra) Date: Thu, 20 Mar 2008 13:02:03 +0800 Subject: [ofa-general] You told me that you will reply back Message-ID: <853202791.11977207452486@4d-konsult.se> Hello! I am bored today. I am nice girl that would like to chat with you. Email me at Ingeborg at BestGolova.com only, because I am using my friend's email to write this. Hope you wanna see my pics. From q03a6ix542x at gmail.com Wed Mar 19 22:11:18 2008 From: q03a6ix542x at gmail.com (=?BIG5?B?p/W78a9d?=) Date: Thu, 20 Mar 2008 13:11:18 +0800 Subject: [ofa-general] =?big5?b?vdC8t6W0MDk0MS01MzAtMjA4ICioQzMwrO0ypLgp?= =?big5?b?IK30rfSkSK5htaWnQajTt1K3UrPhIQ==?= Message-ID: <543c2f2e0803192211q6800b08au2c04a3c1d0c97220@mail.gmail.com> S0aLw0gA04mFN6hGb0kZj3oXv1uRp1pDs4hO 我是辣美眉，想找人聊天嗎? 喜歡重口味的台灣狼，照過來看唷 我最喜歡聊天、唱歌、看電影若你還不錯歡迎來找我！ 激情的夜晚…期盼你的到來… 請撥打 0941-530-208(每30秒2元) rM9LYXzf3b3m5vMesORTLarIvo7ZBNXPapSFuTf18oUgA K1eB37oCH4xRO6zSH4yXC8gZa7eNL3oQm3gZ 4OARRScPSEntyVtcIxAM5WJ64KfzuQsK0oBWzVgDmjTkE From shellyd at seminar.org Wed Mar 19 22:29:23 2008 From: shellyd at seminar.org (=?windows-1255?B?+ezp?=) Date: Thu, 20 Mar 2008 07:29:23 +0200 Subject: [ofa-general] =?windows-1255?b?6/Hz?= Message-ID: <20080320053018.13DC0E60C2D@openfabrics.org> בלי כסף שחור אי אפשר להרוויח במדינה הזו !! ככה זה אצל כולם, אין מה לעשות. המיתון שוחק אותנו, המדינה עושקת אותנו, אז אנחנו מעגלים פינות........... זה לא חייב להיות ככה! מינוסים בבנק, חובות אסטרונומיים, פשיטות רגל ... מסתבר שרוב הבעיות הכספיות של חברות נובעות מניהול כספי לא נכון. רוב העסקים עושים שוב ושוב את אותן שגיאות כספיות ורבים גורמים לנפילת העסק שלהם בעצמם. בעוד שאפשר לשבת שנים וללמוד תואר בחשבונאות, בניהול ובכלכלה ולתלות את התעודה המושקעת על הקיר כדי שכל נושה יוכל לראות. אתה ואני יודעים שהתואר זה לא מה שיציל אותנו מהברוך. סמינר ייחודי נוסף למנהלים והפעם... השגיאות הנפוצות בניהול כספי בסמינר נתמקד רק בדברים שיצילו לך את העסק... מהם 7 הסודות לניהול כספי נכון של מי שמצליח ובגדול? מה הטעות הכי גדולה שלך בהוצאות של העסק? מהי הדרך הבטוחה לניהול כספי בלי להגיע לגירעונות ומימון זר? כיצד לחסוך "הון לבן" ל"ימים שחורים"? מהי הדרך להחזיר חובות, לסגור מינוס ועדיין להישאר בחיים? תיאורית האפונים ועוד הסמינר יתקיים ביום חמישי 3 לאפריל 2008 במלון יוקרתי בת"א בין השעות 09:30-18:00 בוא תלמד כיצד לעשות את ה"בלתי אפשרי": לחיות ולנהל עסק בישראל - ולהרוויח יותר טוב. אותנו לא מעניין תעודות - אלא רק מה שעובד בשטח ומביא לך יותר הצלחה וכסף! כדי לתת חיים חדשים לעסק שלך - שלח מייל ל- bizseminars at gmail.com (נא רשום שם וטלפון) להסרה לחץ כאן unlist -------------- next part -------------- An HTML attachment was scrubbed... URL: From rajouri.jammu at gmail.com Wed Mar 19 22:50:57 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Wed, 19 Mar 2008 22:50:57 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <47D68888.3090709@mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> Message-ID: <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> Hi, Is there a way to tell the HCA driver to use more of the system memory as context memory? When I tried changing the module load parameters num_mpt and num_mtt I got the following error. kernel: ib_mthca 0000:01:00.0: Profile requires 0x8000000 bytes; won't fit in 0x7a00000 bytes of context memory. So it's currently using 128MB as context memory? I'm using dual-port DDR memfree HCA. Also, where can I set the module parameter overrides? I added this line to /etc/init.d/openibd but that didn't make any difference. if [ "X${MTHCA_LOAD}" == "Xyes" ]; then #/sbin/modprobe ib_mthca > /dev/null 2>&1 /sbin/modprobe ib_mthca num_mpt=262144 num_mtt=2097152 > /dev/null 2>&1 The only way I could experiment was to disable openibd autostart and manually run modprobe ib_mtcha num_mpt=262144 num_mtt=2097152 Any info would be greatly appreciated. On Tue, Mar 11, 2008 at 6:26 AM, Tziporet Koren wrote: > Rajouri Jammu wrote: > > The HCA I'm using reports max_mr = 131056. > > > > Our application needs large number of memory regions, much more than > > the 128K limit. > > > > Is there a way to increase this limit? > > I suppose it's bound by memory on the HCA. Is there an HCA that can > > use host memory to increase the number of MRs that can be supported. > > > Which HCA you are using? > In mlx4 and mthca drivers there are modules parameters to change the > amount of memory regions (by enlarging the number of MPTs and MTTs) > > for example: > mlx4: > log_num_mpt: log maximum number of memory protection table > entries per HCA (default is 17; max is 20) > log_num_mtt: log maximum number of memory translation table > segments per HCA (default is 20; max is 20) > > mthca: > num_mpt - maximum number of memory protection table > entries per HCA (int) > num_mtt - maximum number of memory translation table > segments per HCA (int) > > Tziporet > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Wed Mar 19 22:58:49 2008 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 19 Mar 2008 22:58:49 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> (Rajouri Jammu's message of "Wed, 19 Mar 2008 22:50:57 -0700") References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> Message-ID: > kernel: ib_mthca 0000:01:00.0: Profile requires 0x8000000 bytes; won't fit > in 0x7a00000 bytes of context memory. > So it's currently using 128MB as context memory? > > I'm using dual-port DDR memfree HCA. Are you sure it's memfree? It's acting like it's not. What is the output if you load the ib_mthca module with debug_level=1 set? What firmware version do you have installed? > Also, where can I set the module parameter overrides? You should be able to add parameters using modprobe.conf. See the manpage for modprobe.conf for more details. > modprobe ib_mtcha num_mpt=262144 num_mtt=2097152 I tried that here and it works, and I get ib_mthca 0000:0b:00.0: HCA context memory: reserving 706672 KB in the debug output. - R. From rajouri.jammu at gmail.com Wed Mar 19 23:34:21 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Wed, 19 Mar 2008 23:34:21 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> Message-ID: <3307cdf90803192334i5f353824u8fd8796aea7b730d@mail.gmail.com> On Wed, Mar 19, 2008 at 10:58 PM, Roland Dreier wrote: > > > kernel: ib_mthca 0000:01:00.0: Profile requires 0x8000000 bytes; won't > fit > > in 0x7a00000 bytes of context memory. > > So it's currently using 128MB as context memory? > > > > I'm using dual-port DDR memfree HCA. > > Are you sure it's memfree? It's acting like it's not. > You are correct. I looked up the model number and looks like this HCA has 128MB on-board memory. > > What is the output if you load the ib_mthca module with debug_level=1 > set? What firmware version do you have installed? o/p with debug_level=1 kernel: ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) kernel: ib_mthca: Initializing 0000:01:00.0 kernel: PCI: Enabling device 0000:01:00.0 (0000 -> 0002) kernel: ACPI: PCI Interrupt 0000:01:00.0[A] -> GSI 18 (level, low) -> IRQ 58 kernel: ib_mthca 0000:01:00.0: Profile requires 0x8000000 bytes; won't fit in 0x7a00000 bytes of context memory. kernel: ACPI: PCI interrupt for device 0000:01:00.0 disabled kernel: ib_mthca: probe of 0000:01:00.0 failed with error -12 > > > Also, where can I set the module parameter overrides? > > You should be able to add parameters using modprobe.conf. See the > manpage for modprobe.conf for more details. > That worked, thanks! > > > modprobe ib_mtcha num_mpt=262144 num_mtt=2097152 > > I tried that here and it works, and I get > > ib_mthca 0000:0b:00.0: HCA context memory: reserving 706672 KB > > in the debug output. > That's good news. I will try and get a mem-free HCA and test with it. Thanks a lot for the help! > > - R. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From admkirok at arquired.es Thu Mar 20 00:44:02 2008 From: admkirok at arquired.es (Johnnie Messer) Date: Thu, 20 Mar 2008 08:44:02 +0100 Subject: [ofa-general] Bestes Preisleistungsverhaltnis fuer Software Message-ID: <406920662.31597529089372@arquired.es> Software range expansion-price downfallBekommen Sie Ihre Software unverzueglich. Einfach zahlen und sofort runterladen. Hier sind Programme in allen europaeischen Sprachen verfuegbar, programmiert fuer Windows und Macintosh. Alle Softwaren sind sehr guenstig, es handelt sich dabei garantiert um originale, komplette und voellig funktionale Versionen. http://geocities.com/jasoncampos68/* Office Enterprise 2007: $79.95 * Windows XP Professional With SP2 Full Version: $59.95 * Adobe Creative Suite 3 Design Premium: $229.95 * Office System Professional 2003 (5 Cds): $59.95 http://geocities.com/jasoncampos68/Sie werden nach dem Kauf nicht alleine gelassen. Unsere freundlichen Kundenberater werden Ihnen bei der Installation helfen, falls Sie es benotigen. Schnelle Antworten und Geld-Zurueck-Garantie sind bei uns selbstverstandlich. -------------- next part -------------- An HTML attachment was scrubbed... URL: From satietyhjc at gayorlando.com Thu Mar 20 00:46:23 2008 From: satietyhjc at gayorlando.com (Stacey Castro) Date: Thu, 20 Mar 2008 09:46:23 +0200 Subject: [ofa-general] Microsoft Office 2007 OEM version Message-ID: <01c88a6f$41ccc180$6a47eb58@satietyhjc> Microsoft Office Enterprise 2007 includes: � Access 2007 � Communicator 2007 � Excel 2007 � Groove 2007 � InfoPath 2007 � OneNote 2007 � Outlook 2007 � PowerPoint 2007 � Publisher 2007 � Word 2007 http://jordanfolkestk839.blogspot.com System Requirements � Intel� Pentium� or AMD� 500 MHz processor � Microsoft Windows� XP Professional or Home Edition with Service Pack 2, Windows Server� 2003 with SP1, Microsoft Windows Vista. � 256 Mb of RAM � 2GB of available hard-disk space. � 1024x768 or higher resolution monitor � Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From eli at dev.mellanox.co.il Thu Mar 20 01:05:25 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 20 Mar 2008 10:05:25 +0200 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: References: <1205767427.25950.137.camel@mtls03> Message-ID: <1206000325.25950.233.camel@mtls03> On Wed, 2008-03-19 at 16:06 -0700, Roland Dreier wrote: > > @@ -505,6 +509,7 @@ struct ib_qp_init_attr { > > enum ib_sig_type sq_sig_type; > > enum ib_qp_type qp_type; > > u8 port_num; /* special QP types only */ > > + enum qp_create_flags create_flags; > > }; > > I'm dubious about this. It seems to me like everything (including the > mlx4 low-level driver changes for LSO) would be simpler to implement > and use if we just extend the qp_type to include IB_QPT_UD_LSO. For > example it eliminates the need to test if someone tries to create an > LSO QP for a non-UD transport and return an error (although I don't > see that test in your code anyway). I could do with defining a new QP type but the point is that you can still post LSO WRs even if you did not create an LSO QP - it's just that in some cases the post may fail due to insufficient buffer space. Specifying the creation flag tells the driver to reserve memory for LSO needs and so to ensure that any WR with any number of gather entries will succeed. If you prefer I a new QP type I can re-send the patch. > > As I recall, the XRC patches want this flags field too. But does it > work there if we just add another XRC QP type too? From eli at dev.mellanox.co.il Thu Mar 20 01:05:25 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 20 Mar 2008 10:05:25 +0200 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: References: <1205767427.25950.137.camel@mtls03> Message-ID: <1206000325.25950.233.camel@mtls03> On Wed, 2008-03-19 at 16:06 -0700, Roland Dreier wrote: > > @@ -505,6 +509,7 @@ struct ib_qp_init_attr { > > enum ib_sig_type sq_sig_type; > > enum ib_qp_type qp_type; > > u8 port_num; /* special QP types only */ > > + enum qp_create_flags create_flags; > > }; > > I'm dubious about this. It seems to me like everything (including the > mlx4 low-level driver changes for LSO) would be simpler to implement > and use if we just extend the qp_type to include IB_QPT_UD_LSO. For > example it eliminates the need to test if someone tries to create an > LSO QP for a non-UD transport and return an error (although I don't > see that test in your code anyway). I could do with defining a new QP type but the point is that you can still post LSO WRs even if you did not create an LSO QP - it's just that in some cases the post may fail due to insufficient buffer space. Specifying the creation flag tells the driver to reserve memory for LSO needs and so to ensure that any WR with any number of gather entries will succeed. If you prefer I a new QP type I can re-send the patch. > > As I recall, the XRC patches want this flags field too. But does it > work there if we just add another XRC QP type too? From ruchavit at grandhomemart.com Thu Mar 20 01:18:47 2008 From: ruchavit at grandhomemart.com (Orie Hernandez) Date: Thu, 20 Mar 2008 09:18:47 +0100 Subject: [ofa-general] Hier finden Sie unbedingt jede Software Message-ID: <824773164.84394464134430@grandhomemart.com> An HTML attachment was scrubbed... URL: From disinherith4 at eltingnw.com Thu Mar 20 01:19:55 2008 From: disinherith4 at eltingnw.com (Carmela Carroll) Date: Thu, 20 Mar 2008 10:19:55 +0200 Subject: [ofa-general] If this man come to one of these women and ask her telephone number she will definitely give. Message-ID: <01c88a73$f10b9780$4d7c505c@disinherith4> The extracts are Pueraria tuberose 75 mg, Mucuna pruriens 75 mg, Asteracantha longifolia 75 mg. http://ursulachronisterdf729.blogspot.com From kliteyn at dev.mellanox.co.il Thu Mar 20 02:50:36 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 20 Mar 2008 11:50:36 +0200 Subject: [ofa-general] [PATCH] infiniband-diags/Makefile.am: fix 'make install' Message-ID: <47E2336C.4060806@dev.mellanox.co.il> When the code compiled not from the source dir itself, 'make install' fails. Fixing the wrong file location. I'd like to see this fix in ofed_1_3 as well, but it's your call. Signed-off-by: Yevgeny Kliteynik --- infiniband-diags/Makefile.am | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index ca66e2d..26306f9 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -117,4 +117,4 @@ dist-hook: # install this to a default location. install-data-hook: - $(top_srcdir)/config/install-sh -m 444 scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm + $(top_srcdir)/config/install-sh -m 444 $(top_srcdir)/scripts/IBswcountlimits.pm $(DESTDIR)/$(PERL_INSTALLDIR)/IBswcountlimits.pm -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Thu Mar 20 03:00:49 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 20 Mar 2008 12:00:49 +0200 Subject: [ofa-general] [PATCH] infiniband-diags/saquery: print SL in MCast groups Message-ID: <47E235D1.1060708@dev.mellanox.co.il> Printing SL in the MCast groups query. Signed-off-by: Yevgeny Kliteynik --- infiniband-diags/src/saquery.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c index 1b9e2d3..f515a9f 100644 --- a/infiniband-diags/src/saquery.c +++ b/infiniband-diags/src/saquery.c @@ -325,6 +325,9 @@ static void dump_portinfo_record(void *data) static void print_multicast_group_record(ib_member_rec_t *p_mcmr) { + + uint8_t sl; + ib_member_get_sl_flow_hop(p_mcmr->sl_flow_hop, &sl, NULL, NULL); printf("MCMemberRecord group dump:\n" "\t\tMGID....................0x%016" PRIx64 " : " "0x%016" PRIx64 "\n" @@ -332,13 +335,15 @@ print_multicast_group_record(ib_member_rec_t *p_mcmr) "\t\tMtu.....................0x%X\n" "\t\tpkey....................0x%X\n" "\t\tRate....................0x%X\n" + "\t\tSL......................0x%X\n" "", cl_ntoh64( p_mcmr->mgid.unicast.prefix ), cl_ntoh64( p_mcmr->mgid.unicast.interface_id ), cl_ntoh16( p_mcmr->mlid ), p_mcmr->mtu, cl_ntoh16( p_mcmr->pkey ), - p_mcmr->rate + p_mcmr->rate, + sl ); } -- 1.5.1.4 From hans.zettl at t-online.de Thu Mar 20 03:24:04 2008 From: hans.zettl at t-online.de (Kelsey Conway) Date: Thu, 20 Mar 2008 13:24:04 +0300 Subject: [ofa-general] {Viagra_onli2_de} Message-ID: <398803327.89370868985648@t-online.de> Sie leben nur einmal - warum dann nicht was neues ausprobieren? Pr. .. Eise die keine Konk... ..Urrenz kennen - Bequem und dis kret 0... . N-line! be... .Stellen. - Disk rete Verpackung und Zahlung - Kein peinlicher Arz t besuch erforderlich - keine versteckte Kos// Ten - Kos... Tenlose, arztliche Telefon-Beratung - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen Originalme/ dikamente Ciia..aa^_^aaalis...... 10 Pack. 21,00 Euro Viia..aa^_^aaagra... 10 Pack. 11,00 Euro Klicken Sie HIER und Sie erhalten vier Dosen umsonst RedBull fur ihr bestes Stuck (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bs at q-leap.de Thu Mar 20 04:30:40 2008 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 20 Mar 2008 12:30:40 +0100 Subject: [ofa-general] RcvSwRelayErrors Message-ID: <200803201230.40303.bs@q-leap.de> Hello, on one of our systems we get a rather huge numbers of RcvSwRelayErrors. All I find about RcvSwRelayErrors is "This counter can increase due to a valid network event" But what might cause? Thanks in advance for any help, Bernd [...] 11: [RcvSwRelayErrors == 189] 12: [RcvSwRelayErrors == 196] 16: [RcvSwRelayErrors == 34655] Errors for 0x000b8cffff002b33 "MT47396 Infiniscale-III Mellanox Technologies ()" 1: [RcvSwRelayErrors == 190] 2: [RcvSwRelayErrors == 188] 3: [RcvSwRelayErrors == 195] 4: [RcvSwRelayErrors == 207] 5: [RcvSwRelayErrors == 194] 6: [RcvSwRelayErrors == 189] 8: [RcvSwRelayErrors == 198] 9: [RcvSwRelayErrors == 197] 10: [RcvSwRelayErrors == 190] 11: [RcvSwRelayErrors == 198] 12: [RcvSwRelayErrors == 190] 16: [RcvSwRelayErrors == 34711] Errors for 0x000b8cffff002b43 "MT47396 Infiniscale-III Mellanox Technologies ()" 1: [RcvSwRelayErrors == 196] 3: [RcvSwRelayErrors == 242] [...] -- Bernd Schubert Q-Leap Networks GmbH From dwpriofynm at priofyn.dk Thu Mar 20 04:51:57 2008 From: dwpriofynm at priofyn.dk (Reggie Durham) Date: Thu, 20 Mar 2008 17:21:57 +0530 Subject: [ofa-general] Improve your health supersite Message-ID: <080650974.73016483381108@priofyn.dk> An HTML attachment was scrubbed... URL: From _c at actioninmailing.com Thu Mar 20 03:23:45 2008 From: _c at actioninmailing.com (adolphus nathan) Date: Thu, 20 Mar 2008 10:23:45 +0000 Subject: [ofa-general] Naked Shakira Clip Message-ID: <000701c88a83$03f4123c$db43a491@xnyip> Forenhaftung: Jeder Blogger ein Zensor? Download and Watch -------------- next part -------------- An HTML attachment was scrubbed... URL: From barbsvs at sapcorp.com Thu Mar 20 05:24:51 2008 From: barbsvs at sapcorp.com (Lelia Edmonds) Date: Thu, 20 Mar 2008 13:24:51 +0100 Subject: [ofa-general] Access 2007 Message-ID: <01c88a8d$c70c6800$3e22304d@barbsvs> Microsoft Office Enterprise 2007 includes: • Access 2007 • Communicator 2007 • Excel 2007 • Groove 2007 • InfoPath 2007 • OneNote 2007 • Outlook 2007 • PowerPoint 2007 • Publisher 2007 • Word 2007 http://lenoraalessip703.blogspot.com System Requirements • Intel® Pentium® or AMD® 500 MHz processor • Microsoft Windows® XP Professional or Home Edition with Service Pack 2, Windows Server® 2003 with SP1, Microsoft Windows Vista. • 256 Mb of RAM • 2GB of available hard-disk space. • 1024x768 or higher resolution monitor • Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From abbatis at atlconnect.com Thu Mar 20 05:27:46 2008 From: abbatis at atlconnect.com (Herminia Oconnell) Date: Thu, 20 Mar 2008 14:27:46 +0200 Subject: [ofa-general] Ihnen werden unsere Softwarepreise gefallen Message-ID: <802976203.95189146586970@atlconnect.com> Cut-price OEM software e-shopSie konnen unsere Software sofort runterladen. Zahlen Sie und die Downloads werden Ihnen sofort zur Verfugung stehen. Unsere Programme wurden fuer mehrere OS (Windows und Macintosh) programmiert und sind in allen europaeischen Sprachen verfuegbar. Wir verkaufen nur originale Vollversionen, haben aber die guenstigsten Preise. http://geocities.com/morganlawson42/* Office Enterprise 2007: $79.95 * Windows XP Professional With SP2 Full Version: $59.95 * Office System Professional 2003 (5 Cds): $59.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/morganlawson42/Bei uns kaufen Sie sicher ein, denn unsere kompetenten Mitarbeiten vom Kundencenter werden Ihnen bei der Softwareinstallation weiterhelfen. Wir antworten unverzueglich und Sie bekommen von uns eine Geld-Zurueck-Garantie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at mellanox.co.il Thu Mar 20 05:31:02 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Thu, 20 Mar 2008 14:31:02 +0200 Subject: [ofa-general] Cannot get MSI-X verctors on RHEL5-up1 Message-ID: <47E25906.9040202@mellanox.co.il> Hi Doug, We found that on RHEL5 up1 we do not succeed to get MSI-X vectors with for ConnectX with the mlx4 driver. Note that on RHEL5 we do get them. Any help will be appreciated. Thanks Tziporet This is the output of lspci -vv so you can see that the MSI-X is available by the device 13:00.0 InfiniBand: Mellanox Technologies MT25418 [ConnectX IB DDR] (rev a0) Subsystem: Mellanox Technologies MT25418 [ConnectX IB DDR] Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- References: <1205866458.11393.244.camel@hrosenstock-ws.xsigo.com> Message-ID: <47E25AC2.8040706@mellanox.co.il> Roland Dreier wrote: > > Can mthca share an IRQ with some other device (not operating in MSI or > > MSI-X mode if mthca is configured this way) ? Are there any known issues > > in doing this ? I seem to vaguely recall some discussion of this quite a > > long time ago but couldn't dig out the email threads on this. Thanks in > > advance for any insights on this. > > As far as I know there should be no problem if an mthca device is > sharing an interrupt line with another device in INTx mode. > > > We also had some systems here that used this mode without any problem Tziporet From jackm at dev.mellanox.co.il Thu Mar 20 05:45:22 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 20 Mar 2008 14:45:22 +0200 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: References: <1205767427.25950.137.camel@mtls03> Message-ID: <200803201445.23368.jackm@dev.mellanox.co.il> On Thursday 20 March 2008 01:06, Roland Dreier wrote: > As I recall, the XRC patches want this flags field too.  But does it > work there if we just add another XRC QP type too? > Not as simple. XRC has 2 QP types -- a regular XRC QP, and a receive-only XRC QP (created only by a userspace app for receiving into XRC SRQs only, but owned by the kernel -- so that the creating app can terminate without the XRC RCV QP being destroyed). There are many places where I test for IB_QPT_XRC -- and I would need to test for all the XRC qp types in these places. Just doing a grep on the kernel_patches/fixes for the mlx4 driver yields: ..../ofed_kernel/kernel_patches/fixes> grep IB_QPT_XRC mlx4* mlx4_0070_xrc.patch:+ if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) { mlx4_0070_xrc.patch:+ if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) { mlx4_0070_xrc.patch:+ if (init_attr->qp_type == IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!pd->uobject && !init_attr->srq && init_attr->qp_type != IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!qp->ibqp.srq && qp->ibqp.qp_type != IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!qp->ibqp.srq && qp->ibqp.qp_type != IB_QPT_XRC) mlx4_0070_xrc.patch:+ case IB_QPT_XRC: mlx4_0070_xrc.patch:+ if (init_attr->qp_type == IB_QPT_XRC) mlx4_0070_xrc.patch:+ case IB_QPT_XRC: return MLX4_QP_ST_XRC; mlx4_0070_xrc.patch:+ if (ibqp->qp_type == IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!ibqp->srq && ibqp->qp_type != IB_QPT_XRC && mlx4_0070_xrc.patch:+ if (!ibqp->srq && ibqp->qp_type != IB_QPT_XRC) mlx4_0120_xrc_kernel.patch:+ case IB_QPT_XRC: mlx4_0125_xrc_kernel_missed.patch:mlx4: fix some missed spots for IB_QPT_XRC qps. mlx4_0125_xrc_kernel_missed.patch:+ case IB_QPT_XRC: mlx4_0125_xrc_kernel_missed.patch:+static const int mlx4_ib_qp_attr_mask_table[IB_QPT_XRC + 1] = { mlx4_0125_xrc_kernel_missed.patch:+ [IB_QPT_XRC] = (IB_QP_PKEY_INDEX | mlx4_0125_xrc_kernel_missed.patch:+ qp->ibqp.qp_type == IB_QPT_XRC) { mlx4_0170_shrinking_wqe.patch: if (!ibqp->srq && ibqp->qp_type != IB_QPT_XRC) mlx4_0210_xrc_rcv.patch:+ if (!(ibqp->qp_type == IB_QPT_XRC && mlx4_0210_xrc_rcv.patch:+ if (init_attr->qp_type == IB_QPT_XRC && mlx4_0210_xrc_rcv.patch:+ ia.qp_type = IB_QPT_XRC; mlx4_0210_xrc_rcv.patch:+ qp->ibqp.qp_type == IB_QPT_XRC) { Its much simpler if I just have a create-flag: mlx4_0210_xrc_rcv.patch:+ MLX4_XRC_RCV = 1 << 1, mlx4_0210_xrc_rcv.patch:+ mqp->flags & MLX4_XRC_RCV)) { mlx4_0210_xrc_rcv.patch:+ init_attr->create_flags & QP_CREATE_XRC_RCV) mlx4_0210_xrc_rcv.patch:+ qp->flags |= MLX4_XRC_RCV; Furthermore, what happens if a "create-flag" applies to more than a single base-qp type? Do we then create multiple new QP types for this flag, one for each base type? I think the create_flags (essentially, a create-modifier) is the better choice. - Jack From jackm at dev.mellanox.co.il Thu Mar 20 05:45:22 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Thu, 20 Mar 2008 14:45:22 +0200 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: References: <1205767427.25950.137.camel@mtls03> Message-ID: <200803201445.23368.jackm@dev.mellanox.co.il> On Thursday 20 March 2008 01:06, Roland Dreier wrote: > As I recall, the XRC patches want this flags field too.  But does it > work there if we just add another XRC QP type too? > Not as simple. XRC has 2 QP types -- a regular XRC QP, and a receive-only XRC QP (created only by a userspace app for receiving into XRC SRQs only, but owned by the kernel -- so that the creating app can terminate without the XRC RCV QP being destroyed). There are many places where I test for IB_QPT_XRC -- and I would need to test for all the XRC qp types in these places. Just doing a grep on the kernel_patches/fixes for the mlx4 driver yields: ..../ofed_kernel/kernel_patches/fixes> grep IB_QPT_XRC mlx4* mlx4_0070_xrc.patch:+ if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) { mlx4_0070_xrc.patch:+ if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) { mlx4_0070_xrc.patch:+ if (init_attr->qp_type == IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!init_attr->srq && init_attr->qp_type != IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!pd->uobject && !init_attr->srq && init_attr->qp_type != IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!qp->ibqp.srq && qp->ibqp.qp_type != IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!qp->ibqp.srq && qp->ibqp.qp_type != IB_QPT_XRC) mlx4_0070_xrc.patch:+ case IB_QPT_XRC: mlx4_0070_xrc.patch:+ if (init_attr->qp_type == IB_QPT_XRC) mlx4_0070_xrc.patch:+ case IB_QPT_XRC: return MLX4_QP_ST_XRC; mlx4_0070_xrc.patch:+ if (ibqp->qp_type == IB_QPT_XRC) mlx4_0070_xrc.patch:+ if (!ibqp->srq && ibqp->qp_type != IB_QPT_XRC && mlx4_0070_xrc.patch:+ if (!ibqp->srq && ibqp->qp_type != IB_QPT_XRC) mlx4_0120_xrc_kernel.patch:+ case IB_QPT_XRC: mlx4_0125_xrc_kernel_missed.patch:mlx4: fix some missed spots for IB_QPT_XRC qps. mlx4_0125_xrc_kernel_missed.patch:+ case IB_QPT_XRC: mlx4_0125_xrc_kernel_missed.patch:+static const int mlx4_ib_qp_attr_mask_table[IB_QPT_XRC + 1] = { mlx4_0125_xrc_kernel_missed.patch:+ [IB_QPT_XRC] = (IB_QP_PKEY_INDEX | mlx4_0125_xrc_kernel_missed.patch:+ qp->ibqp.qp_type == IB_QPT_XRC) { mlx4_0170_shrinking_wqe.patch: if (!ibqp->srq && ibqp->qp_type != IB_QPT_XRC) mlx4_0210_xrc_rcv.patch:+ if (!(ibqp->qp_type == IB_QPT_XRC && mlx4_0210_xrc_rcv.patch:+ if (init_attr->qp_type == IB_QPT_XRC && mlx4_0210_xrc_rcv.patch:+ ia.qp_type = IB_QPT_XRC; mlx4_0210_xrc_rcv.patch:+ qp->ibqp.qp_type == IB_QPT_XRC) { Its much simpler if I just have a create-flag: mlx4_0210_xrc_rcv.patch:+ MLX4_XRC_RCV = 1 << 1, mlx4_0210_xrc_rcv.patch:+ mqp->flags & MLX4_XRC_RCV)) { mlx4_0210_xrc_rcv.patch:+ init_attr->create_flags & QP_CREATE_XRC_RCV) mlx4_0210_xrc_rcv.patch:+ qp->flags |= MLX4_XRC_RCV; Furthermore, what happens if a "create-flag" applies to more than a single base-qp type? Do we then create multiple new QP types for this flag, one for each base type? I think the create_flags (essentially, a create-modifier) is the better choice. - Jack From tensery64 at fsai.net Thu Mar 20 05:49:40 2008 From: tensery64 at fsai.net (Louie Lyon) Date: Thu, 20 Mar 2008 04:49:40 -0800 Subject: [ofa-general] 100mg x 30 pills $3.33 per pill buy now Message-ID: <01c88a45$ce628a00$c492515c@tensery64> buy now Viagra 60mg x 30 pills http://sharronmilesxc447.blogspot.com From hrosenstock at xsigo.com Thu Mar 20 05:27:36 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 20 Mar 2008 05:27:36 -0700 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <200803201230.40303.bs@q-leap.de> References: <200803201230.40303.bs@q-leap.de> Message-ID: <1206016056.11393.561.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > Hello, > > on one of our systems we get a rather huge numbers of RcvSwRelayErrors. All I > find about RcvSwRelayErrors is > > "This counter can increase due to a valid network event" > > But what might cause? Are you running IB multicast (e.g. IPoIB) ? That's the most common cause. -- Hal > Thanks in advance for any help, > Bernd > > > [...] > 11: [RcvSwRelayErrors == 189] > 12: [RcvSwRelayErrors == 196] > 16: [RcvSwRelayErrors == 34655] > Errors for 0x000b8cffff002b33 "MT47396 Infiniscale-III Mellanox Technologies > ()" > 1: [RcvSwRelayErrors == 190] > 2: [RcvSwRelayErrors == 188] > 3: [RcvSwRelayErrors == 195] > 4: [RcvSwRelayErrors == 207] > 5: [RcvSwRelayErrors == 194] > 6: [RcvSwRelayErrors == 189] > 8: [RcvSwRelayErrors == 198] > 9: [RcvSwRelayErrors == 197] > 10: [RcvSwRelayErrors == 190] > 11: [RcvSwRelayErrors == 198] > 12: [RcvSwRelayErrors == 190] > 16: [RcvSwRelayErrors == 34711] > Errors for 0x000b8cffff002b43 "MT47396 Infiniscale-III Mellanox Technologies > ()" > 1: [RcvSwRelayErrors == 196] > 3: [RcvSwRelayErrors == 242] > [...] > From bs at q-leap.de Thu Mar 20 05:54:53 2008 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 20 Mar 2008 13:54:53 +0100 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <1206016056.11393.561.camel@hrosenstock-ws.xsigo.com> References: <200803201230.40303.bs@q-leap.de> <1206016056.11393.561.camel@hrosenstock-ws.xsigo.com> Message-ID: <200803201354.53826.bs@q-leap.de> On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > Hello, > > > > on one of our systems we get a rather huge numbers of RcvSwRelayErrors. > > All I find about RcvSwRelayErrors is > > > > "This counter can increase due to a valid network event" > > > > But what might cause? Ooops. This should have been "But what might cause it?" > > Are you running IB multicast (e.g. IPoIB) ? That's the most common > cause. IPoIB is up, but so far only used initially by lustre for initial lnet o2ib setup, but then AFAIK not any more. I think some MPI stacks/applications also do their intial connection using IPoIB. But in general, once these connections are established, IPoIB is not much used anymore. Thanks, Bernd > > -- Hal > > > Thanks in advance for any help, > > Bernd > > > > > > [...] > > 11: [RcvSwRelayErrors == 189] > > 12: [RcvSwRelayErrors == 196] > > 16: [RcvSwRelayErrors == 34655] > > Errors for 0x000b8cffff002b33 "MT47396 Infiniscale-III Mellanox > > Technologies ()" > > 1: [RcvSwRelayErrors == 190] > > 2: [RcvSwRelayErrors == 188] > > 3: [RcvSwRelayErrors == 195] > > 4: [RcvSwRelayErrors == 207] > > 5: [RcvSwRelayErrors == 194] > > 6: [RcvSwRelayErrors == 189] > > 8: [RcvSwRelayErrors == 198] > > 9: [RcvSwRelayErrors == 197] > > 10: [RcvSwRelayErrors == 190] > > 11: [RcvSwRelayErrors == 198] > > 12: [RcvSwRelayErrors == 190] > > 16: [RcvSwRelayErrors == 34711] > > Errors for 0x000b8cffff002b43 "MT47396 Infiniscale-III Mellanox > > Technologies ()" > > 1: [RcvSwRelayErrors == 196] > > 3: [RcvSwRelayErrors == 242] > > [...] -- Bernd Schubert Q-Leap Networks GmbH From nasalizingiyi0 at vocalyze.com Thu Mar 20 06:01:42 2008 From: nasalizingiyi0 at vocalyze.com (Stacey Zapata) Date: Thu, 20 Mar 2008 21:01:42 +0800 Subject: [ofa-general] Women enlarge their bosoms and for them it become normally. Message-ID: <01c88acd$99017f00$4004e774@nasalizingiyi0> Pureria tuberosa (Fabaceae) 20 mg http://yolandahoffmaneu366.blogspot.com From eli at mellanox.co.il Thu Mar 20 06:07:51 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Thu, 20 Mar 2008 15:07:51 +0200 Subject: [ofa-general] Re: Cannot get MSI-X verctors on RHEL5-up1 In-Reply-To: <47E25906.9040202@mellanox.co.il> References: <47E25906.9040202@mellanox.co.il> Message-ID: <1206018471.25950.245.camel@mtls03> I opened a bug in Redhat bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=438330 On Thu, 2008-03-20 at 14:31 +0200, Tziporet Koren wrote: > Hi Doug, > > We found that on RHEL5 up1 we do not succeed to get MSI-X vectors with for ConnectX with the mlx4 driver. > Note that on RHEL5 we do get them. > > Any help will be appreciated. > > Thanks > Tziporet > > This is the output of lspci -vv so you can see that the MSI-X is available by the device > > 13:00.0 InfiniBand: Mellanox Technologies MT25418 [ConnectX IB DDR] (rev a0) > Subsystem: Mellanox Technologies MT25418 [ConnectX IB DDR] > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 177 > Region 0: Memory at fdf00000 (64-bit, non-prefetchable) [size=1M] > Region 2: Memory at f7000000 (64-bit, prefetchable) [size=8M] > Region 4: Memory at fdef0000 (64-bit, non-prefetchable) [size=8K] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [48] Vital Product Data > Capabilities: [9c] MSI-X: Enable- Mask- TabSize=256 > Vector table: BAR=4 offset=00000000 > PBA: BAR=4 offset=00001000 > Capabilities: [60] Express Endpoint IRQ 0 > Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag+ > Device: Latency L0s <64ns, L1 unlimited > Device: AtnBtn- AtnInd- PwrInd- > Device: Errors: Correctable- Non-Fatal+ Fatal+ Unsupported- > Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- > Device: MaxPayload 256 bytes, MaxReadReq 4096 bytes > Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s, Port 8 > Link: Latency L0s unlimited, L1 unlimited > Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch- > Link: Speed 2.5Gb/s, Width x8 > From tziporet at dev.mellanox.co.il Thu Mar 20 06:34:18 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 20 Mar 2008 15:34:18 +0200 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803192334i5f353824u8fd8796aea7b730d@mail.gmail.com> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> <3307cdf90803192334i5f353824u8fd8796aea7b730d@mail.gmail.com> Message-ID: <47E267DA.9010102@mellanox.co.il> Rajouri Jammu wrote: > > > That's good news. I will try and get a mem-free HCA and test with it. > If its Arbel card you just need to burn it with different FW for mem-free For FW release please see http://www.mellanox.com/support/firmware_table_IH3Ex.php Tziporet From eli at dev.mellanox.co.il Thu Mar 20 06:37:28 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 20 Mar 2008 15:37:28 +0200 Subject: [ofa-general] Re: Cannot get MSI-X verctors on RHEL5-up1 In-Reply-To: <1206018471.25950.245.camel@mtls03> References: <47E25906.9040202@mellanox.co.il> <1206018471.25950.245.camel@mtls03> Message-ID: <1206020248.25950.248.camel@mtls03> Adding Doug whom I omitted by mistake. On Thu, 2008-03-20 at 15:07 +0200, Eli Cohen wrote: > I opened a bug in Redhat bugzilla: > > https://bugzilla.redhat.com/show_bug.cgi?id=438330 > > On Thu, 2008-03-20 at 14:31 +0200, Tziporet Koren wrote: > > Hi Doug, > > > > We found that on RHEL5 up1 we do not succeed to get MSI-X vectors with for ConnectX with the mlx4 driver. > > Note that on RHEL5 we do get them. > > > > Any help will be appreciated. > > > > Thanks > > Tziporet > > > > This is the output of lspci -vv so you can see that the MSI-X is available by the device > > > > 13:00.0 InfiniBand: Mellanox Technologies MT25418 [ConnectX IB DDR] (rev a0) > > Subsystem: Mellanox Technologies MT25418 [ConnectX IB DDR] > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- > Latency: 0, Cache Line Size: 64 bytes > > Interrupt: pin A routed to IRQ 177 > > Region 0: Memory at fdf00000 (64-bit, non-prefetchable) [size=1M] > > Region 2: Memory at f7000000 (64-bit, prefetchable) [size=8M] > > Region 4: Memory at fdef0000 (64-bit, non-prefetchable) [size=8K] > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) > > Status: D0 PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [48] Vital Product Data > > Capabilities: [9c] MSI-X: Enable- Mask- TabSize=256 > > Vector table: BAR=4 offset=00000000 > > PBA: BAR=4 offset=00001000 > > Capabilities: [60] Express Endpoint IRQ 0 > > Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag+ > > Device: Latency L0s <64ns, L1 unlimited > > Device: AtnBtn- AtnInd- PwrInd- > > Device: Errors: Correctable- Non-Fatal+ Fatal+ Unsupported- > > Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- > > Device: MaxPayload 256 bytes, MaxReadReq 4096 bytes > > Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s, Port 8 > > Link: Latency L0s unlimited, L1 unlimited > > Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch- > > Link: Speed 2.5Gb/s, Width x8 > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hrosenstock at xsigo.com Thu Mar 20 07:12:03 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 20 Mar 2008 07:12:03 -0700 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <200803201354.53826.bs@q-leap.de> References: <200803201230.40303.bs@q-leap.de> <1206016056.11393.561.camel@hrosenstock-ws.xsigo.com> <200803201354.53826.bs@q-leap.de> Message-ID: <1206022323.7332.34.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-20 at 13:54 +0100, Bernd Schubert wrote: > On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > > Hello, > > > > > > on one of our systems we get a rather huge numbers of RcvSwRelayErrors. > > > All I find about RcvSwRelayErrors is > > > > > > "This counter can increase due to a valid network event" > > > > > > But what might cause? > > Ooops. This should have been "But what might cause it?" > > > > > Are you running IB multicast (e.g. IPoIB) ? That's the most common > > cause. > > IPoIB is up, but so far only used initially by lustre for initial lnet o2ib > setup, but then AFAIK not any more. I think some MPI stacks/applications also > do their intial connection using IPoIB. > > But in general, once these connections are established, IPoIB is not much used > anymore. The causes are: 1. DLID mapping 2. VL mapping 3. looping (out port = in port) Is your subnet unstable in some way ? Are you using QoS ? -- Hal > > Thanks, > Bernd > > > > > > -- Hal > > > > > Thanks in advance for any help, > > > Bernd > > > > > > > > > [...] > > > 11: [RcvSwRelayErrors == 189] > > > 12: [RcvSwRelayErrors == 196] > > > 16: [RcvSwRelayErrors == 34655] > > > Errors for 0x000b8cffff002b33 "MT47396 Infiniscale-III Mellanox > > > Technologies ()" > > > 1: [RcvSwRelayErrors == 190] > > > 2: [RcvSwRelayErrors == 188] > > > 3: [RcvSwRelayErrors == 195] > > > 4: [RcvSwRelayErrors == 207] > > > 5: [RcvSwRelayErrors == 194] > > > 6: [RcvSwRelayErrors == 189] > > > 8: [RcvSwRelayErrors == 198] > > > 9: [RcvSwRelayErrors == 197] > > > 10: [RcvSwRelayErrors == 190] > > > 11: [RcvSwRelayErrors == 198] > > > 12: [RcvSwRelayErrors == 190] > > > 16: [RcvSwRelayErrors == 34711] > > > Errors for 0x000b8cffff002b43 "MT47396 Infiniscale-III Mellanox > > > Technologies ()" > > > 1: [RcvSwRelayErrors == 196] > > > 3: [RcvSwRelayErrors == 242] > > > [...] > > > From satirizej at lacolonial.com Thu Mar 20 07:24:19 2008 From: satirizej at lacolonial.com (Aimee Hayes) Date: Thu, 20 Mar 2008 16:24:19 +0200 Subject: [ofa-general] Microsoft Office Enterprise 2007 Message-ID: <01c88aa6$d9013b80$5a77b74f@satirizej> Microsoft Office Enterprise 2007 includes: • Access 2007 • Communicator 2007 • Excel 2007 • Groove 2007 • InfoPath 2007 • OneNote 2007 • Outlook 2007 • PowerPoint 2007 • Publisher 2007 • Word 2007 http://raehoyerhd479.blogspot.com System Requirements • Intel® Pentium® or AMD® 500 MHz processor • Microsoft Windows® XP Professional or Home Edition with Service Pack 2, Windows Server® 2003 with SP1, Microsoft Windows Vista. • 256 Mb of RAM • 2GB of available hard-disk space. • 1024x768 or higher resolution monitor • Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From bs at q-leap.de Thu Mar 20 07:27:38 2008 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 20 Mar 2008 15:27:38 +0100 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <1206022323.7332.34.camel@hrosenstock-ws.xsigo.com> References: <200803201230.40303.bs@q-leap.de> <200803201354.53826.bs@q-leap.de> <1206022323.7332.34.camel@hrosenstock-ws.xsigo.com> Message-ID: <200803201527.38470.bs@q-leap.de> On Thursday 20 March 2008 15:12:03 Hal Rosenstock wrote: > On Thu, 2008-03-20 at 13:54 +0100, Bernd Schubert wrote: > > On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > > > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > > > Hello, > > > > > > > > on one of our systems we get a rather huge numbers of > > > > RcvSwRelayErrors. All I find about RcvSwRelayErrors is > > > > > > > > "This counter can increase due to a valid network event" > > > > > > > > But what might cause? > > > > Ooops. This should have been "But what might cause it?" > > > > > Are you running IB multicast (e.g. IPoIB) ? That's the most common > > > cause. > > > > IPoIB is up, but so far only used initially by lustre for initial lnet > > o2ib setup, but then AFAIK not any more. I think some MPI > > stacks/applications also do their intial connection using IPoIB. > > > > But in general, once these connections are established, IPoIB is not much > > used anymore. > > The causes are: > 1. DLID mapping > 2. VL mapping > 3. looping (out port = in port) > > Is your subnet unstable in some way ? Are you using QoS ? > We have seen some odd problems with opensm (from ofef-1.2.5) in the past and once only rebooting the switches did help. Yesterday I started monitoring the the fabric and even though there's not much traffic, I immediately noticed these errors. We are not using QoS. Thanks for your help, Bernd -- Bernd Schubert Q-Leap Networks GmbH From hrosenstock at xsigo.com Thu Mar 20 07:29:35 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 20 Mar 2008 07:29:35 -0700 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <200803201527.38470.bs@q-leap.de> References: <200803201230.40303.bs@q-leap.de> <200803201354.53826.bs@q-leap.de> <1206022323.7332.34.camel@hrosenstock-ws.xsigo.com> <200803201527.38470.bs@q-leap.de> Message-ID: <1206023375.7332.38.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-20 at 15:27 +0100, Bernd Schubert wrote: > On Thursday 20 March 2008 15:12:03 Hal Rosenstock wrote: > > On Thu, 2008-03-20 at 13:54 +0100, Bernd Schubert wrote: > > > On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > > > > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > > > > Hello, > > > > > > > > > > on one of our systems we get a rather huge numbers of > > > > > RcvSwRelayErrors. All I find about RcvSwRelayErrors is > > > > > > > > > > "This counter can increase due to a valid network event" > > > > > > > > > > But what might cause? > > > > > > Ooops. This should have been "But what might cause it?" > > > > > > > Are you running IB multicast (e.g. IPoIB) ? That's the most common > > > > cause. > > > > > > IPoIB is up, but so far only used initially by lustre for initial lnet > > > o2ib setup, but then AFAIK not any more. I think some MPI > > > stacks/applications also do their intial connection using IPoIB. > > > > > > But in general, once these connections are established, IPoIB is not much > > > used anymore. > > > > The causes are: > > 1. DLID mapping > > 2. VL mapping > > 3. looping (out port = in port) > > > > Is your subnet unstable in some way ? Are you using QoS ? > > > > We have seen some odd problems with opensm (from ofef-1.2.5) in the past and > once only rebooting the switches did help. You might want to update OpenSM to OFED 1.3 version. > Yesterday I started monitoring the the fabric and even though there's not much > traffic, I immediately noticed these errors. Were the counters cleared before you started looking ? -- Hal > We are not using QoS. > > > Thanks for your help, > Bernd > > From bs at q-leap.de Thu Mar 20 07:33:47 2008 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 20 Mar 2008 15:33:47 +0100 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <1206023375.7332.38.camel@hrosenstock-ws.xsigo.com> References: <200803201230.40303.bs@q-leap.de> <200803201527.38470.bs@q-leap.de> <1206023375.7332.38.camel@hrosenstock-ws.xsigo.com> Message-ID: <200803201533.47801.bs@q-leap.de> On Thursday 20 March 2008 15:29:35 Hal Rosenstock wrote: > On Thu, 2008-03-20 at 15:27 +0100, Bernd Schubert wrote: > > On Thursday 20 March 2008 15:12:03 Hal Rosenstock wrote: > > > On Thu, 2008-03-20 at 13:54 +0100, Bernd Schubert wrote: > > > > On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > > > > > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > > > > > Hello, > > > > > > > > > > > > on one of our systems we get a rather huge numbers of > > > > > > RcvSwRelayErrors. All I find about RcvSwRelayErrors is > > > > > > > > > > > > "This counter can increase due to a valid network event" > > > > > > > > > > > > But what might cause? > > > > > > > > Ooops. This should have been "But what might cause it?" > > > > > > > > > Are you running IB multicast (e.g. IPoIB) ? That's the most common > > > > > cause. > > > > > > > > IPoIB is up, but so far only used initially by lustre for initial > > > > lnet o2ib setup, but then AFAIK not any more. I think some MPI > > > > stacks/applications also do their intial connection using IPoIB. > > > > > > > > But in general, once these connections are established, IPoIB is not > > > > much used anymore. > > > > > > The causes are: > > > 1. DLID mapping > > > 2. VL mapping > > > 3. looping (out port = in port) > > > > > > Is your subnet unstable in some way ? Are you using QoS ? > > > > We have seen some odd problems with opensm (from ofef-1.2.5) in the past > > and once only rebooting the switches did help. > > You might want to update OpenSM to OFED 1.3 version. I won't manage to build new debian packages today, but I will do over Easter. Hope to also find the time to clean the debian rules a bit, to have it officially included in Debian. But will a new opensm help for these errors? > > > Yesterday I started monitoring the the fabric and even though there's not > > much traffic, I immediately noticed these errors. > > Were the counters cleared before you started looking ? Yes, sure. Thanks a lot for your help, Bernd -- Bernd Schubert Q-Leap Networks GmbH From hrosenstock at xsigo.com Thu Mar 20 07:34:55 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 20 Mar 2008 07:34:55 -0700 Subject: [ofa-general] [PATCH][TRIVIAL] OpenSM release notes: Add byacc as alternative to bison for qos parser Message-ID: <1206023695.8099.28.camel@hrosenstock-ws.xsigo.com> OpenSM release notes: Add byacc as alternative to bison for qos parser Signed-off-by: Hal Rosenstock diff --git a/opensm/doc/opensm_release_notes-3.1.10.txt b/opensm/doc/opensm_release_notes-3.1.10.txt index 610f910..732ff59 100644 --- a/opensm/doc/opensm_release_notes-3.1.10.txt +++ b/opensm/doc/opensm_release_notes-3.1.10.txt @@ -128,8 +128,8 @@ OFED 1.0, OpenIB gen2 (e.g. IBG2 distribution), OpenIB gen1 (e.g. IBGD distribution), or Mellanox VAPI stacks. The qualified driver versions are provided in Table 2, "Qualified IB Stacks". -Also building of QoS manager policy file parser requires flex and bison -installed. +Also building of QoS manager policy file parser requires flex, and either +bison or byacc installed. 1.5 Supported Devices Firmware From hrosenstock at xsigo.com Thu Mar 20 07:39:01 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 20 Mar 2008 07:39:01 -0700 Subject: [ofa-general] [PATCH][TRIVIAL] opensm/doc/partition-config.txt: Update default file name Message-ID: <1206023941.8099.32.camel@hrosenstock-ws.xsigo.com> opensm/doc/partition-config.txt: Update default file name Signed-off-by: Hal Rosenstock diff --git a/opensm/doc/partition-config.txt b/opensm/doc/partition-config.txt index 7937058..602cc66 100644 --- a/opensm/doc/partition-config.txt +++ b/opensm/doc/partition-config.txt @@ -2,7 +2,7 @@ OpenSM Partition configuration =============================== The default name of OpenSM partitions configuration file is -'/etc/opensm/opensm-partitions.conf'. The default may be changed by +'/etc/opensm/partitions.conf'. The default may be changed by using --Pconfig (-P) option with OpenSM. The default partition will be created by OpenSM unconditionally even From hrosenstock at xsigo.com Thu Mar 20 07:41:40 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 20 Mar 2008 07:41:40 -0700 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <200803201533.47801.bs@q-leap.de> References: <200803201230.40303.bs@q-leap.de> <200803201527.38470.bs@q-leap.de> <1206023375.7332.38.camel@hrosenstock-ws.xsigo.com> <200803201533.47801.bs@q-leap.de> Message-ID: <1206024100.8099.35.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-20 at 15:33 +0100, Bernd Schubert wrote: > On Thursday 20 March 2008 15:29:35 Hal Rosenstock wrote: > > On Thu, 2008-03-20 at 15:27 +0100, Bernd Schubert wrote: > > > On Thursday 20 March 2008 15:12:03 Hal Rosenstock wrote: > > > > On Thu, 2008-03-20 at 13:54 +0100, Bernd Schubert wrote: > > > > > On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > > > > > > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > > > > > > Hello, > > > > > > > > > > > > > > on one of our systems we get a rather huge numbers of > > > > > > > RcvSwRelayErrors. All I find about RcvSwRelayErrors is > > > > > > > > > > > > > > "This counter can increase due to a valid network event" > > > > > > > > > > > > > > But what might cause? > > > > > > > > > > Ooops. This should have been "But what might cause it?" > > > > > > > > > > > Are you running IB multicast (e.g. IPoIB) ? That's the most common > > > > > > cause. > > > > > > > > > > IPoIB is up, but so far only used initially by lustre for initial > > > > > lnet o2ib setup, but then AFAIK not any more. I think some MPI > > > > > stacks/applications also do their intial connection using IPoIB. > > > > > > > > > > But in general, once these connections are established, IPoIB is not > > > > > much used anymore. > > > > > > > > The causes are: > > > > 1. DLID mapping > > > > 2. VL mapping > > > > 3. looping (out port = in port) > > > > > > > > Is your subnet unstable in some way ? Are you using QoS ? > > > > > > We have seen some odd problems with opensm (from ofef-1.2.5) in the past > > > and once only rebooting the switches did help. > > > > You might want to update OpenSM to OFED 1.3 version. > > I won't manage to build new debian packages today, but I will do over Easter. > Hope to also find the time to clean the debian rules a bit, to have it > officially included in Debian. > > But will a new opensm help for these errors? Perhaps; but not knowing more about the cause it's hard to say. It might be interesting to see if there are any errors in your OpenSM log. -- Hal > > > > > Yesterday I started monitoring the the fabric and even though there's not > > > much traffic, I immediately noticed these errors. > > > > Were the counters cleared before you started looking ? > > Yes, sure. > > > Thanks a lot for your help, > Bernd > From hrosenstock at xsigo.com Thu Mar 20 07:45:18 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 20 Mar 2008 07:45:18 -0700 Subject: [ofa-general] OpenSM release notes and Connect-X Message-ID: <1206024318.8099.39.camel@hrosenstock-ws.xsigo.com> Yevgeny, We should get Connect-X added into table 3 below copied from the opensm release notes: Mellanox Device | FW versions --------|----------------------------------------------------------- MT43132 | InfiniScale - fw-43132 5.2.0 (and later) MT47396 | InfiniScale III - fw-47396 0.5.0 (and later) MT23108 | InfiniHost - fw-23108 3.3.2 (and later) MT25204 | InfiniHost III Lx - fw-25204 1.0.1i (and later) MT25208 | InfiniHost III Ex (InfiniHost Mode) - fw-25208 4.6.2 (and later) MT25208 | InfiniHost III Ex (MemFree Mode) - fw-25218 5.0.1 (and later) What are the device number(s) ? Also, it was said that any firmware version would suffice but I've seen: mlx4_core 0000:01:00.0: Installed FW has unsupported command interface revision 1. mlx4_core 0000:01:00.0: (Installed FW version is 2.0.123) mlx4_core 0000:01:00.0: This driver version supports only revisions 2 to 3. mlx4_core 0000:01:00.0: QUERY_FW command failed, aborting. Thanks. -- Hal From bs at q-leap.de Thu Mar 20 08:10:13 2008 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 20 Mar 2008 16:10:13 +0100 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <1206024100.8099.35.camel@hrosenstock-ws.xsigo.com> References: <200803201230.40303.bs@q-leap.de> <200803201533.47801.bs@q-leap.de> <1206024100.8099.35.camel@hrosenstock-ws.xsigo.com> Message-ID: <200803201610.13725.bs@q-leap.de> On Thursday 20 March 2008 15:41:40 Hal Rosenstock wrote: > On Thu, 2008-03-20 at 15:33 +0100, Bernd Schubert wrote: > > On Thursday 20 March 2008 15:29:35 Hal Rosenstock wrote: > > > On Thu, 2008-03-20 at 15:27 +0100, Bernd Schubert wrote: > > > > On Thursday 20 March 2008 15:12:03 Hal Rosenstock wrote: > > > > > On Thu, 2008-03-20 at 13:54 +0100, Bernd Schubert wrote: > > > > > > On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > > > > > > > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > > > > > > > Hello, > > > > > > > > > > > > > > > > on one of our systems we get a rather huge numbers of > > > > > > > > RcvSwRelayErrors. All I find about RcvSwRelayErrors is > > > > > > > > > > > > > > > > "This counter can increase due to a valid network event" > > > > > > > > > > > > > > > > But what might cause? > > > > > > > > > > > > Ooops. This should have been "But what might cause it?" > > > > > > > > > > > > > Are you running IB multicast (e.g. IPoIB) ? That's the most > > > > > > > common cause. > > > > > > > > > > > > IPoIB is up, but so far only used initially by lustre for initial > > > > > > lnet o2ib setup, but then AFAIK not any more. I think some MPI > > > > > > stacks/applications also do their intial connection using IPoIB. > > > > > > > > > > > > But in general, once these connections are established, IPoIB is > > > > > > not much used anymore. > > > > > > > > > > The causes are: > > > > > 1. DLID mapping > > > > > 2. VL mapping > > > > > 3. looping (out port = in port) > > > > > > > > > > Is your subnet unstable in some way ? Are you using QoS ? > > > > > > > > We have seen some odd problems with opensm (from ofef-1.2.5) in the > > > > past and once only rebooting the switches did help. > > > > > > You might want to update OpenSM to OFED 1.3 version. > > > > I won't manage to build new debian packages today, but I will do over > > Easter. Hope to also find the time to clean the debian rules a bit, to > > have it officially included in Debian. > > > > But will a new opensm help for these errors? > > Perhaps; but not knowing more about the cause it's hard to say. It might > be interesting to see if there are any errors in your OpenSM log. Well, these opensm logs are a big mystery for me, I have not the slightest idea, what it wants to tell me with this: osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x0020 GID:0xfe80000000000000,0x000b8cffff002b41 Can I find some doku somewhere or does only reading the source help? Here's the logs from the last day: Mar 19 17:20:53 463683 [44007960] -> SUBNET UP Mar 20 10:22:27 864281 [44007960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x01 num:128 Producer:2 from LID:0x002E TID:0x000000000000001f Mar 20 10:22:27 864533 [44007960] -> osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x002E GID:0xfe80000000000000,0x000b8cffff002b50 Mar 20 10:22:28 153211 [42003960] -> osm_report_notice: Reporting Generic Notice type:3 num:65 from LID:0x001A GID:0xfe80000000000000,0x0002c9020025ae35 Mar 20 10:22:28 153231 [42003960] -> __osm_drop_mgr_remove_port: Removed port with GUID:0x0002c902002587c6 LID range [0xF7,0xF7] of node:MT25408 ConnectX Mellanox Technologies Mar 20 10:22:28 192987 [42003960] -> osm_ucast_mgr_process: null (min-hop) tables configured on all switches Mar 20 10:22:28 270978 [44007960] -> SUBNET UP Mar 20 10:25:50 333350 [44007960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x01 num:128 Producer:2 from LID:0x002E TID:0x0000000000000020 Mar 20 10:25:50 333579 [44007960] -> osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x002E GID:0xfe80000000000000,0x000b8cffff002b50 Mar 20 10:25:50 644817 [42003960] -> osm_report_notice: Reporting Generic Notice type:3 num:64 from LID:0x001A GID:0xfe80000000000000,0x0002c9020025ae35 Mar 20 10:25:50 644840 [42003960] -> __osm_state_mgr_report_new_ports: Discovered new port with GUID:0x0002c902002587c6 LID range [0xF7,0xF7] of node:MT25408 ConnectX Mellanox Technologies Mar 20 10:25:50 679661 [42003960] -> osm_ucast_mgr_process: null (min-hop) tables configured on all switches Mar 20 10:25:50 755437 [41001960] -> SUBNET UP Mar 20 14:24:04 611501 [42003960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x01 num:128 Producer:2 from LID:0x0020 TID:0x0000000000000051 Mar 20 14:24:04 611713 [42003960] -> osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x0020 GID:0xfe80000000000000,0x000b8cffff002b41 Mar 20 14:24:04 913422 [44808960] -> osm_report_notice: Reporting Generic Notice type:3 num:65 from LID:0x001A GID:0xfe80000000000000,0x0002c9020025ae35 Mar 20 14:24:04 913444 [44808960] -> __osm_drop_mgr_remove_port: Removed port with GUID:0x0002c9020025871d LID range [0x8,0x8] of node:MT25408 ConnectX Mellanox Technologies Mar 20 14:24:04 952959 [44808960] -> osm_ucast_mgr_process: null (min-hop) tables configured on all switches Mar 20 14:24:05 027280 [41802960] -> SUBNET UP Mar 20 14:26:49 795337 [41802960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x01 num:128 Producer:2 from LID:0x0020 TID:0x0000000000000052 Mar 20 14:26:49 795578 [41802960] -> osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x0020 GID:0xfe80000000000000,0x000b8cffff002b41 Mar 20 14:26:50 096861 [42804960] -> osm_report_notice: Reporting Generic Notice type:3 num:64 from LID:0x001A GID:0xfe80000000000000,0x0002c9020025ae35 Mar 20 14:26:50 096874 [42804960] -> __osm_state_mgr_report_new_ports: Discovered new port with GUID:0x0002c9020025871d LID range [0x8,0x8] of node:MT25408 ConnectX Mellanox Technologies Mar 20 14:26:50 131620 [42804960] -> osm_ucast_mgr_process: null (min-hop) tables configured on all switches Mar 20 14:26:50 207641 [43806960] -> SUBNET UP Mar 20 14:28:06 751962 [43806960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x04 num:144 Producer:1 from LID:0x0008 TID:0x0000000000000000 Thanks again for your help, Bernd -- Bernd Schubert Q-Leap Networks GmbH From hrosenstock at xsigo.com Thu Mar 20 08:13:02 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 20 Mar 2008 08:13:02 -0700 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <200803201610.13725.bs@q-leap.de> References: <200803201230.40303.bs@q-leap.de> <200803201533.47801.bs@q-leap.de> <1206024100.8099.35.camel@hrosenstock-ws.xsigo.com> <200803201610.13725.bs@q-leap.de> Message-ID: <1206025982.8099.65.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-20 at 16:10 +0100, Bernd Schubert wrote: > On Thursday 20 March 2008 15:41:40 Hal Rosenstock wrote: > > On Thu, 2008-03-20 at 15:33 +0100, Bernd Schubert wrote: > > > On Thursday 20 March 2008 15:29:35 Hal Rosenstock wrote: > > > > On Thu, 2008-03-20 at 15:27 +0100, Bernd Schubert wrote: > > > > > On Thursday 20 March 2008 15:12:03 Hal Rosenstock wrote: > > > > > > On Thu, 2008-03-20 at 13:54 +0100, Bernd Schubert wrote: > > > > > > > On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > > > > > > > > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > on one of our systems we get a rather huge numbers of > > > > > > > > > RcvSwRelayErrors. All I find about RcvSwRelayErrors is > > > > > > > > > > > > > > > > > > "This counter can increase due to a valid network event" > > > > > > > > > > > > > > > > > > But what might cause? > > > > > > > > > > > > > > Ooops. This should have been "But what might cause it?" > > > > > > > > > > > > > > > Are you running IB multicast (e.g. IPoIB) ? That's the most > > > > > > > > common cause. > > > > > > > > > > > > > > IPoIB is up, but so far only used initially by lustre for initial > > > > > > > lnet o2ib setup, but then AFAIK not any more. I think some MPI > > > > > > > stacks/applications also do their intial connection using IPoIB. > > > > > > > > > > > > > > But in general, once these connections are established, IPoIB is > > > > > > > not much used anymore. > > > > > > > > > > > > The causes are: > > > > > > 1. DLID mapping > > > > > > 2. VL mapping > > > > > > 3. looping (out port = in port) > > > > > > > > > > > > Is your subnet unstable in some way ? Are you using QoS ? > > > > > > > > > > We have seen some odd problems with opensm (from ofef-1.2.5) in the > > > > > past and once only rebooting the switches did help. > > > > > > > > You might want to update OpenSM to OFED 1.3 version. > > > > > > I won't manage to build new debian packages today, but I will do over > > > Easter. Hope to also find the time to clean the debian rules a bit, to > > > have it officially included in Debian. > > > > > > But will a new opensm help for these errors? > > > > Perhaps; but not knowing more about the cause it's hard to say. It might > > be interesting to see if there are any errors in your OpenSM log. > > Well, these opensm logs are a big mystery for me, I have not the slightest > idea, what it wants to tell me with this: > > osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x0020 GID:0xfe80000000000000,0x000b8cffff002b41 > > Can I find some doku somewhere or does only reading the source help? AFAIK there's no doc and it takes some combination of looking at the messages and the IB spec and possibly looking at the source as well. > Here's the logs from the last day: > > Mar 19 17:20:53 463683 [44007960] -> SUBNET UP > Mar 20 10:22:27 864281 [44007960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x01 num:128 Producer:2 from LID:0x002E TID:0x000000000000001f > Mar 20 10:22:27 864533 [44007960] -> osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x002E GID:0xfe80000000000000,0x000b8cffff002b50 > Mar 20 10:22:28 153211 [42003960] -> osm_report_notice: Reporting Generic Notice type:3 num:65 from LID:0x001A GID:0xfe80000000000000,0x0002c9020025ae35 > Mar 20 10:22:28 153231 [42003960] -> __osm_drop_mgr_remove_port: Removed port with GUID:0x0002c902002587c6 LID range [0xF7,0xF7] of node:MT25408 ConnectX Mellanox Technologies > Mar 20 10:22:28 192987 [42003960] -> osm_ucast_mgr_process: null (min-hop) tables configured on all switches > Mar 20 10:22:28 270978 [44007960] -> SUBNET UP > Mar 20 10:25:50 333350 [44007960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x01 num:128 Producer:2 from LID:0x002E TID:0x0000000000000020 > Mar 20 10:25:50 333579 [44007960] -> osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x002E GID:0xfe80000000000000,0x000b8cffff002b50 > Mar 20 10:25:50 644817 [42003960] -> osm_report_notice: Reporting Generic Notice type:3 num:64 from LID:0x001A GID:0xfe80000000000000,0x0002c9020025ae35 > Mar 20 10:25:50 644840 [42003960] -> __osm_state_mgr_report_new_ports: Discovered new port with GUID:0x0002c902002587c6 LID range [0xF7,0xF7] of node:MT25408 ConnectX Mellanox Technologies > Mar 20 10:25:50 679661 [42003960] -> osm_ucast_mgr_process: null (min-hop) tables configured on all switches > Mar 20 10:25:50 755437 [41001960] -> SUBNET UP > Mar 20 14:24:04 611501 [42003960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x01 num:128 Producer:2 from LID:0x0020 TID:0x0000000000000051 > Mar 20 14:24:04 611713 [42003960] -> osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x0020 GID:0xfe80000000000000,0x000b8cffff002b41 > Mar 20 14:24:04 913422 [44808960] -> osm_report_notice: Reporting Generic Notice type:3 num:65 from LID:0x001A GID:0xfe80000000000000,0x0002c9020025ae35 > Mar 20 14:24:04 913444 [44808960] -> __osm_drop_mgr_remove_port: Removed port with GUID:0x0002c9020025871d LID range [0x8,0x8] of node:MT25408 ConnectX Mellanox Technologies > Mar 20 14:24:04 952959 [44808960] -> osm_ucast_mgr_process: null (min-hop) tables configured on all switches > Mar 20 14:24:05 027280 [41802960] -> SUBNET UP > Mar 20 14:26:49 795337 [41802960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x01 num:128 Producer:2 from LID:0x0020 TID:0x0000000000000052 > Mar 20 14:26:49 795578 [41802960] -> osm_report_notice: Reporting Generic Notice type:1 num:128 from LID:0x0020 GID:0xfe80000000000000,0x000b8cffff002b41 > Mar 20 14:26:50 096861 [42804960] -> osm_report_notice: Reporting Generic Notice type:3 num:64 from LID:0x001A GID:0xfe80000000000000,0x0002c9020025ae35 > Mar 20 14:26:50 096874 [42804960] -> __osm_state_mgr_report_new_ports: Discovered new port with GUID:0x0002c9020025871d LID range [0x8,0x8] of node:MT25408 ConnectX Mellanox Technologies > Mar 20 14:26:50 131620 [42804960] -> osm_ucast_mgr_process: null (min-hop) tables configured on all switches > Mar 20 14:26:50 207641 [43806960] -> SUBNET UP > Mar 20 14:28:06 751962 [43806960] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x04 num:144 Producer:1 from LID:0x0008 TID:0x0000000000000000 Looks like there is something going on with your ConnectX ports or are these just booting up ? -- Hal > Thanks again for your help, > Bernd > From liu1880 at 163.com Thu Mar 20 08:10:12 2008 From: liu1880 at 163.com (liu1880) Date: Thu, 20 Mar 2008 23:10:12 +0800 (CST) Subject: [ofa-general] =?gbk?b?veLC68n6w/yjrNTsuKPIy8Dg?= Message-ID: <13724197.1428681206025812437.JavaMail.coremail@bj163app35.163.com> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 解码生命 造福人类1.doc.doc Type: application/msword Size: 38912 bytes Desc: not available URL: From 348skydiver28 at jathomas.com Thu Mar 20 08:25:55 2008 From: 348skydiver28 at jathomas.com (Gordon Cramer) Date: Thu, 20 Mar 2008 16:25:55 +0100 Subject: [ofa-general] Hier bekommen Sie die besten Pillen Message-ID: <01c88aa7$1239ab80$08e5194f@348skydiver28> Die Situation mit der Qualitaet der angebotenen Medikamente hat sich mit der Eroeffnung der Onlineapotheken verbessert. Insbesondere handelt es sich um Apotheken, die ganz professionell arbeiten und Ihren Kunden die besten Medikamente anbieten koennen. So eine Onlineapotheke ist auch die EuropeanPharmacy.100%-tige Sicherheit zu den niedrigsten Preisen. http://hyeutg.considerleg.cn/?802535549478Bringen Sie sich Ihre Gesundheit zurueck. Vollkommen anonymer und sicherer Kauf. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at dev.mellanox.co.il Thu Mar 20 08:34:13 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 20 Mar 2008 17:34:13 +0200 Subject: [ofa-general] Re: OpenSM release notes and Connect-X In-Reply-To: <1206024318.8099.39.camel@hrosenstock-ws.xsigo.com> References: <1206024318.8099.39.camel@hrosenstock-ws.xsigo.com> Message-ID: <47E283F5.5060200@dev.mellanox.co.il> Hal Rosenstock wrote: > Yevgeny, > > We should get Connect-X added into table 3 below copied from the opensm > release notes: > > Mellanox > Device | FW versions > --------|----------------------------------------------------------- > MT43132 | InfiniScale - fw-43132 5.2.0 (and later) > MT47396 | InfiniScale III - fw-47396 0.5.0 (and later) > MT23108 | InfiniHost - fw-23108 3.3.2 (and later) > MT25204 | InfiniHost III Lx - fw-25204 1.0.1i (and later) > MT25208 | InfiniHost III Ex (MemFree Mode) - fw-25218 5.0.1 (and later) > MT25208 | InfiniHost III Ex (InfiniHost Mode) - fw-25208 4.6.2 (and later) > > What are the device number(s) ? > Also, it was said that any firmware version would suffice but I've seen: > > mlx4_core 0000:01:00.0: Installed FW has unsupported command interface revision 1. > mlx4_core 0000:01:00.0: (Installed FW version is 2.0.123) > mlx4_core 0000:01:00.0: This driver version supports only revisions 2 to 3. > mlx4_core 0000:01:00.0: QUERY_FW command failed, aborting. According to OFED release notes, it's ConnectX IB - fw-25408 Rev 2.3.000 (and later). I'll check the device IDs and update the OSM release notes. -- Yevgeny > Thanks. > > -- Hal > > From ossrosch at linux.vnet.ibm.com Thu Mar 20 09:56:50 2008 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Thu, 20 Mar 2008 17:56:50 +0100 Subject: [ofa-general] [PATCH] ehca: remove tgid checking Message-ID: <200803201756.52818.ossrosch@linux.vnet.ibm.com> This patch removes pid_t in ehca, as discussed with roland here: http://lkml.org/lkml/2008/3/19/169 It will apply against roland's git tree. Signed-off-by: Hoang-Nam Nguyen --- drivers/infiniband/hw/ehca/ehca_av.c | 31 ------------------ drivers/infiniband/hw/ehca/ehca_classes.h | 2 - drivers/infiniband/hw/ehca/ehca_cq.c | 19 ----------- drivers/infiniband/hw/ehca/ehca_mrmw.c | 32 ------------------- drivers/infiniband/hw/ehca/ehca_pd.c | 11 ------ drivers/infiniband/hw/ehca/ehca_qp.c | 48 ----------------------------- drivers/infiniband/hw/ehca/ehca_uverbs.c | 19 ----------- 7 files changed, 0 insertions(+), 162 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_av.c b/drivers/infiniband/hw/ehca/ehca_av.c index 194c1c3..56735ea 100644 --- a/drivers/infiniband/hw/ehca/ehca_av.c +++ b/drivers/infiniband/hw/ehca/ehca_av.c @@ -41,9 +41,6 @@ * POSSIBILITY OF SUCH DAMAGE. */ - -#include - #include "ehca_tools.h" #include "ehca_iverbs.h" #include "hcp_if.h" @@ -170,17 +167,8 @@ int ehca_modify_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr) { struct ehca_av *av; struct ehca_ud_av new_ehca_av; - struct ehca_pd *my_pd = container_of(ah->pd, struct ehca_pd, ib_pd); struct ehca_shca *shca = container_of(ah->pd->device, struct ehca_shca, ib_device); - u32 cur_pid = current->tgid; - - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ah->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } memset(&new_ehca_av, 0, sizeof(new_ehca_av)); new_ehca_av.sl = ah_attr->sl; @@ -242,15 +230,6 @@ int ehca_modify_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr) int ehca_query_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr) { struct ehca_av *av = container_of(ah, struct ehca_av, ib_ah); - struct ehca_pd *my_pd = container_of(ah->pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; - - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ah->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } memcpy(&ah_attr->grh.dgid, &av->av.grh.word_3, sizeof(ah_attr->grh.dgid)); @@ -273,16 +252,6 @@ int ehca_query_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr) int ehca_destroy_ah(struct ib_ah *ah) { - struct ehca_pd *my_pd = container_of(ah->pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; - - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ah->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } - kmem_cache_free(av_cache, container_of(ah, struct ehca_av, ib_ah)); return 0; diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index 92cce8a..0d13fe0 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h @@ -132,7 +132,6 @@ struct ehca_shca { struct ehca_pd { struct ib_pd ib_pd; struct ipz_pd fw_pd; - u32 ownpid; /* small queue mgmt */ struct mutex lock; struct list_head free[2]; @@ -215,7 +214,6 @@ struct ehca_cq { atomic_t nr_events; /* #events seen */ wait_queue_head_t wait_completion; spinlock_t task_lock; - u32 ownpid; /* mmap counter for resources mapped into user space */ u32 mm_count_queue; u32 mm_count_galpa; diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c index 0467c15..ec0cfcf 100644 --- a/drivers/infiniband/hw/ehca/ehca_cq.c +++ b/drivers/infiniband/hw/ehca/ehca_cq.c @@ -43,8 +43,6 @@ * POSSIBILITY OF SUCH DAMAGE. */ -#include - #include "ehca_iverbs.h" #include "ehca_classes.h" #include "ehca_irq.h" @@ -148,7 +146,6 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector, spin_lock_init(&my_cq->task_lock); atomic_set(&my_cq->nr_events, 0); init_waitqueue_head(&my_cq->wait_completion); - my_cq->ownpid = current->tgid; cq = &my_cq->ib_cq; @@ -320,7 +317,6 @@ int ehca_destroy_cq(struct ib_cq *cq) struct ehca_shca *shca = container_of(device, struct ehca_shca, ib_device); struct ipz_adapter_handle adapter_handle = shca->ipz_hca_handle; - u32 cur_pid = current->tgid; unsigned long flags; if (cq->uobject) { @@ -329,12 +325,6 @@ int ehca_destroy_cq(struct ib_cq *cq) "user space cq_num=%x", my_cq->cq_number); return -EINVAL; } - if (my_cq->ownpid != cur_pid) { - ehca_err(device, "Invalid caller pid=%x ownpid=%x " - "cq_num=%x", - cur_pid, my_cq->ownpid, my_cq->cq_number); - return -EINVAL; - } } /* @@ -374,15 +364,6 @@ int ehca_destroy_cq(struct ib_cq *cq) int ehca_resize_cq(struct ib_cq *cq, int cqe, struct ib_udata *udata) { - struct ehca_cq *my_cq = container_of(cq, struct ehca_cq, ib_cq); - u32 cur_pid = current->tgid; - - if (cq->uobject && my_cq->ownpid != cur_pid) { - ehca_err(cq->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_cq->ownpid); - return -EINVAL; - } - /* TODO: proper resize needs to be done */ ehca_err(cq->device, "not implemented yet"); diff --git a/drivers/infiniband/hw/ehca/ehca_mrmw.c b/drivers/infiniband/hw/ehca/ehca_mrmw.c index 5e99c45..f26997f 100644 --- a/drivers/infiniband/hw/ehca/ehca_mrmw.c +++ b/drivers/infiniband/hw/ehca/ehca_mrmw.c @@ -40,8 +40,6 @@ * POSSIBILITY OF SUCH DAMAGE. */ -#include - #include #include "ehca_iverbs.h" @@ -419,7 +417,6 @@ int ehca_rereg_phys_mr(struct ib_mr *mr, struct ehca_shca *shca = container_of(mr->device, struct ehca_shca, ib_device); struct ehca_mr *e_mr = container_of(mr, struct ehca_mr, ib.ib_mr); - struct ehca_pd *my_pd = container_of(mr->pd, struct ehca_pd, ib_pd); u64 new_size; u64 *new_start; u32 new_acl; @@ -429,15 +426,6 @@ int ehca_rereg_phys_mr(struct ib_mr *mr, u32 num_kpages = 0; u32 num_hwpages = 0; struct ehca_mr_pginfo pginfo; - u32 cur_pid = current->tgid; - - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - (my_pd->ownpid != cur_pid)) { - ehca_err(mr->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - ret = -EINVAL; - goto rereg_phys_mr_exit0; - } if (!(mr_rereg_mask & IB_MR_REREG_TRANS)) { /* TODO not supported, because PHYP rereg hCall needs pages */ @@ -577,19 +565,9 @@ int ehca_query_mr(struct ib_mr *mr, struct ib_mr_attr *mr_attr) struct ehca_shca *shca = container_of(mr->device, struct ehca_shca, ib_device); struct ehca_mr *e_mr = container_of(mr, struct ehca_mr, ib.ib_mr); - struct ehca_pd *my_pd = container_of(mr->pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; unsigned long sl_flags; struct ehca_mr_hipzout_parms hipzout; - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - (my_pd->ownpid != cur_pid)) { - ehca_err(mr->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - ret = -EINVAL; - goto query_mr_exit0; - } - if ((e_mr->flags & EHCA_MR_FLAG_FMR)) { ehca_err(mr->device, "not supported for FMR, mr=%p e_mr=%p " "e_mr->flags=%x", mr, e_mr, e_mr->flags); @@ -634,16 +612,6 @@ int ehca_dereg_mr(struct ib_mr *mr) struct ehca_shca *shca = container_of(mr->device, struct ehca_shca, ib_device); struct ehca_mr *e_mr = container_of(mr, struct ehca_mr, ib.ib_mr); - struct ehca_pd *my_pd = container_of(mr->pd, struct ehca_pd, ib_pd); - u32 cur_pid = current->tgid; - - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - (my_pd->ownpid != cur_pid)) { - ehca_err(mr->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - ret = -EINVAL; - goto dereg_mr_exit0; - } if ((e_mr->flags & EHCA_MR_FLAG_FMR)) { ehca_err(mr->device, "not supported for FMR, mr=%p e_mr=%p " diff --git a/drivers/infiniband/hw/ehca/ehca_pd.c b/drivers/infiniband/hw/ehca/ehca_pd.c index 43bcf08..2fe5548 100644 --- a/drivers/infiniband/hw/ehca/ehca_pd.c +++ b/drivers/infiniband/hw/ehca/ehca_pd.c @@ -38,8 +38,6 @@ * POSSIBILITY OF SUCH DAMAGE. */ -#include - #include "ehca_tools.h" #include "ehca_iverbs.h" @@ -58,7 +56,6 @@ struct ib_pd *ehca_alloc_pd(struct ib_device *device, return ERR_PTR(-ENOMEM); } - pd->ownpid = current->tgid; for (i = 0; i < 2; i++) { INIT_LIST_HEAD(&pd->free[i]); INIT_LIST_HEAD(&pd->full[i]); @@ -85,18 +82,10 @@ struct ib_pd *ehca_alloc_pd(struct ib_device *device, int ehca_dealloc_pd(struct ib_pd *pd) { - u32 cur_pid = current->tgid; struct ehca_pd *my_pd = container_of(pd, struct ehca_pd, ib_pd); int i, leftovers = 0; struct ipz_small_queue_page *page, *tmp; - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(pd->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } - for (i = 0; i < 2; i++) { list_splice(&my_pd->full[i], &my_pd->free[i]); list_for_each_entry_safe(page, tmp, &my_pd->free[i], list) { diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index 1012f15..a9fd419 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -43,9 +43,6 @@ * POSSIBILITY OF SUCH DAMAGE. */ - -#include - #include "ehca_classes.h" #include "ehca_tools.h" #include "ehca_qes.h" @@ -1526,16 +1523,6 @@ int ehca_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int attr_mask, struct ehca_shca *shca = container_of(ibqp->device, struct ehca_shca, ib_device); struct ehca_qp *my_qp = container_of(ibqp, struct ehca_qp, ib_qp); - struct ehca_pd *my_pd = container_of(my_qp->ib_qp.pd, struct ehca_pd, - ib_pd); - u32 cur_pid = current->tgid; - - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ibqp->pd->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } /* The if-block below caches qp_attr to be modified for GSI and SMI * qps during the initialization by ib_mad. When the respective port @@ -1636,23 +1623,13 @@ int ehca_query_qp(struct ib_qp *qp, int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr) { struct ehca_qp *my_qp = container_of(qp, struct ehca_qp, ib_qp); - struct ehca_pd *my_pd = container_of(my_qp->ib_qp.pd, struct ehca_pd, - ib_pd); struct ehca_shca *shca = container_of(qp->device, struct ehca_shca, ib_device); struct ipz_adapter_handle adapter_handle = shca->ipz_hca_handle; struct hcp_modify_qp_control_block *qpcb; - u32 cur_pid = current->tgid; int cnt, ret = 0; u64 h_ret; - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(qp->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } - if (qp_attr_mask & QP_ATTR_QUERY_NOT_SUPPORTED) { ehca_err(qp->device, "Invalid attribute mask " "ehca_qp=%p qp_num=%x qp_attr_mask=%x ", @@ -1797,8 +1774,6 @@ int ehca_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr, { struct ehca_qp *my_qp = container_of(ibsrq, struct ehca_qp, ib_srq); - struct ehca_pd *my_pd = - container_of(ibsrq->pd, struct ehca_pd, ib_pd); struct ehca_shca *shca = container_of(ibsrq->pd->device, struct ehca_shca, ib_device); struct hcp_modify_qp_control_block *mqpcb; @@ -1806,14 +1781,6 @@ int ehca_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr, u64 h_ret; int ret = 0; - u32 cur_pid = current->tgid; - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(ibsrq->pd->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } - mqpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL); if (!mqpcb) { ehca_err(ibsrq->device, "Could not get zeroed page for mqpcb " @@ -1864,22 +1831,13 @@ modify_srq_exit0: int ehca_query_srq(struct ib_srq *srq, struct ib_srq_attr *srq_attr) { struct ehca_qp *my_qp = container_of(srq, struct ehca_qp, ib_srq); - struct ehca_pd *my_pd = container_of(srq->pd, struct ehca_pd, ib_pd); struct ehca_shca *shca = container_of(srq->device, struct ehca_shca, ib_device); struct ipz_adapter_handle adapter_handle = shca->ipz_hca_handle; struct hcp_modify_qp_control_block *qpcb; - u32 cur_pid = current->tgid; int ret = 0; u64 h_ret; - if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context && - my_pd->ownpid != cur_pid) { - ehca_err(srq->device, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } - qpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL); if (!qpcb) { ehca_err(srq->device, "Out of memory for qpcb " @@ -1919,7 +1877,6 @@ static int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp, struct ehca_pd *my_pd = container_of(my_qp->ib_qp.pd, struct ehca_pd, ib_pd); struct ehca_sport *sport = &shca->sport[my_qp->init_attr.port_num - 1]; - u32 cur_pid = current->tgid; u32 qp_num = my_qp->real_qp_num; int ret; u64 h_ret; @@ -1934,11 +1891,6 @@ static int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp, "user space qp_num=%x", qp_num); return -EINVAL; } - if (my_pd->ownpid != cur_pid) { - ehca_err(dev, "Invalid caller pid=%x ownpid=%x", - cur_pid, my_pd->ownpid); - return -EINVAL; - } } if (my_qp->send_cq) { diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c index 5234d6c..1b07f2b 100644 --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c @@ -40,8 +40,6 @@ * POSSIBILITY OF SUCH DAMAGE. */ -#include - #include "ehca_classes.h" #include "ehca_iverbs.h" #include "ehca_mrmw.h" @@ -253,11 +251,9 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) u32 idr_handle = fileoffset & 0x1FFFFFF; u32 q_type = (fileoffset >> 27) & 0x1; /* CQ, QP,... */ u32 rsrc_type = (fileoffset >> 25) & 0x3; /* sq,rq,cmnd_window */ - u32 cur_pid = current->tgid; u32 ret; struct ehca_cq *cq; struct ehca_qp *qp; - struct ehca_pd *pd; struct ib_uobject *uobject; switch (q_type) { @@ -270,13 +266,6 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) if (!cq) return -EINVAL; - if (cq->ownpid != cur_pid) { - ehca_err(cq->ib_cq.device, - "Invalid caller pid=%x ownpid=%x", - cur_pid, cq->ownpid); - return -ENOMEM; - } - if (!cq->ib_cq.uobject || cq->ib_cq.uobject->context != context) return -EINVAL; @@ -298,14 +287,6 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) if (!qp) return -EINVAL; - pd = container_of(qp->ib_qp.pd, struct ehca_pd, ib_pd); - if (pd->ownpid != cur_pid) { - ehca_err(qp->ib_qp.device, - "Invalid caller pid=%x ownpid=%x", - cur_pid, pd->ownpid); - return -ENOMEM; - } - uobject = IS_SRQ(qp) ? qp->ib_srq.uobject : qp->ib_qp.uobject; if (!uobject || uobject->context != context) return -EINVAL; -- 1.5.2 From weiny2 at llnl.gov Thu Mar 20 10:23:47 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 20 Mar 2008 10:23:47 -0700 Subject: [ofa-general] RcvSwRelayErrors In-Reply-To: <200803201354.53826.bs@q-leap.de> References: <200803201230.40303.bs@q-leap.de> <1206016056.11393.561.camel@hrosenstock-ws.xsigo.com> <200803201354.53826.bs@q-leap.de> Message-ID: <20080320102347.749d38e6.weiny2@llnl.gov> Bernd, On Thu, 20 Mar 2008 13:54:53 +0100 Bernd Schubert wrote: > On Thursday 20 March 2008 13:27:36 Hal Rosenstock wrote: > > On Thu, 2008-03-20 at 12:30 +0100, Bernd Schubert wrote: > > > Hello, > > > > > > on one of our systems we get a rather huge numbers of RcvSwRelayErrors. > > > All I find about RcvSwRelayErrors is > > > > > > "This counter can increase due to a valid network event" > > > > > > But what might cause? > > Ooops. This should have been "But what might cause it?" > > > > > Are you running IB multicast (e.g. IPoIB) ? That's the most common > > cause. > > IPoIB is up, but so far only used initially by lustre for initial lnet o2ib > setup, but then AFAIK not any more. I think some MPI stacks/applications also > do their intial connection using IPoIB. > > But in general, once these connections are established, IPoIB is not much used > anymore. > Just FYI, we completely ignore these errors here. Perhaps not what we should do but we run a number of things over IPoIB. I suggest you try to clear the errors, run for ~1 hour and then check them. Could you report back an approximate error rate using this method? Ira From rajouri.jammu at gmail.com Thu Mar 20 10:44:59 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Thu, 20 Mar 2008 10:44:59 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <47E267DA.9010102@mellanox.co.il> References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> <3307cdf90803192334i5f353824u8fd8796aea7b730d@mail.gmail.com> <47E267DA.9010102@mellanox.co.il> Message-ID: <3307cdf90803201044p4a3f81e5p32984b6caa817440@mail.gmail.com> My current HCA info is: vendor_part_id: 25208 hw_ver: 0xA0 board_id: MT_0200000001 >From the firmware_table, would the following f/w be compatible to the above card? Also, I presume I will be able to go back to the original f/w? fw-25218-5_3_000-MHGA28-XTC_A1-A2.bin.zip MHGA28-XTCRev A1/A2:MT_0370110002 MemFree PCI Express x8 Tall Bracket, RoHS-R5 HCA Card On Thu, Mar 20, 2008 at 6:34 AM, Tziporet Koren wrote: > Rajouri Jammu wrote: > > > > > > That's good news. I will try and get a mem-free HCA and test with it. > > > If its Arbel card you just need to burn it with different FW for mem-free > For FW release please see > http://www.mellanox.com/support/firmware_table_IH3Ex.php > > Tziporet > -------------- next part -------------- An HTML attachment was scrubbed... URL: From swise at opengridcomputing.com Thu Mar 20 12:07:02 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 20 Mar 2008 14:07:02 -0500 Subject: [ofa-general] [PATCH/RFC 1/2] IB/core: Add support for "send with invalidate" work requests In-Reply-To: References: Message-ID: <47E2B5D6.3040605@opengridcomputing.com> Roland Dreier wrote: > Add an IB_SEND_INVALIDATE send flag that can be used to mark a "send > with invalidate" work request as defined in the iWARP verbs and the > InfiniBand base memory management extensions. Also add a new > "invalidate" structure to the struct ib_send_wr.wr union that can be > used to pass in an R_Key/STag to be invalidated. Add this new > structure to struct ib_uverbs_send_wr and add code to copy the rkey > field in ib_uverbs_post_send(). > > Also, move the existing IB_DEVICE_SEND_W_INV flag to a new bit, since > the iWARP drivers currently in the tree set the bit. The amso1100 > driver at least will silently fail to honor the IB_SEND_INVALIDATE bit > if passed in as part of userspace send requests (since it does not > implement kernel bypass work request queueing). Remove the flag from > all existing drivers that set it until we know which ones are OK. > > The values chosen for the new flags are not consecutive to avoid > clashing with flags defined in the IPoIB large send offload and XRC > patches, which are not merged yet but which are already in use and are > likely to be merged soon. > > This resurrects a patch sent long ago by Mikkel Hagen . > > Signed-off-by: Roland Dreier > --- > I tried to avoid clashes with any flags values defined in OFED 1.3, > just to make things a little simpler for users. However a double > check that I succeeded in avoiding already-used values would be good. > > Also I think redefining the IB_DEVICE_SEND_W_INV to a new value avoids > all the backwards and forwards compatibility problems that were a > sticking point when this idea originally appeared. > > Anyway if this looks good to people I will merge it for 2.6.26. > > Looks ok to me. Steve. From swise at opengridcomputing.com Thu Mar 20 12:09:11 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 20 Mar 2008 14:09:11 -0500 Subject: [ofa-general] [PATCH/RFC 2/2] RDMA/amso1100: Add support for "send with invalidate" work requests In-Reply-To: References: Message-ID: <47E2B657.2080908@opengridcomputing.com> Roland Dreier wrote: > Handle work requests that have the IB_SEND_INVALIDATE flag set. > > This resurrects a patch sent long ago by Mikkel Hagen . > > Signed-off-by: Roland Dreier > Looks ok to me. Steve. From rdreier at cisco.com Thu Mar 20 12:40:34 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 20 Mar 2008 12:40:34 -0700 Subject: [ofa-general] Re: [PATCH] ehca: remove tgid checking In-Reply-To: <200803201756.52818.ossrosch@linux.vnet.ibm.com> (Stefan Roscher's message of "Thu, 20 Mar 2008 17:56:50 +0100") References: <200803201756.52818.ossrosch@linux.vnet.ibm.com> Message-ID: thanks for the quick patch, applied! From sashak at voltaire.com Thu Mar 20 13:00:08 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 20 Mar 2008 20:00:08 +0000 Subject: [ofa-general] Re: [PATCH] infiniband-diags/Makefile.am: fix 'make install' In-Reply-To: <47E2336C.4060806@dev.mellanox.co.il> References: <47E2336C.4060806@dev.mellanox.co.il> Message-ID: <20080320200008.GB18860@sashak.voltaire.com> On 11:50 Thu 20 Mar , Yevgeny Kliteynik wrote: > When the code compiled not from the source dir itself, 'make install' > fails. Fixing the wrong file location. > > I'd like to see this fix in ofed_1_3 as well, but it's your call. This is really nice fix, but I don't think that it is critical enough for OFED. > Signed-off-by: Yevgeny Kliteynik Please rebase your patches against master branch. So I will not need to fix this manually. Applied. Thanks. Sasha From sashak at voltaire.com Thu Mar 20 13:03:02 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 20 Mar 2008 20:03:02 +0000 Subject: [ofa-general] Re: [PATCH] infiniband-diags/saquery: print SL in MCast groups In-Reply-To: <47E235D1.1060708@dev.mellanox.co.il> References: <47E235D1.1060708@dev.mellanox.co.il> Message-ID: <20080320200302.GC18860@sashak.voltaire.com> On 12:00 Thu 20 Mar , Yevgeny Kliteynik wrote: > Printing SL in the MCast groups query. > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. > --- > infiniband-diags/src/saquery.c | 7 ++++++- > 1 files changed, 6 insertions(+), 1 deletions(-) > > diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c > index 1b9e2d3..f515a9f 100644 > --- a/infiniband-diags/src/saquery.c > +++ b/infiniband-diags/src/saquery.c > @@ -325,6 +325,9 @@ static void dump_portinfo_record(void *data) > static void > print_multicast_group_record(ib_member_rec_t *p_mcmr) > { > + No need extra empty line here. Sasha From sashak at voltaire.com Thu Mar 20 13:04:02 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 20 Mar 2008 20:04:02 +0000 Subject: [ofa-general] Re: [PATCH][TRIVIAL] OpenSM release notes: Add byacc as alternative to bison for qos parser In-Reply-To: <1206023695.8099.28.camel@hrosenstock-ws.xsigo.com> References: <1206023695.8099.28.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080320200402.GD18860@sashak.voltaire.com> On 07:34 Thu 20 Mar , Hal Rosenstock wrote: > OpenSM release notes: Add byacc as alternative to bison for qos parser > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From sashak at voltaire.com Thu Mar 20 13:04:47 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 20 Mar 2008 20:04:47 +0000 Subject: [ofa-general] Re: [PATCH][TRIVIAL] opensm/doc/partition-config.txt: Update default file name In-Reply-To: <1206023941.8099.32.camel@hrosenstock-ws.xsigo.com> References: <1206023941.8099.32.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080320200447.GE18860@sashak.voltaire.com> On 07:39 Thu 20 Mar , Hal Rosenstock wrote: > opensm/doc/partition-config.txt: Update default file name > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From HNGUYEN at de.ibm.com Thu Mar 20 12:59:59 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Thu, 20 Mar 2008 20:59:59 +0100 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: Message-ID: Hi Roland and Eli! >+enum qp_create_flags { >+ QP_CREATE_LSO = 1 << 0, >+}; Our ehca2 supports sort of low-latency QP for UD and RC. It would be great if we can e.g. enhance above enum like this QP_CREATE_LL = 1 << 1 If you agree with, I'll create a patch. For kernel space the changes within ehca should not be that much. Thx Nam From HNGUYEN at de.ibm.com Thu Mar 20 12:59:59 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Thu, 20 Mar 2008 20:59:59 +0100 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: Message-ID: Hi Roland and Eli! >+enum qp_create_flags { >+ QP_CREATE_LSO = 1 << 0, >+}; Our ehca2 supports sort of low-latency QP for UD and RC. It would be great if we can e.g. enhance above enum like this QP_CREATE_LL = 1 << 1 If you agree with, I'll create a patch. For kernel space the changes within ehca should not be that much. Thx Nam From lindahlsheena at successmanifesto.com Thu Mar 20 14:11:08 2008 From: lindahlsheena at successmanifesto.com (University Degree) Date: Thu, 20 Mar 2008 18:11:08 -0300 Subject: [ofa-general] University Degree with no efforts Message-ID: <611369363.69874344602384@successmanifesto.com> 1-410-230-1849 University Degree These are real, genuine degrees that include Bachelors, Masters, MBA and Doctorate Degrees. Confidentiality Assured 1-410-230-1849 24 hours a day, 7 days a week including Sundays and Holidays -------------- next part -------------- An HTML attachment was scrubbed... URL: From gstreiff at NetEffect.com Thu Mar 20 14:51:31 2008 From: gstreiff at NetEffect.com (Glenn Streiff) Date: Thu, 20 Mar 2008 16:51:31 -0500 Subject: [ofa-general] RE: [PATCH 3/3] Infiniband: don't use task_struct.tgid in make_cm_node() In-Reply-To: Message-ID: <5E701717F2B2ED4EA60F87C8AA57B7CC0795008A@venom2> Acked-by Glenn Streiff > From: Roland Dreier [mailto:rdreier at cisco.com] > The patch > below compiles and links fine for me, so is the simplest fix? (I > don't think hardware is looking at session_id behind our back, is it?) > > diff --git a/drivers/infiniband/hw/nes/nes_cm.c > b/drivers/infiniband/hw/nes/nes_cm.c > index c1a85f3..31a28e9 100644 > --- a/drivers/infiniband/hw/nes/nes_cm.c > +++ b/drivers/infiniband/hw/nes/nes_cm.c > @@ -365,7 +365,6 @@ static void print_core(struct nes_cm_core *core) > if (!core) > return; > nes_debug(NES_DBG_CM, > "---------------------------------------------\n"); > - nes_debug(NES_DBG_CM, "Session ID : %u \n", > atomic_read(&core->session_id)); > > nes_debug(NES_DBG_CM, "State : %u \n", core->state); > > @@ -1100,8 +1099,6 @@ static struct nes_cm_node > *make_cm_node(struct nes_cm_core *cm_core, > cm_node->tcp_cntxt.rcv_nxt = 0; > /* get a unique session ID , add thread_id to an > upcounter to handle race */ > atomic_inc(&cm_core->node_cnt); > - atomic_inc(&cm_core->session_id); > - cm_node->session_id = > (u32)(atomic_read(&cm_core->session_id) + current->tgid); > cm_node->conn_type = cm_info->conn_type; > cm_node->apbvt_set = 0; > cm_node->accept_pend = 0; > @@ -1628,9 +1625,7 @@ static struct nes_cm_listener > *mini_cm_listen(struct nes_cm_core *cm_core, > listener->cm_core = cm_core; > listener->nesvnic = nesvnic; > atomic_inc(&cm_core->node_cnt); > - atomic_inc(&cm_core->session_id); > > - listener->session_id = > (u32)(atomic_read(&cm_core->session_id) + current->tgid); > listener->conn_type = cm_info->conn_type; > listener->backlog = cm_info->backlog; > listener->listener_state = NES_CM_LISTENER_ACTIVE_STATE; > @@ -1943,7 +1938,6 @@ static struct nes_cm_core > *nes_cm_alloc_core(void) > cm_core->state = NES_CM_STATE_INITED; > cm_core->free_tx_pkt_max = NES_CM_DEFAULT_FREE_PKTS; > > - atomic_set(&cm_core->session_id, 0); > atomic_set(&cm_core->events_posted, 0); > > /* init the packet lists */ > diff --git a/drivers/infiniband/hw/nes/nes_cm.h > b/drivers/infiniband/hw/nes/nes_cm.h > index 980fb67..7717cb2 100644 > --- a/drivers/infiniband/hw/nes/nes_cm.h > +++ b/drivers/infiniband/hw/nes/nes_cm.h > @@ -225,7 +225,6 @@ enum nes_cm_listener_state { > > struct nes_cm_listener { > struct list_head list; > - u64 session_id; > struct nes_cm_core *cm_core; > u8 loc_mac[ETH_ALEN]; > nes_addr_t loc_addr; > @@ -242,7 +241,6 @@ struct nes_cm_listener { > > /* per connection node and node state information */ > struct nes_cm_node { > - u64 session_id; > u32 hashkey; > > nes_addr_t loc_addr, rem_addr; > @@ -327,7 +325,6 @@ struct nes_cm_event { > > struct nes_cm_core { > enum nes_cm_node_state state; > - atomic_t session_id; > > atomic_t listen_node_cnt; > struct nes_cm_node listen_list; > From gstreiff at neteffect.com Thu Mar 20 17:55:30 2008 From: gstreiff at neteffect.com (gstreiff at neteffect.com) Date: Thu, 20 Mar 2008 19:55:30 -0500 Subject: [ofa-general] [PATCH] RDMA/nes: Fix MSS calculation on RDMA path Message-ID: <200803210055.m2L0tUjU031839@velma.neteffect.com> From: Chien Tung Fix the calculation of the MSS for RDMA connections: we need to allow space in frames for a VLAN tag too. Signed-off-by: Chien Tung Signed-off-by: Glenn Streiff --- drivers/infiniband/hw/nes/nes_cm.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c index d68aa0d..81c8e46 100644 --- a/drivers/infiniband/hw/nes/nes_cm.c +++ b/drivers/infiniband/hw/nes/nes_cm.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -1072,7 +1073,7 @@ static struct nes_cm_node *make_cm_node(struct nes_cm_core *cm_core, ts = current_kernel_time(); cm_node->tcp_cntxt.loc_seq_num = htonl(ts.tv_nsec); cm_node->tcp_cntxt.mss = nesvnic->max_frame_size - sizeof(struct iphdr) - - sizeof(struct tcphdr) - ETH_HLEN; + sizeof(struct tcphdr) - ETH_HLEN - VLAN_HLEN; cm_node->tcp_cntxt.rcv_nxt = 0; /* get a unique session ID , add thread_id to an upcounter to handle race */ atomic_inc(&cm_core->node_cnt); From ralph.campbell at qlogic.com Thu Mar 20 17:17:35 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:17:35 -0700 Subject: [ofa-general] [PATCH 0/10] IB/ipath -- patches in for-roland for 2.6.26 Message-ID: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> The following patches fix a byte order compile warning in the ib_ipath driver and some changes to prepare for the DDR HCA code to be submitted soon. I think these are appropriate for 2.6.26 although if you want to put the small fixes into 2.6.25, I won't object :-). These can also be pulled into Roland's infiniband.git for-2.6.26 repo using: git pull git://git.qlogic.com/ipath-linux-2.6 for-roland From ralph.campbell at qlogic.com Thu Mar 20 17:17:40 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:17:40 -0700 Subject: [ofa-general] [PATCH 01/10] IB/ipath - fix a byte order compile check warning In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001740.13431.27631.stgit@eng-46.mv.qlogic.com> This patch fixes a compiler warning when the driver is compiled with: make M=drivers/infiniband/hw/ipath C=2 CF=-D__CHECK_ENDIAN__ Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_intr.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_intr.c b/drivers/infiniband/hw/ipath/ipath_intr.c index 3b89952..d12dfad 100644 --- a/drivers/infiniband/hw/ipath/ipath_intr.c +++ b/drivers/infiniband/hw/ipath/ipath_intr.c @@ -798,7 +798,7 @@ static int handle_errors(struct ipath_devdata *dd, ipath_err_t errs) void ipath_clear_freeze(struct ipath_devdata *dd) { int i, im; - __le64 val; + u64 val; unsigned long flags; /* disable error interrupts, to avoid confusion */ @@ -835,8 +835,8 @@ void ipath_clear_freeze(struct ipath_devdata *dd) /* deal with 6110 chip bug */ im = i > 3 ? i ^ 1 : i; val = ipath_read_kreg64(dd, (0x1000 / sizeof(u64)) + im); - dd->ipath_pioavailregs_dma[i] = dd->ipath_pioavailshadow[i] - = le64_to_cpu(val); + dd->ipath_pioavailregs_dma[i] = cpu_to_le64(val); + dd->ipath_pioavailshadow[i] = val; } /* From ralph.campbell at qlogic.com Thu Mar 20 17:17:45 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:17:45 -0700 Subject: [ofa-general] [PATCH 02/10] IB/ipath - fix error recovery for send buffer status after chip freeze mode In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001745.13431.95605.stgit@eng-46.mv.qlogic.com> The error recovery code for updating the driver's cached status information for which send buffers are busy or free wasn't updated for IBA7220. It should be similar to the initialization code in enable_chip(). Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_intr.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_intr.c b/drivers/infiniband/hw/ipath/ipath_intr.c index d12dfad..ed2a227 100644 --- a/drivers/infiniband/hw/ipath/ipath_intr.c +++ b/drivers/infiniband/hw/ipath/ipath_intr.c @@ -833,7 +833,8 @@ void ipath_clear_freeze(struct ipath_devdata *dd) */ for (i = 0; i < dd->ipath_pioavregs; i++) { /* deal with 6110 chip bug */ - im = i > 3 ? i ^ 1 : i; + im = (i > 3 && (dd->ipath_flags & IPATH_SWAP_PIOBUFS)) ? + i ^ 1 : i; val = ipath_read_kreg64(dd, (0x1000 / sizeof(u64)) + im); dd->ipath_pioavailregs_dma[i] = cpu_to_le64(val); dd->ipath_pioavailshadow[i] = val; From ralph.campbell at qlogic.com Thu Mar 20 17:17:50 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:17:50 -0700 Subject: [ofa-general] [PATCH 03/10] IB/ipath - fix link up LED display In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001750.13431.53716.stgit@eng-46.mv.qlogic.com> From: Arthur Jones The check for link up was incorrect, thus setting the LED display inconsistently with the link state. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_iba6120.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_iba6120.c b/drivers/infiniband/hw/ipath/ipath_iba6120.c index a9fc804..f1447be 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6120.c +++ b/drivers/infiniband/hw/ipath/ipath_iba6120.c @@ -853,7 +853,7 @@ static void ipath_setup_pe_setextled(struct ipath_devdata *dd, u64 lst, extctl = dd->ipath_extctrl & ~(INFINIPATH_EXTC_LED1PRIPORT_ON | INFINIPATH_EXTC_LED2PRIPORT_ON); - if (ltst & INFINIPATH_IBCS_LT_STATE_LINKUP) + if (ltst == INFINIPATH_IBCS_LT_STATE_LINKUP) extctl |= INFINIPATH_EXTC_LED2PRIPORT_ON; if (lst == INFINIPATH_IBCS_L_STATE_ACTIVE) extctl |= INFINIPATH_EXTC_LED1PRIPORT_ON; From ralph.campbell at qlogic.com Thu Mar 20 17:17:55 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:17:55 -0700 Subject: [ofa-general] [PATCH 04/10] IB/ipath - Don't try to handle freeze mode HW errors if in diagnostic mode In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001755.13431.92225.stgit@eng-46.mv.qlogic.com> Don't try to handle freeze mode HW errors if the driver is in diagnostic mode since some tests can cause errors that shouldn't be processed. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_iba6120.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_iba6120.c b/drivers/infiniband/hw/ipath/ipath_iba6120.c index f1447be..5206b12 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6120.c +++ b/drivers/infiniband/hw/ipath/ipath_iba6120.c @@ -481,7 +481,7 @@ static void ipath_pe_handle_hwerrors(struct ipath_devdata *dd, char *msg, (hwerrs & ~dd->ipath_hwe_bitsextant)); ctrl = ipath_read_kreg32(dd, dd->ipath_kregs->kr_control); - if (ctrl & INFINIPATH_C_FREEZEMODE) { + if ((ctrl & INFINIPATH_C_FREEZEMODE) && !ipath_diag_inuse) { /* * parity errors in send memory are recoverable, * just cancel the send (if indicated in * sendbuffererror), From ralph.campbell at qlogic.com Thu Mar 20 17:18:01 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:18:01 -0700 Subject: [ofa-general] [PATCH 05/10] IB/ipath - Make debug error message match the constraint that is checked for In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001800.13431.14233.stgit@eng-46.mv.qlogic.com> Make debug error message match the constraint that is checked for. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_iba6120.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_iba6120.c b/drivers/infiniband/hw/ipath/ipath_iba6120.c index 5206b12..ce0f40f 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6120.c +++ b/drivers/infiniband/hw/ipath/ipath_iba6120.c @@ -1275,7 +1275,7 @@ static void ipath_pe_put_tid(struct ipath_devdata *dd, u64 __iomem *tidptr, if (pa != dd->ipath_tidinvalid) { if (pa & ((1U << 11) - 1)) { dev_info(&dd->pcidev->dev, "BUG: physaddr %lx " - "not 4KB aligned!\n", pa); + "not 2KB aligned!\n", pa); return; } pa >>= 11; From ralph.campbell at qlogic.com Thu Mar 20 17:18:06 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:18:06 -0700 Subject: [ofa-general] [PATCH 06/10] IB/ipath: Head of Line blocking vs forward progress of user apps In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001806.13431.23447.stgit@eng-46.mv.qlogic.com> From: John Gregor <> There's a conflict between our need to quiesce PSM-based applications to avoid HoL blocking when the IB link goes down and the apps' desire to remain running so that their quiescence timout mechanism can keep running. The compromise is to STOP the processes for a fixed period of time and then alternate between CONT and STOP until the link is again active. If there are poor interactions with subnet manager configuration at a given site, the interval can be adjusted via a module paramter. Signed-off-by: John Gregor --- drivers/infiniband/hw/ipath/ipath_debug.h | 1 drivers/infiniband/hw/ipath/ipath_diag.c | 19 +- drivers/infiniband/hw/ipath/ipath_driver.c | 129 +++++++++++ drivers/infiniband/hw/ipath/ipath_init_chip.c | 6 + drivers/infiniband/hw/ipath/ipath_intr.c | 287 ++++++++++++------------- drivers/infiniband/hw/ipath/ipath_kernel.h | 32 +++ drivers/infiniband/hw/ipath/ipath_registers.h | 20 -- 7 files changed, 312 insertions(+), 182 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_debug.h b/drivers/infiniband/hw/ipath/ipath_debug.h index d6f6953..7170bd2 100644 --- a/drivers/infiniband/hw/ipath/ipath_debug.h +++ b/drivers/infiniband/hw/ipath/ipath_debug.h @@ -66,6 +66,7 @@ #define __IPATH_IPATHERR 0x40000 /* Ethernet (IPATH) errors */ #define __IPATH_IPATHPD 0x80000 /* Ethernet (IPATH) packet dump */ #define __IPATH_IPATHTABLE 0x100000 /* Ethernet (IPATH) table dump */ +#define __IPATH_LINKVERBDBG 0x200000 /* very verbose linkchange debug */ #else /* _IPATH_DEBUGGING */ diff --git a/drivers/infiniband/hw/ipath/ipath_diag.c b/drivers/infiniband/hw/ipath/ipath_diag.c index 4137c77..4ee9479 100644 --- a/drivers/infiniband/hw/ipath/ipath_diag.c +++ b/drivers/infiniband/hw/ipath/ipath_diag.c @@ -330,6 +330,7 @@ static ssize_t ipath_diagpkt_write(struct file *fp, struct ipath_devdata *dd; ssize_t ret = 0; u64 val; + u32 l_state, lt_state; /* LinkState, LinkTrainingState */ if (count != sizeof(dp)) { ret = -EINVAL; @@ -396,10 +397,17 @@ static ssize_t ipath_diagpkt_write(struct file *fp, ret = -ENODEV; goto bail; } - /* Check link state, but not if we have custom PBC */ - val = dd->ipath_lastibcstat & IPATH_IBSTATE_MASK; - if (!dp.pbc_wd && val != IPATH_IBSTATE_INIT && - val != IPATH_IBSTATE_ARM && val != IPATH_IBSTATE_ACTIVE) { + /* + * Want to skip check for l_state if using custom PBC, + * because we might be trying to force an SM packet out. + * first-cut, skip _all_ state checking in that case. + */ + val = ipath_ib_state(dd, dd->ipath_lastibcstat); + lt_state = ipath_ib_linktrstate(dd, dd->ipath_lastibcstat); + l_state = ipath_ib_linkstate(dd, dd->ipath_lastibcstat); + if (!dp.pbc_wd && (lt_state != INFINIPATH_IBCS_LT_STATE_LINKUP || + (val != dd->ib_init && val != dd->ib_arm && + val != dd->ib_active))) { ipath_cdbg(VERBOSE, "unit %u not ready (state %llx)\n", dd->ipath_unit, (unsigned long long) val); ret = -EINVAL; @@ -438,6 +446,9 @@ static ssize_t ipath_diagpkt_write(struct file *fp, ret = -EBUSY; goto bail; } + /* disarm it just to be extra sure */ + ipath_disarm_piobufs(dd, pbufn, 1); + plen >>= 2; /* in dwords */ diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index 7121fe8..5579195 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -77,6 +77,11 @@ unsigned ipath_mtu4096 = 1; /* max 4KB IB mtu by default, if supported */ module_param_named(mtu4096, ipath_mtu4096, uint, S_IRUGO); MODULE_PARM_DESC(mtu4096, "enable MTU of 4096 bytes, if supported"); +static unsigned ipath_hol_timeout_ms = 13000; +module_param_named(hol_timeout_ms, ipath_hol_timeout_ms, uint, S_IRUGO); +MODULE_PARM_DESC(hol_timeout_ms, + "duration of user app suspension after link failure"); + MODULE_LICENSE("GPL"); MODULE_AUTHOR("QLogic "); MODULE_DESCRIPTION("QLogic InfiniPath driver"); @@ -1670,11 +1675,8 @@ static void ipath_set_ib_lstate(struct ipath_devdata *dd, int which) ipath_cdbg(VERBOSE, "Trying to move unit %u to %s, current ltstate " "is %s\n", dd->ipath_unit, what[linkcmd], - ipath_ibcstatus_str[ - (ipath_read_kreg64 - (dd, dd->ipath_kregs->kr_ibcstatus) >> - INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT) & - INFINIPATH_IBCS_LINKTRAININGSTATE_MASK]); + ipath_ibcstatus_str[ipath_ib_linktrstate(dd, + ipath_read_kreg64(dd, dd->ipath_kregs->kr_ibcstatus))]); /* flush all queued sends when going to DOWN to be sure that * they don't block MAD packets */ if (linkcmd == INFINIPATH_IBCC_LINKCMD_DOWN) @@ -1925,9 +1927,8 @@ static void ipath_run_led_override(unsigned long opaque) */ val = ipath_read_kreg64(dd, dd->ipath_kregs->kr_ibcstatus); ltstate = (val >> INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT) & - INFINIPATH_IBCS_LINKTRAININGSTATE_MASK; - lstate = (val >> INFINIPATH_IBCS_LINKSTATE_SHIFT) & - INFINIPATH_IBCS_LINKSTATE_MASK; + dd->ibcs_lts_mask; + lstate = (val >> dd->ibcs_ls_shift) & INFINIPATH_IBCS_LINKSTATE_MASK; dd->ipath_f_setextled(dd, lstate, ltstate); mod_timer(&dd->ipath_led_override_timer, jiffies + timeoff); @@ -1988,6 +1989,8 @@ void ipath_shutdown_device(struct ipath_devdata *dd) ipath_dbg("Shutting down the device\n"); + ipath_hol_up(dd); /* make sure user processes aren't suspended */ + dd->ipath_flags |= IPATH_LINKUNK; dd->ipath_flags &= ~(IPATH_INITTED | IPATH_LINKDOWN | IPATH_LINKINIT | IPATH_LINKARMED | @@ -2037,6 +2040,8 @@ void ipath_shutdown_device(struct ipath_devdata *dd) */ dd->ipath_f_quiet_serdes(dd); + /* stop all the timers that might still be running */ + del_timer_sync(&dd->ipath_hol_timer); if (dd->ipath_stats_timer_active) { del_timer_sync(&dd->ipath_stats_timer); dd->ipath_stats_timer_active = 0; @@ -2252,6 +2257,114 @@ bail: return ret; } +/* + * send a signal to all the processes that have the driver open + * through the normal interfaces (i.e., everything other than diags + * interface). Returns number of signalled processes. + */ +int ipath_signal_procs(struct ipath_devdata *dd, int sig) +{ + int i, sub, any = 0; + pid_t pid; + + if (!dd->ipath_pd) + return 0; + for (i = 1; i < dd->ipath_cfgports; i++) { + if (!dd->ipath_pd[i] || !dd->ipath_pd[i]->port_cnt || + !dd->ipath_pd[i]->port_pid) + continue; + pid = dd->ipath_pd[i]->port_pid; + dev_info(&dd->pcidev->dev, "context %d in use " + "(PID %u), sending signal %d\n", + i, pid, sig); + kill_proc(pid, sig, 1); + any++; + for (sub = 0; sub < INFINIPATH_MAX_SUBPORT; sub++) { + pid = dd->ipath_pd[i]->port_subpid[sub]; + if (!pid) + continue; + dev_info(&dd->pcidev->dev, "sub-context " + "%d:%d in use (PID %u), sending " + "signal %d\n", i, sub, pid, sig); + kill_proc(pid, sig, 1); + any++; + } + } + return any; +} + +static void ipath_hol_signal_down(struct ipath_devdata *dd) +{ + if (ipath_signal_procs(dd, SIGSTOP)) + ipath_dbg("Stopped some processes\n"); + ipath_cancel_sends(dd, 1); +} + + +static void ipath_hol_signal_up(struct ipath_devdata *dd) +{ + if (ipath_signal_procs(dd, SIGCONT)) + ipath_dbg("Continued some processes\n"); +} + +/* + * link is down, stop any users processes, and flush pending sends + * to prevent HoL blocking, then start the HoL timer that + * periodically continues, then stop procs, so they can detect + * link down if they want, and do something about it. + * Timer may already be running, so use __mod_timer, not add_timer. + */ +void ipath_hol_down(struct ipath_devdata *dd) +{ + dd->ipath_hol_state = IPATH_HOL_DOWN; + ipath_hol_signal_down(dd); + dd->ipath_hol_next = IPATH_HOL_DOWNCONT; + dd->ipath_hol_timer.expires = jiffies + + msecs_to_jiffies(ipath_hol_timeout_ms); + __mod_timer(&dd->ipath_hol_timer, dd->ipath_hol_timer.expires); +} + +/* + * link is up, continue any user processes, and ensure timer + * is a nop, if running. Let timer keep running, if set; it + * will nop when it sees the link is up + */ +void ipath_hol_up(struct ipath_devdata *dd) +{ + ipath_hol_signal_up(dd); + dd->ipath_hol_state = IPATH_HOL_UP; +} + +/* + * toggle the running/not running state of user proceses + * to prevent HoL blocking on chip resources, but still allow + * user processes to do link down special case handling. + * Should only be called via the timer + */ +void ipath_hol_event(unsigned long opaque) +{ + struct ipath_devdata *dd = (struct ipath_devdata *)opaque; + + if (dd->ipath_hol_next == IPATH_HOL_DOWNSTOP + && dd->ipath_hol_state != IPATH_HOL_UP) { + dd->ipath_hol_next = IPATH_HOL_DOWNCONT; + ipath_dbg("Stopping processes\n"); + ipath_hol_signal_down(dd); + } else { /* may do "extra" if also in ipath_hol_up() */ + dd->ipath_hol_next = IPATH_HOL_DOWNSTOP; + ipath_dbg("Continuing processes\n"); + ipath_hol_signal_up(dd); + } + if (dd->ipath_hol_state == IPATH_HOL_UP) + ipath_dbg("link's up, don't resched timer\n"); + else { + dd->ipath_hol_timer.expires = jiffies + + msecs_to_jiffies(ipath_hol_timeout_ms); + __mod_timer(&dd->ipath_hol_timer, + dd->ipath_hol_timer.expires); + } +} + int ipath_set_rx_pol_inv(struct ipath_devdata *dd, u8 new_pol_inv) { u64 val; diff --git a/drivers/infiniband/hw/ipath/ipath_init_chip.c b/drivers/infiniband/hw/ipath/ipath_init_chip.c index f0d7848..bed0927 100644 --- a/drivers/infiniband/hw/ipath/ipath_init_chip.c +++ b/drivers/infiniband/hw/ipath/ipath_init_chip.c @@ -908,6 +908,12 @@ int ipath_init_chip(struct ipath_devdata *dd, int reinit) dd->ipath_stats_timer_active = 1; } + /* Set up HoL state */ + init_timer(&dd->ipath_hol_timer); + dd->ipath_hol_timer.function = ipath_hol_event; + dd->ipath_hol_timer.data = (unsigned long)dd; + dd->ipath_hol_state = IPATH_HOL_UP; + done: if (!ret) { *dd->ipath_statusp |= IPATH_STATUS_CHIP_PRESENT; diff --git a/drivers/infiniband/hw/ipath/ipath_intr.c b/drivers/infiniband/hw/ipath/ipath_intr.c index ed2a227..dde5dfc 100644 --- a/drivers/infiniband/hw/ipath/ipath_intr.c +++ b/drivers/infiniband/hw/ipath/ipath_intr.c @@ -32,6 +32,7 @@ */ #include +#include #include "ipath_kernel.h" #include "ipath_verbs.h" @@ -256,24 +257,20 @@ void ipath_format_hwerrors(u64 hwerrs, } /* return the strings for the most common link states */ -static char *ib_linkstate(u32 linkstate) +static char *ib_linkstate(struct ipath_devdata *dd, u64 ibcs) { char *ret; + u32 state; - switch (linkstate) { - case IPATH_IBSTATE_INIT: + state = ipath_ib_state(dd, ibcs); + if (state == dd->ib_init) ret = "Init"; - break; - case IPATH_IBSTATE_ARM: + else if (state == dd->ib_arm) ret = "Arm"; - break; - case IPATH_IBSTATE_ACTIVE: + else if (state == dd->ib_active) ret = "Active"; - break; - default: + else ret = "Down"; - } - return ret; } @@ -288,103 +285,137 @@ void signal_ib_event(struct ipath_devdata *dd, enum ib_event_type ev) } static void handle_e_ibstatuschanged(struct ipath_devdata *dd, - ipath_err_t errs, int noprint) + ipath_err_t errs) { - u64 val; - u32 ltstate, lstate; + u32 ltstate, lstate, ibstate, lastlstate; + u32 init = dd->ib_init; + u32 arm = dd->ib_arm; + u32 active = dd->ib_active; + const u64 ibcs = ipath_read_kreg64(dd, dd->ipath_kregs->kr_ibcstatus); + + lstate = ipath_ib_linkstate(dd, ibcs); /* linkstate */ + ibstate = ipath_ib_state(dd, ibcs); + /* linkstate at last interrupt */ + lastlstate = ipath_ib_linkstate(dd, dd->ipath_lastibcstat); + ltstate = ipath_ib_linktrstate(dd, ibcs); /* linktrainingtate */ /* - * even if diags are enabled, we want to notice LINKINIT, etc. - * We just don't want to change the LED state, or - * dd->ipath_kregs->kr_ibcctrl + * if linkstate transitions into INIT from any of the various down + * states, or if it transitions from any of the up (INIT or better) + * states into any of the down states (except link recovery), then + * call the chip-specific code to take appropriate actions. */ - val = ipath_read_kreg64(dd, dd->ipath_kregs->kr_ibcstatus); - lstate = val & IPATH_IBSTATE_MASK; + if (lstate >= INFINIPATH_IBCS_L_STATE_INIT && + lastlstate == INFINIPATH_IBCS_L_STATE_DOWN) { + /* transitioned to UP */ + if (dd->ipath_f_ib_updown(dd, 1, ibcs)) { + ipath_cdbg(LINKVERB, "LinkUp handled, skipped\n"); + goto skip_ibchange; /* chip-code handled */ + } + } else if ((lastlstate >= INFINIPATH_IBCS_L_STATE_INIT || + (dd->ipath_flags & IPATH_IB_FORCE_NOTIFY)) && + ltstate <= INFINIPATH_IBCS_LT_STATE_CFGDEBOUNCE && + ltstate != INFINIPATH_IBCS_LT_STATE_LINKUP) { + int handled; + handled = dd->ipath_f_ib_updown(dd, 0, ibcs); + dd->ipath_flags &= ~IPATH_IB_FORCE_NOTIFY; + if (handled) { + ipath_cdbg(LINKVERB, "LinkDown handled, skipped\n"); + goto skip_ibchange; /* chip-code handled */ + } + } /* - * this is confusing enough when it happens that I want to always put it - * on the console and in the logs. If it was a requested state change, - * we'll have already cleared the flags, so we won't print this warning + * Significant enough to always print and get into logs, if it was + * unexpected. If it was a requested state change, we'll have + * already cleared the flags, so we won't print this warning */ - if ((lstate != IPATH_IBSTATE_ARM && lstate != IPATH_IBSTATE_ACTIVE) - && (dd->ipath_flags & (IPATH_LINKARMED | IPATH_LINKACTIVE))) { - dev_info(&dd->pcidev->dev, "Link state changed from %s to %s\n", - (dd->ipath_flags & IPATH_LINKARMED) ? "ARM" : "ACTIVE", - ib_linkstate(lstate)); - /* - * Flush all queued sends when link went to DOWN or INIT, - * to be sure that they don't block SMA and other MAD packets - */ - ipath_cancel_sends(dd, 1); - } - else if (lstate == IPATH_IBSTATE_INIT || lstate == IPATH_IBSTATE_ARM || - lstate == IPATH_IBSTATE_ACTIVE) { - /* - * only print at SMA if there is a change, debug if not - * (sometimes we want to know that, usually not). - */ - if (lstate == ((unsigned) dd->ipath_lastibcstat - & IPATH_IBSTATE_MASK)) { - ipath_dbg("Status change intr but no change (%s)\n", - ib_linkstate(lstate)); - } - else - ipath_cdbg(VERBOSE, "Unit %u link state %s, last " - "was %s\n", dd->ipath_unit, - ib_linkstate(lstate), - ib_linkstate((unsigned) - dd->ipath_lastibcstat - & IPATH_IBSTATE_MASK)); - } - else { - lstate = dd->ipath_lastibcstat & IPATH_IBSTATE_MASK; - if (lstate == IPATH_IBSTATE_INIT || - lstate == IPATH_IBSTATE_ARM || - lstate == IPATH_IBSTATE_ACTIVE) - ipath_cdbg(VERBOSE, "Unit %u link state down" - " (state 0x%x), from %s\n", - dd->ipath_unit, - (u32)val & IPATH_IBSTATE_MASK, - ib_linkstate(lstate)); - else - ipath_cdbg(VERBOSE, "Unit %u link state changed " - "to 0x%x from down (%x)\n", - dd->ipath_unit, (u32) val, lstate); + if ((ibstate != arm && ibstate != active) && + (dd->ipath_flags & (IPATH_LINKARMED | IPATH_LINKACTIVE))) { + dev_info(&dd->pcidev->dev, "Link state changed from %s " + "to %s\n", (dd->ipath_flags & IPATH_LINKARMED) ? + "ARM" : "ACTIVE", ib_linkstate(dd, ibcs)); } - ltstate = (val >> INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT) & - INFINIPATH_IBCS_LINKTRAININGSTATE_MASK; - lstate = (val >> INFINIPATH_IBCS_LINKSTATE_SHIFT) & - INFINIPATH_IBCS_LINKSTATE_MASK; if (ltstate == INFINIPATH_IBCS_LT_STATE_POLLACTIVE || ltstate == INFINIPATH_IBCS_LT_STATE_POLLQUIET) { - u32 last_ltstate; - + u32 lastlts; + lastlts = ipath_ib_linktrstate(dd, dd->ipath_lastibcstat); /* - * Ignore cycling back and forth from Polling.Active - * to Polling.Quiet while waiting for the other end of - * the link to come up. We will cycle back and forth - * between them if no cable is plugged in, - * the other device is powered off or disabled, etc. + * Ignore cycling back and forth from Polling.Active to + * Polling.Quiet while waiting for the other end of the link + * to come up, except to try and decide if we are connected + * to a live IB device or not. We will cycle back and + * forth between them if no cable is plugged in, the other + * device is powered off or disabled, etc. */ - last_ltstate = (dd->ipath_lastibcstat >> - INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT) - & INFINIPATH_IBCS_LINKTRAININGSTATE_MASK; - if (last_ltstate == INFINIPATH_IBCS_LT_STATE_POLLACTIVE - || last_ltstate == - INFINIPATH_IBCS_LT_STATE_POLLQUIET) { - if (dd->ipath_ibpollcnt > 40) { + if (lastlts == INFINIPATH_IBCS_LT_STATE_POLLACTIVE || + lastlts == INFINIPATH_IBCS_LT_STATE_POLLQUIET) { + if (++dd->ipath_ibpollcnt == 40) { dd->ipath_flags |= IPATH_NOCABLE; *dd->ipath_statusp |= IPATH_STATUS_IB_NOCABLE; - } else - dd->ipath_ibpollcnt++; + ipath_cdbg(LINKVERB, "Set NOCABLE\n"); + } + ipath_cdbg(LINKVERB, "POLL change to %s (%x)\n", + ipath_ibcstatus_str[ltstate], ibstate); goto skip_ibchange; } } - dd->ipath_ibpollcnt = 0; /* some state other than 2 or 3 */ + + dd->ipath_ibpollcnt = 0; /* not poll*, now */ ipath_stats.sps_iblink++; - if (ltstate != INFINIPATH_IBCS_LT_STATE_LINKUP) { + + if (ibstate == init || ibstate == arm || ibstate == active) { + *dd->ipath_statusp &= ~IPATH_STATUS_IB_NOCABLE; + if (ibstate == init || ibstate == arm) { + *dd->ipath_statusp &= ~IPATH_STATUS_IB_READY; + if (dd->ipath_flags & IPATH_LINKACTIVE) + signal_ib_event(dd, IB_EVENT_PORT_ERR); + } + if (ibstate == arm) { + dd->ipath_flags |= IPATH_LINKARMED; + dd->ipath_flags &= ~(IPATH_LINKUNK | + IPATH_LINKINIT | IPATH_LINKDOWN | + IPATH_LINKACTIVE | IPATH_NOCABLE); + ipath_hol_down(dd); + } else if (ibstate == init) { + /* + * set INIT and DOWN. Down is checked by + * most of the other code, but INIT is + * useful to know in a few places. + */ + dd->ipath_flags |= IPATH_LINKINIT | + IPATH_LINKDOWN; + dd->ipath_flags &= ~(IPATH_LINKUNK | + IPATH_LINKARMED | IPATH_LINKACTIVE | + IPATH_NOCABLE); + ipath_hol_down(dd); + } else { /* active */ + *dd->ipath_statusp |= + IPATH_STATUS_IB_READY | IPATH_STATUS_IB_CONF; + dd->ipath_flags |= IPATH_LINKACTIVE; + dd->ipath_flags &= ~(IPATH_LINKUNK | IPATH_LINKINIT + | IPATH_LINKDOWN | IPATH_LINKARMED | + IPATH_NOCABLE); + signal_ib_event(dd, IB_EVENT_PORT_ACTIVE); + /* LED active not handled in chip _f_updown */ + dd->ipath_f_setextled(dd, lstate, ltstate); + ipath_hol_up(dd); + } + + /* + * print after we've already done the work, so as not to + * delay the state changes and notifications, for debugging + */ + if (lstate == lastlstate) + ipath_cdbg(LINKVERB, "Unchanged from last: %s " + "(%x)\n", ib_linkstate(dd, ibcs), ibstate); + else + ipath_cdbg(VERBOSE, "Unit %u: link up to %s %s (%x)\n", + dd->ipath_unit, ib_linkstate(dd, ibcs), + ipath_ibcstatus_str[ltstate], ibstate); + } else { /* down */ if (dd->ipath_flags & IPATH_LINKACTIVE) signal_ib_event(dd, IB_EVENT_PORT_ERR); dd->ipath_flags |= IPATH_LINKDOWN; @@ -393,65 +424,22 @@ static void handle_e_ibstatuschanged(struct ipath_devdata *dd, IPATH_LINKARMED); *dd->ipath_statusp &= ~IPATH_STATUS_IB_READY; dd->ipath_lli_counter = 0; - if (!noprint) { - if (((dd->ipath_lastibcstat >> - INFINIPATH_IBCS_LINKSTATE_SHIFT) & - INFINIPATH_IBCS_LINKSTATE_MASK) - == INFINIPATH_IBCS_L_STATE_ACTIVE) - /* if from up to down be more vocal */ - ipath_cdbg(VERBOSE, - "Unit %u link now down (%s)\n", - dd->ipath_unit, - ipath_ibcstatus_str[ltstate]); - else - ipath_cdbg(VERBOSE, "Unit %u link is " - "down (%s)\n", dd->ipath_unit, - ipath_ibcstatus_str[ltstate]); - } - dd->ipath_f_setextled(dd, lstate, ltstate); - } else if ((val & IPATH_IBSTATE_MASK) == IPATH_IBSTATE_ACTIVE) { - dd->ipath_flags |= IPATH_LINKACTIVE; - dd->ipath_flags &= - ~(IPATH_LINKUNK | IPATH_LINKINIT | IPATH_LINKDOWN | - IPATH_LINKARMED | IPATH_NOCABLE); - *dd->ipath_statusp &= ~IPATH_STATUS_IB_NOCABLE; - *dd->ipath_statusp |= - IPATH_STATUS_IB_READY | IPATH_STATUS_IB_CONF; - dd->ipath_f_setextled(dd, lstate, ltstate); - signal_ib_event(dd, IB_EVENT_PORT_ACTIVE); - } else if ((val & IPATH_IBSTATE_MASK) == IPATH_IBSTATE_INIT) { - if (dd->ipath_flags & IPATH_LINKACTIVE) - signal_ib_event(dd, IB_EVENT_PORT_ERR); - /* - * set INIT and DOWN. Down is checked by most of the other - * code, but INIT is useful to know in a few places. - */ - dd->ipath_flags |= IPATH_LINKINIT | IPATH_LINKDOWN; - dd->ipath_flags &= - ~(IPATH_LINKUNK | IPATH_LINKACTIVE | IPATH_LINKARMED - | IPATH_NOCABLE); - *dd->ipath_statusp &= ~(IPATH_STATUS_IB_NOCABLE - | IPATH_STATUS_IB_READY); - dd->ipath_f_setextled(dd, lstate, ltstate); - } else if ((val & IPATH_IBSTATE_MASK) == IPATH_IBSTATE_ARM) { - if (dd->ipath_flags & IPATH_LINKACTIVE) - signal_ib_event(dd, IB_EVENT_PORT_ERR); - dd->ipath_flags |= IPATH_LINKARMED; - dd->ipath_flags &= - ~(IPATH_LINKUNK | IPATH_LINKDOWN | IPATH_LINKINIT | - IPATH_LINKACTIVE | IPATH_NOCABLE); - *dd->ipath_statusp &= ~(IPATH_STATUS_IB_NOCABLE - | IPATH_STATUS_IB_READY); - dd->ipath_f_setextled(dd, lstate, ltstate); - } else { - if (!noprint) - ipath_dbg("IBstatuschange unit %u: %s (%x)\n", - dd->ipath_unit, - ipath_ibcstatus_str[ltstate], ltstate); + if (lastlstate != INFINIPATH_IBCS_L_STATE_DOWN) + ipath_cdbg(VERBOSE, "Unit %u link state down " + "(state 0x%x), from %s\n", + dd->ipath_unit, lstate, + ib_linkstate(dd, dd->ipath_lastibcstat)); + else + ipath_cdbg(LINKVERB, "Unit %u link state changed " + "to %s (0x%x) from down (%x)\n", + dd->ipath_unit, + ipath_ibcstatus_str[ltstate], + ibstate, lastlstate); } + skip_ibchange: - dd->ipath_lastibcstat = val; + dd->ipath_lastibcstat = ibcs; } static void handle_supp_msgs(struct ipath_devdata *dd, @@ -743,16 +731,13 @@ static int handle_errors(struct ipath_devdata *dd, ipath_err_t errs) dd->ipath_flags &= ~(IPATH_LINKUNK | IPATH_LINKINIT | IPATH_LINKARMED | IPATH_LINKACTIVE); *dd->ipath_statusp &= ~IPATH_STATUS_IB_READY; - if (!noprint) { - u64 st = ipath_read_kreg64( - dd, dd->ipath_kregs->kr_ibcstatus); - ipath_dbg("Lost link, link now down (%s)\n", - ipath_ibcstatus_str[st & 0xf]); - } + ipath_dbg("Lost link, link now down (%s)\n", + ipath_ibcstatus_str[ipath_read_kreg64(dd, + dd->ipath_kregs->kr_ibcstatus) & 0xf]); } if (errs & INFINIPATH_E_IBSTATUSCHANGED) - handle_e_ibstatuschanged(dd, errs, noprint); + handle_e_ibstatuschanged(dd, errs); if (errs & INFINIPATH_E_RESET) { if (!noprint) diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 70c0a0d..0cedc3f 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -427,6 +427,11 @@ struct ipath_devdata { unsigned long ipath_ureg_align; /* user register alignment */ + /* HoL blocking / user app forward-progress state */ + unsigned ipath_hol_state; + unsigned ipath_hol_next; + struct timer_list ipath_hol_timer; + /* * Shadow copies of registers; size indicates read access size. * Most of them are readonly, but some are write-only register, @@ -706,6 +711,13 @@ struct ipath_devdata { u16 ipath_jint_max_packets; /* max packets across all ports */ }; +/* ipath_hol_state values (stopping/starting user proc, send flushing) */ +#define IPATH_HOL_UP 0 +#define IPATH_HOL_DOWN 1 +/* ipath_hol_next toggle values, used when hol_state IPATH_HOL_DOWN */ +#define IPATH_HOL_DOWNSTOP 0 +#define IPATH_HOL_DOWNCONT 1 + /* Private data for file operations */ struct ipath_filedata { struct ipath_portdata *pd; @@ -722,6 +734,7 @@ void ipath_disable_wc(struct ipath_devdata *dd); int ipath_count_units(int *npresentp, int *nupp, int *maxportsp); void ipath_shutdown_device(struct ipath_devdata *); void ipath_clear_freeze(struct ipath_devdata *); +int ipath_signal_procs(struct ipath_devdata *, int); struct file_operations; int ipath_cdev_init(int minor, char *name, const struct file_operations *fops, @@ -775,6 +788,9 @@ int ipath_set_lid(struct ipath_devdata *, u32, u8); int ipath_set_rx_pol_inv(struct ipath_devdata *dd, u8 new_pol_inv); void ipath_enable_armlaunch(struct ipath_devdata *); void ipath_disable_armlaunch(struct ipath_devdata *); +void ipath_hol_down(struct ipath_devdata *); +void ipath_hol_up(struct ipath_devdata *); +void ipath_hol_event(unsigned long); /* for use in system calls, where we want to know device type, etc. */ #define port_fp(fp) ((struct ipath_filedata *)(fp)->private_data)->pd @@ -830,6 +846,7 @@ void ipath_disable_armlaunch(struct ipath_devdata *); /* Suppress heartbeat, even if turning off loopback */ #define IPATH_NO_HRTBT 0x1000000 #define IPATH_HAS_MULT_IB_SPEED 0x8000000 +#define IPATH_IB_FORCE_NOTIFY 0x80000000 /* force notify on next ib change */ /* Bits in GPIO for the added interrupts */ #define IPATH_GPIO_PORT0_BIT 2 @@ -1030,6 +1047,21 @@ static inline u32 ipath_ib_linktrstate(struct ipath_devdata *dd, u64 ibcs) } /* + * from contents of IBCStatus (or a saved copy), return logical link state + * combination of link state and linktraining state (down, active, init, + * arm, etc. + */ +static inline u32 ipath_ib_state(struct ipath_devdata *dd, u64 ibcs) +{ + u32 ibs; + ibs = (u32)(ibcs >> INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT) & + dd->ibcs_lts_mask; + ibs |= (u32)(ibcs & + (INFINIPATH_IBCS_LINKSTATE_MASK << dd->ibcs_ls_shift)); + return ibs; +} + +/* * sysfs interface. */ diff --git a/drivers/infiniband/hw/ipath/ipath_registers.h b/drivers/infiniband/hw/ipath/ipath_registers.h index cb19ea2..16d0d74 100644 --- a/drivers/infiniband/hw/ipath/ipath_registers.h +++ b/drivers/infiniband/hw/ipath/ipath_registers.h @@ -200,7 +200,6 @@ #define INFINIPATH_IBCC_LINKDOWNDEFAULTSTATE 0x4000000000000000ULL /* kr_ibcstatus bits */ -#define INFINIPATH_IBCS_LINKTRAININGSTATE_MASK 0xF #define INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT 0 #define INFINIPATH_IBCS_LINKSTATE_MASK 0x7 #define INFINIPATH_IBCS_LINKSTATE_SHIFT 4 @@ -221,30 +220,13 @@ #define INFINIPATH_IBCS_LT_STATE_RECOVERRETRAIN 0x0c #define INFINIPATH_IBCS_LT_STATE_RECOVERWAITRMT 0x0e #define INFINIPATH_IBCS_LT_STATE_RECOVERIDLE 0x0f -/* link state machine states (shift by INFINIPATH_IBCS_LINKSTATE_SHIFT) */ +/* link state machine states (shift by ibcs_ls_shift) */ #define INFINIPATH_IBCS_L_STATE_DOWN 0x0 #define INFINIPATH_IBCS_L_STATE_INIT 0x1 #define INFINIPATH_IBCS_L_STATE_ARM 0x2 #define INFINIPATH_IBCS_L_STATE_ACTIVE 0x3 #define INFINIPATH_IBCS_L_STATE_ACT_DEFER 0x4 -/* combination link status states that we use with some frequency */ -#define IPATH_IBSTATE_MASK ((INFINIPATH_IBCS_LINKTRAININGSTATE_MASK \ - << INFINIPATH_IBCS_LINKTRAININGSTATE_SHIFT) | \ - (INFINIPATH_IBCS_LINKSTATE_MASK \ - < References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001811.13431.47996.stgit@eng-46.mv.qlogic.com> This patch adds code to get/set portinfo to support multiple link speeds and widths. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_kernel.h | 2 + drivers/infiniband/hw/ipath/ipath_mad.c | 89 +++++++++++++++++++--------- drivers/infiniband/hw/ipath/ipath_verbs.c | 11 ++- 3 files changed, 69 insertions(+), 33 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 0cedc3f..9321ad4 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -802,6 +802,8 @@ void ipath_hol_event(unsigned long); /* * values for ipath_flags */ + /* chip can report link latency (IB 1.2) */ +#define IPATH_HAS_LINK_LATENCY 0x1 /* The chip is up and initted */ #define IPATH_INITTED 0x2 /* set if any user code has set kr_rcvhdrsize */ diff --git a/drivers/infiniband/hw/ipath/ipath_mad.c b/drivers/infiniband/hw/ipath/ipath_mad.c index aca876b..7516a26 100644 --- a/drivers/infiniband/hw/ipath/ipath_mad.c +++ b/drivers/infiniband/hw/ipath/ipath_mad.c @@ -146,6 +146,15 @@ static int recv_subn_get_guidinfo(struct ib_smp *smp, return reply(smp); } +static void set_link_width_enabled(struct ipath_devdata *dd, u32 w) +{ + (void) dd->ipath_f_set_ib_cfg(dd, IPATH_IB_CFG_LWID_ENB, w); +} + +static void set_link_speed_enabled(struct ipath_devdata *dd, u32 s) +{ + (void) dd->ipath_f_set_ib_cfg(dd, IPATH_IB_CFG_SPD_ENB, s); +} static int get_overrunthreshold(struct ipath_devdata *dd) { @@ -226,6 +235,7 @@ static int recv_subn_get_portinfo(struct ib_smp *smp, struct ib_device *ibdev, u8 port) { struct ipath_ibdev *dev; + struct ipath_devdata *dd; struct ib_port_info *pip = (struct ib_port_info *)smp->data; u16 lid; u8 ibcstat; @@ -239,6 +249,7 @@ static int recv_subn_get_portinfo(struct ib_smp *smp, } dev = to_idev(ibdev); + dd = dev->dd; /* Clear all fields. Only set the non-zero fields. */ memset(smp->data, 0, sizeof(smp->data)); @@ -248,25 +259,28 @@ static int recv_subn_get_portinfo(struct ib_smp *smp, dev->mkeyprot == 0) pip->mkey = dev->mkey; pip->gid_prefix = dev->gid_prefix; - lid = dev->dd->ipath_lid; + lid = dd->ipath_lid; pip->lid = lid ? cpu_to_be16(lid) : IB_LID_PERMISSIVE; pip->sm_lid = cpu_to_be16(dev->sm_lid); pip->cap_mask = cpu_to_be32(dev->port_cap_flags); /* pip->diag_code; */ pip->mkey_lease_period = cpu_to_be16(dev->mkey_lease_period); pip->local_port_num = port; - pip->link_width_enabled = dev->link_width_enabled; - pip->link_width_supported = 3; /* 1x or 4x */ - pip->link_width_active = 2; /* 4x */ - pip->linkspeed_portstate = 0x10; /* 2.5Gbps */ - ibcstat = dev->dd->ipath_lastibcstat; - pip->linkspeed_portstate |= ((ibcstat >> 4) & 0x3) + 1; + pip->link_width_enabled = dd->ipath_link_width_enabled; + pip->link_width_supported = dd->ipath_link_width_supported; + pip->link_width_active = dd->ipath_link_width_active; + pip->linkspeed_portstate = dd->ipath_link_speed_supported << 4; + ibcstat = dd->ipath_lastibcstat; + /* map LinkState to IB portinfo values. */ + pip->linkspeed_portstate |= ipath_ib_linkstate(dd, ibcstat) + 1; + pip->portphysstate_linkdown = - (ipath_cvt_physportstate[ibcstat & 0xf] << 4) | - (get_linkdowndefaultstate(dev->dd) ? 1 : 2); - pip->mkeyprot_resv_lmc = (dev->mkeyprot << 6) | dev->dd->ipath_lmc; - pip->linkspeedactive_enabled = 0x11; /* 2.5Gbps, 2.5Gbps */ - switch (dev->dd->ipath_ibmtu) { + (ipath_cvt_physportstate[ibcstat & dd->ibcs_lts_mask] << 4) | + (get_linkdowndefaultstate(dd) ? 1 : 2); + pip->mkeyprot_resv_lmc = (dev->mkeyprot << 6) | dd->ipath_lmc; + pip->linkspeedactive_enabled = (dd->ipath_link_speed_active << 4) | + dd->ipath_link_speed_enabled; + switch (dd->ipath_ibmtu) { case 4096: mtu = IB_MTU_4096; break; @@ -300,7 +314,7 @@ static int recv_subn_get_portinfo(struct ib_smp *smp, pip->mkey_violations = cpu_to_be16(dev->mkey_violations); /* P_KeyViolations are counted by hardware. */ pip->pkey_violations = - cpu_to_be16((ipath_get_cr_errpkey(dev->dd) - + cpu_to_be16((ipath_get_cr_errpkey(dd) - dev->z_pkey_violations) & 0xFFFF); pip->qkey_violations = cpu_to_be16(dev->qkey_violations); /* Only the hardware GUID is supported for now */ @@ -309,10 +323,17 @@ static int recv_subn_get_portinfo(struct ib_smp *smp, /* 32.768 usec. response time (guessing) */ pip->resv_resptimevalue = 3; pip->localphyerrors_overrunerrors = - (get_phyerrthreshold(dev->dd) << 4) | - get_overrunthreshold(dev->dd); + (get_phyerrthreshold(dd) << 4) | + get_overrunthreshold(dd); /* pip->max_credit_hint; */ - /* pip->link_roundtrip_latency[3]; */ + if (dev->port_cap_flags & IB_PORT_LINK_LATENCY_SUP) { + u32 v; + + v = dd->ipath_f_get_ib_cfg(dd, IPATH_IB_CFG_LINKLATENCY); + pip->link_roundtrip_latency[0] = v >> 16; + pip->link_roundtrip_latency[1] = v >> 8; + pip->link_roundtrip_latency[2] = v; + } ret = reply(smp); @@ -440,19 +461,25 @@ static int recv_subn_set_portinfo(struct ib_smp *smp, ib_dispatch_event(&event); } - /* Only 4x supported but allow 1x or 4x to be set (see 14.2.6.6). */ + /* Allow 1x or 4x to be set (see 14.2.6.6). */ lwe = pip->link_width_enabled; - if ((lwe >= 4 && lwe <= 8) || (lwe >= 0xC && lwe <= 0xFE)) - goto err; - if (lwe == 0xFF) - dev->link_width_enabled = 3; /* 1x or 4x */ - else if (lwe) - dev->link_width_enabled = lwe; + if (lwe) { + if (lwe == 0xFF) + lwe = dd->ipath_link_width_supported; + else if (lwe >= 16 || (lwe & ~dd->ipath_link_width_supported)) + goto err; + set_link_width_enabled(dd, lwe); + } - /* Only 2.5 Gbs supported. */ + /* Allow 2.5 or 5.0 Gbs. */ lse = pip->linkspeedactive_enabled & 0xF; - if (lse >= 2 && lse <= 0xE) - goto err; + if (lse) { + if (lse == 15) + lse = dd->ipath_link_speed_supported; + else if (lse >= 8 || (lse & ~dd->ipath_link_speed_supported)) + goto err; + set_link_speed_enabled(dd, lse); + } /* Set link down default state. */ switch (pip->portphysstate_linkdown & 0xF) { @@ -946,10 +973,14 @@ static int recv_pma_get_portsamplescontrol(struct ib_perf *pmp, * nsec. 0 == 4 nsec., 1 == 8 nsec., ..., 255 == 1020 nsec. Sample * intervals are counted in ticks. Since we use Linux timers, that * count in jiffies, we can't sample for less than 1000 ticks if HZ - * == 1000 (4000 ticks if HZ is 250). + * == 1000 (4000 ticks if HZ is 250). link_speed_active returns 2 for + * DDR, 1 for SDR, set the tick to 1 for DDR, 0 for SDR on chips that + * have hardware support for delaying packets. */ - /* XXX This is WRONG. */ - p->tick = 250; /* 1 usec. */ + if (crp->cr_psstat) + p->tick = dev->dd->ipath_link_speed_active - 1; + else + p->tick = 250; /* 1 usec. */ p->counter_width = 4; /* 32 bit counters */ p->counter_mask0_9 = COUNTER_MASK0_9; spin_lock_irqsave(&dev->pending_lock, flags); diff --git a/drivers/infiniband/hw/ipath/ipath_verbs.c b/drivers/infiniband/hw/ipath/ipath_verbs.c index 012ccb4..2f9bc29 100644 --- a/drivers/infiniband/hw/ipath/ipath_verbs.c +++ b/drivers/infiniband/hw/ipath/ipath_verbs.c @@ -1183,7 +1183,9 @@ static int ipath_query_port(struct ib_device *ibdev, props->sm_lid = dev->sm_lid; props->sm_sl = dev->sm_sl; ibcstat = dd->ipath_lastibcstat; - props->state = ((ibcstat >> 4) & 0x3) + 1; + /* map LinkState to IB portinfo values. */ + props->state = ipath_ib_linkstate(dd, ibcstat) + 1; + /* See phys_state_show() */ props->phys_state = /* MEA: assumes shift == 0 */ ipath_cvt_physportstate[dd->ipath_lastibcstat & @@ -1195,9 +1197,9 @@ static int ipath_query_port(struct ib_device *ibdev, props->bad_pkey_cntr = ipath_get_cr_errpkey(dd) - dev->z_pkey_violations; props->qkey_viol_cntr = dev->qkey_violations; - props->active_width = IB_WIDTH_4X; + props->active_width = dd->ipath_link_width_active; /* See rate_show() */ - props->active_speed = 1; /* Regular 10Mbs speed. */ + props->active_speed = dd->ipath_link_speed_active; props->max_vl_num = 1; /* VLCap = VL0 */ props->init_type_reply = 0; @@ -1629,12 +1631,13 @@ int ipath_register_ib_device(struct ipath_devdata *dd) idev->pending_index = 0; idev->port_cap_flags = IB_PORT_SYS_IMAGE_GUID_SUP | IB_PORT_CLIENT_REG_SUP; + if (dd->ipath_flags & IPATH_HAS_LINK_LATENCY) + idev->port_cap_flags |= IB_PORT_LINK_LATENCY_SUP; idev->pma_counter_select[0] = IB_PMA_PORT_XMIT_DATA; idev->pma_counter_select[1] = IB_PMA_PORT_RCV_DATA; idev->pma_counter_select[2] = IB_PMA_PORT_XMIT_PKTS; idev->pma_counter_select[3] = IB_PMA_PORT_RCV_PKTS; idev->pma_counter_select[4] = IB_PMA_PORT_XMIT_WAIT; - idev->link_width_enabled = 3; /* 1x or 4x */ /* Snapshot current HW counters to "clear" them. */ ipath_get_counters(dd, &cntrs); From ralph.campbell at qlogic.com Thu Mar 20 17:18:16 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:18:16 -0700 Subject: [ofa-general] [PATCH 08/10] IB/ipath - HW workaround for case where chip can send but not receive In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001816.13431.94.stgit@eng-46.mv.qlogic.com> Workaround a QLE7140 problem that rarely causes flow control problems after link recovery by forcing a link retrain after recovery. A module parameter is provided to control the behavior in case it causes problems. Signed-off-by: Dave Olson --- drivers/infiniband/hw/ipath/ipath_driver.c | 4 ++++ drivers/infiniband/hw/ipath/ipath_intr.c | 18 ++++++++++++++++++ drivers/infiniband/hw/ipath/ipath_kernel.h | 2 ++ 3 files changed, 24 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index 5579195..b1613a7 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -82,6 +82,10 @@ module_param_named(hol_timeout_ms, ipath_hol_timeout_ms, uint, S_IRUGO); MODULE_PARM_DESC(hol_timeout_ms, "duration of user app suspension after link failure"); +unsigned ipath_linkrecovery = 1; +module_param_named(linkrecovery, ipath_linkrecovery, uint, S_IWUSR | S_IRUGO); +MODULE_PARM_DESC(linkrecovery, "enable workaround for link recovery issue"); + MODULE_LICENSE("GPL"); MODULE_AUTHOR("QLogic "); MODULE_DESCRIPTION("QLogic InfiniPath driver"); diff --git a/drivers/infiniband/hw/ipath/ipath_intr.c b/drivers/infiniband/hw/ipath/ipath_intr.c index dde5dfc..ed82ecb 100644 --- a/drivers/infiniband/hw/ipath/ipath_intr.c +++ b/drivers/infiniband/hw/ipath/ipath_intr.c @@ -366,6 +366,22 @@ static void handle_e_ibstatuschanged(struct ipath_devdata *dd, dd->ipath_ibpollcnt = 0; /* not poll*, now */ ipath_stats.sps_iblink++; + if (ibstate != init && dd->ipath_lastlinkrecov && ipath_linkrecovery) { + u64 linkrecov; + linkrecov = ipath_snap_cntr(dd, + dd->ipath_cregs->cr_iblinkerrrecovcnt); + if (linkrecov != dd->ipath_lastlinkrecov) { + ipath_dbg("IB linkrecov up %Lx (%s %s) recov %Lu\n", + ibcs, ib_linkstate(dd, ibcs), + ipath_ibcstatus_str[ltstate], + linkrecov); + /* and no more until active again */ + dd->ipath_lastlinkrecov = 0; + ipath_set_linkstate(dd, IPATH_IB_LINKDOWN); + goto skip_ibchange; + } + } + if (ibstate == init || ibstate == arm || ibstate == active) { *dd->ipath_statusp &= ~IPATH_STATUS_IB_NOCABLE; if (ibstate == init || ibstate == arm) { @@ -392,6 +408,8 @@ static void handle_e_ibstatuschanged(struct ipath_devdata *dd, IPATH_NOCABLE); ipath_hol_down(dd); } else { /* active */ + dd->ipath_lastlinkrecov = ipath_snap_cntr(dd, + dd->ipath_cregs->cr_iblinkerrrecovcnt); *dd->ipath_statusp |= IPATH_STATUS_IB_READY | IPATH_STATUS_IB_CONF; dd->ipath_flags |= IPATH_LINKACTIVE; diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 9321ad4..7a9121f 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -309,6 +309,7 @@ struct ipath_devdata { ipath_err_t ipath_lasthwerror; /* errors masked because they occur too fast */ ipath_err_t ipath_maskederrs; + u64 ipath_lastlinkrecov; /* link recoveries at last ACTIVE */ /* time in jiffies at which to re-enable maskederrs */ unsigned long ipath_unmasktime; /* count of egrfull errors, combined for all ports */ @@ -1100,6 +1101,7 @@ dma_addr_t ipath_map_single(struct pci_dev *, void *, size_t, int); #endif extern unsigned ipath_debug; /* debugging bit mask */ +extern unsigned ipath_linkrecovery; extern unsigned ipath_mtu4096; #define IPATH_MAX_PARITY_ATTEMPTS 10000 /* max times to try recovery */ From ralph.campbell at qlogic.com Thu Mar 20 17:18:21 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:18:21 -0700 Subject: [ofa-general] [PATCH 09/10] IB/ipath - remove useless comments In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001821.13431.21928.stgit@eng-46.mv.qlogic.com> Remove useless comment about list removal since locks are held and the code checks that the QP is on the list before removing it. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_qp.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 5c8094a..9125e02 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -392,7 +392,6 @@ int ipath_error_qp(struct ipath_qp *qp, enum ib_wc_status err) qp->ibqp.qp_num, qp->remote_qpn, err); spin_lock(&dev->pending_lock); - /* XXX What if its already removed by the timeout code? */ if (!list_empty(&qp->timerwait)) list_del_init(&qp->timerwait); if (!list_empty(&qp->piowait)) @@ -1021,7 +1020,6 @@ void ipath_sqerror_qp(struct ipath_qp *qp, struct ib_wc *wc) qp->ibqp.qp_num, qp->remote_qpn, wc->status); spin_lock(&dev->pending_lock); - /* XXX What if its already removed by the timeout code? */ if (!list_empty(&qp->timerwait)) list_del_init(&qp->timerwait); if (!list_empty(&qp->piowait)) From ralph.campbell at qlogic.com Thu Mar 20 17:18:26 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Thu, 20 Mar 2008 17:18:26 -0700 Subject: [ofa-general] [PATCH 10/10] IB/ipath - Fix sanity checks on QP number of WRs and SGEs In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <20080321001826.13431.64812.stgit@eng-46.mv.qlogic.com> The receive queue number of WRs and SGEs shouldn't be checked if a SRQ is specified. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_qp.c | 26 ++++++++++++++++---------- 1 files changed, 16 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 9125e02..6a4a5e3 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -748,19 +748,25 @@ struct ib_qp *ipath_create_qp(struct ib_pd *ibpd, struct ib_qp *ret; if (init_attr->cap.max_send_sge > ib_ipath_max_sges || - init_attr->cap.max_recv_sge > ib_ipath_max_sges || - init_attr->cap.max_send_wr > ib_ipath_max_qp_wrs || - init_attr->cap.max_recv_wr > ib_ipath_max_qp_wrs) { - ret = ERR_PTR(-ENOMEM); + init_attr->cap.max_send_wr > ib_ipath_max_qp_wrs) { + ret = ERR_PTR(-EINVAL); goto bail; } - if (init_attr->cap.max_send_sge + - init_attr->cap.max_recv_sge + - init_attr->cap.max_send_wr + - init_attr->cap.max_recv_wr == 0) { - ret = ERR_PTR(-EINVAL); - goto bail; + /* Check receive queue parameters if no SRQ is specified. */ + if (!init_attr->srq) { + if (init_attr->cap.max_recv_sge > ib_ipath_max_sges || + init_attr->cap.max_recv_wr > ib_ipath_max_qp_wrs) { + ret = ERR_PTR(-EINVAL); + goto bail; + } + if (init_attr->cap.max_send_sge + + init_attr->cap.max_send_wr + + init_attr->cap.max_recv_sge + + init_attr->cap.max_recv_wr == 0) { + ret = ERR_PTR(-EINVAL); + goto bail; + } } switch (init_attr->qp_type) { From weiny2 at llnl.gov Thu Mar 20 18:13:34 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 20 Mar 2008 18:13:34 -0700 Subject: [ofa-general] [PATCH 0/3] trap 144, node descriptor change handling. Message-ID: <20080320181334.64b53e5b.weiny2@llnl.gov> This patch series starts to handle the new trap 144 features in the 1.2.1 spec; specifically the node descriptor change flag. The first 2 patches enabled testing of the feature and the last patch enables opensm to send a get node descriptor request for ports which send this trap. Future patches will hopefully clean up this trap handling code but I did not want to mix that with this basic functionality. Ira From weiny2 at llnl.gov Thu Mar 20 18:13:36 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 20 Mar 2008 18:13:36 -0700 Subject: [ofa-general] [PATCH 1/3] infiniband-diags/scripts/set_nodedesc.sh: enhance to be able to set names other than hostname and to provide feedback on the names assigned. Message-ID: <20080320181336.31f31870.weiny2@llnl.gov> >From e4da62e7fa20921af62f65f98d23c8154acdad1a Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Mon, 17 Mar 2008 14:46:45 -0700 Subject: [PATCH] infiniband-diags/scripts/set_nodedesc.sh: enhance to be able to set names other than hostname and to provide feedback on the names assigned. Signed-off-by: Ira K. Weiny --- infiniband-diags/scripts/set_nodedesc.sh | 47 ++++++++++++++++++++++++++++-- 1 files changed, 44 insertions(+), 3 deletions(-) diff --git a/infiniband-diags/scripts/set_nodedesc.sh b/infiniband-diags/scripts/set_nodedesc.sh index 7b817cc..55a2fd1 100755 --- a/infiniband-diags/scripts/set_nodedesc.sh +++ b/infiniband-diags/scripts/set_nodedesc.sh @@ -1,17 +1,58 @@ #!/bin/sh -# set the node_desc field of any hca found to the defined hostname - +if [ -f /etc/sysconfig/network ]; then . /etc/sysconfig/network +fi ib_sysfs="/sys/class/infiniband" +newname="$HOSTNAME" + + +function usage +{ + echo "Usage: `basename $0` [-hv] []" + echo " set the node_desc field of all hca's found in \"$ib_sysfs\"" + echo " -h this help" + echo " -v view all node descriptors" + echo " [] set name to name specified." + echo " Default is to use the hostname: \"$HOSTNAME\"" + exit 2 +} + +function viewall +{ + for hca in `ls $ib_sysfs`; do + if [ -f $ib_sysfs/$hca/node_desc ]; then + echo -n "$hca: " + cat $ib_sysfs/$hca/node_desc + else + logger -s "Failed to set node_desc for : $hca" + fi + done + exit 0 +} + +while getopts "hv" flag +do + case $flag in + "h") usage;; + "v") viewall;; + esac +done + +shift $(($OPTIND - 1)) + +if [ "$1" != "" ]; then + newname="$1" +fi for hca in `ls $ib_sysfs`; do if [ -f $ib_sysfs/$hca/node_desc ]; then - echo -n "$HOSTNAME" >> $ib_sysfs/$hca/node_desc + echo -n "$newname" >> $ib_sysfs/$hca/node_desc else logger -s "Failed to set node_desc for : $hca" fi done exit 0 + -- 1.5.1 From weiny2 at llnl.gov Thu Mar 20 18:13:39 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 20 Mar 2008 18:13:39 -0700 Subject: [ofa-general] [PATCH 3/3] opensm/opensm/osm_trap_rcv.c: respond to new trap 144 node descriptor update flag Message-ID: <20080320181339.33e0b62f.weiny2@llnl.gov> >From 99c54c9d5dead4098a0f0660d235a845300b1868 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 20 Mar 2008 17:59:01 -0700 Subject: [PATCH] opensm/opensm/osm_trap_rcv.c: respond to new trap 144 node descriptor update flag Signed-off-by: Ira K. Weiny --- opensm/opensm/osm_node_info_rcv.c | 40 +++++++++++++++++++++++++----------- opensm/opensm/osm_trap_rcv.c | 22 ++++++++++++++++++++ 2 files changed, 50 insertions(+), 12 deletions(-) diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 9b2c74c..6818e05 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -306,17 +306,41 @@ __osm_ni_rcv_process_new_node(IN osm_sm_t * sm, /********************************************************************** The plock must be held before calling this function. **********************************************************************/ +void +osm_req_get_node_desc(IN osm_sm_t * sm, + osm_physp_t *p_physp) +{ + ib_api_status_t status = IB_SUCCESS; + osm_madw_context_t context; + + OSM_LOG_ENTER(sm->p_log); + + context.nd_context.node_guid = + osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); + + status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), + IB_MAD_ATTR_NODE_DESC, + 0, CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) + OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " + "Failure initiating NodeDescription request (%s)\n", + ib_get_err_str(status)); + + OSM_LOG_EXIT(sm->p_log); +} + +/********************************************************************** + The plock must be held before calling this function. +**********************************************************************/ static void __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, IN osm_node_t * const p_node, IN const osm_madw_t * const p_madw) { - ib_api_status_t status = IB_SUCCESS; - osm_madw_context_t context; - osm_physp_t *p_physp; ib_node_info_t *p_ni; ib_smp_t *p_smp; uint8_t port_num; + osm_physp_t *p_physp = NULL; OSM_LOG_ENTER(sm->p_log); @@ -334,15 +358,7 @@ __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, */ p_physp = osm_node_get_physp_ptr(p_node, port_num); - context.nd_context.node_guid = osm_node_get_node_guid(p_node); - - status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), - IB_MAD_ATTR_NODE_DESC, - 0, CL_DISP_MSGID_NONE, &context); - if (status != IB_SUCCESS) - OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " - "Failure initiating NodeDescription request (%s)\n", - ib_get_err_str(status)); + osm_req_get_node_desc(sm, p_physp); OSM_LOG_EXIT(sm->p_log); } diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c index 5cf5a21..7453929 100644 --- a/opensm/opensm/osm_trap_rcv.c +++ b/opensm/opensm/osm_trap_rcv.c @@ -61,6 +61,8 @@ #include #include +extern void osm_req_get_node_desc(IN osm_sm_t * sm, osm_physp_t *p_physp); + /********************************************************************** * * TRAP HANDLING: @@ -577,6 +579,26 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, } } + /* Check for node descriptor update. IB Spec v1.2.1 pg 823*/ + if ((p_ntci->data_details.ntc_144.local_changes & TRAP_144_MASK_OTHER_LOCAL_CHANGES) && + (p_ntci->data_details.ntc_144.change_flgs & TRAP_144_MASK_NODE_DESCRIPTION_CHANGE) + ) { + + osm_node_t *p_node = NULL; + + OSM_LOG(sm->p_log, OSM_LOG_INFO, "Trap 144 Node descriptor update\n"); + + if (p_physp) { + CL_PLOCK_ACQUIRE(sm->p_lock); + osm_req_get_node_desc(sm, p_physp); + CL_PLOCK_RELEASE(sm->p_lock); + } else { + OSM_LOG(sm->p_log, OSM_LOG_ERROR, + "ERR 3812: No physical port found for " + "trap 144: \"node descriptor update\"\n"); + } + } + /* do a sweep if we received a trap */ if (sm->p_subn->opt.sweep_on_trap) { /* if this is trap number 128 or run_heavy_sweep is TRUE - update the -- 1.5.1 From weiny2 at llnl.gov Thu Mar 20 18:13:38 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 20 Mar 2008 18:13:38 -0700 Subject: [ofa-general] [PATCH 2/3] New utility ibsendtrap Message-ID: <20080320181338.11498acb.weiny2@llnl.gov> >From bd6213e232cb3e86574fd21cb13fdc00ffcd0a02 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Mon, 17 Mar 2008 15:20:05 -0700 Subject: [PATCH] New utility ibsendtrap Signed-off-by: Ira K. Weiny --- infiniband-diags/Makefile.am | 6 +- infiniband-diags/src/ibsendtrap.c | 179 +++++++++++++++++++++++++++++++++++++ 2 files changed, 184 insertions(+), 1 deletions(-) create mode 100644 infiniband-diags/src/ibsendtrap.c diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index 07a6233..85411ea 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -10,7 +10,7 @@ endif sbin_PROGRAMS = src/ibaddr src/ibnetdiscover src/ibping src/ibportstate \ src/ibroute src/ibstat src/ibsysstat src/ibtracert \ src/perfquery src/sminfo src/smpdump src/smpquery \ - src/saquery src/vendstat + src/saquery src/vendstat src/ibsendtrap sbin_SCRIPTS = scripts/ibcheckerrs scripts/ibchecknet scripts/ibchecknode \ scripts/ibcheckport scripts/ibhosts scripts/ibstatus \ @@ -71,6 +71,10 @@ src_saquery_SOURCES = src/saquery.c src/ibdiag_common.c src_saquery_CFLAGS = -Wall -DOSM_VENDOR_INTF_OPENIB -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) src_saquery_LDFLAGS = -Wl,--rpath -Wl,$(libdir) +src_ibsendtrap_SOURCES = src/ibsendtrap.c src/ibdiag_common.c +src_ibsendtrap_CFLAGS = -Wall $(DBGFLAGS) +src_ibsendtrap_LDFLAGS = -Wl,--rpath -Wl,$(libdir) + src_vendstat_SOURCES = src/vendstat.c src/ibdiag_common.c src_vendstat_CFLAGS = -Wall $(DBGFLAGS) diff --git a/infiniband-diags/src/ibsendtrap.c b/infiniband-diags/src/ibsendtrap.c new file mode 100644 index 0000000..ae5069b --- /dev/null +++ b/infiniband-diags/src/ibsendtrap.c @@ -0,0 +1,179 @@ +/* + * Copyright (c) 2008 Lawrence Livermore National Security + * + * Produced at Lawrence Livermore National Laboratory. + * Written by Ira Weiny . + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +#include +#include + +#define _GNU_SOURCE +#include + +#include +#include + +#include "ibdiag_common.h" + +char *argv0 = ""; +char *sa_hca_name = NULL; +uint32_t sa_port_num = 0; + + +static int +send_144_node_desc_update(void) +{ + ib_portid_t sm_port; + ib_portid_t selfportid; + int selfport; + ib_rpc_t trap_rpc; + ib_mad_notice_attr_t notice; + + if (ib_resolve_self(&selfportid, &selfport, NULL)) + IBERROR("can't resolve self"); + + if (ib_resolve_smlid(&sm_port, 0)) + IBERROR("can't resolve SM destination port"); + + memset(&trap_rpc, 0, sizeof(trap_rpc)); + trap_rpc.mgtclass = IB_SMI_CLASS; + trap_rpc.method = IB_MAD_METHOD_TRAP; + trap_rpc.trid = mad_trid(); + trap_rpc.attr.id = NOTICE; + trap_rpc.datasz = IB_SMP_DATA_SIZE; + trap_rpc.dataoffs = IB_SMP_DATA_OFFS; + + memset(¬ice, 0, sizeof(notice)); + notice.generic_type = 0x80 | IB_NOTICE_TYPE_INFO; + notice.g_or_v.generic.prod_type_lsb = cl_hton16(IB_NODE_TYPE_CA); + notice.g_or_v.generic.trap_num = cl_hton16(144); + notice.issuer_lid = cl_hton16(selfportid.lid); + notice.data_details.ntc_144.lid = cl_hton16(selfportid.lid); + notice.data_details.ntc_144.local_changes = TRAP_144_MASK_OTHER_LOCAL_CHANGES; + notice.data_details.ntc_144.change_flgs = TRAP_144_MASK_NODE_DESCRIPTION_CHANGE; + + return (mad_send(&trap_rpc, &sm_port, NULL, ¬ice)); +} + +typedef struct _trap_def { + char *trap_name; + int (*send_func)(void); +} trap_def_t; + +trap_def_t traps[2] = { + { "node_desc_change", send_144_node_desc_update }, + { NULL, NULL } +}; + +static void +usage(void) +{ + int i = 0; + fprintf(stderr, "Usage: %s [-hV]" + "[-C -P ] \n", + argv0); + fprintf(stderr, " Queries node records by default\n"); + fprintf(stderr, " -C specify the SA query HCA\n"); + fprintf(stderr, " -P specify the SA query port\n"); + fprintf(stderr, " -V print version\n"); + fprintf(stderr, " can be one of the following\n"); + for (i=0; traps[i].trap_name; i++) { + fprintf(stderr, " %s\n", traps[i].trap_name); + } + fprintf(stderr, " default behavior is to send \"%s\"\n", + traps[0].trap_name); + + exit(-1); +} + +int +send_trap(char *trap_name) +{ + int i = 0; + for (i=0; traps[i].trap_name; i++) { + if (strcmp(traps[i].trap_name, trap_name) == 0) { + return (traps[i].send_func()); + } + } + usage(); + exit(1); +} + +int +main(int argc, char **argv) +{ + int mgmt_classes[2] = {IB_SMI_CLASS, IB_SMI_DIRECT_CLASS}; + int ch = 0; + char *trap_name = NULL; + + static char const str_opts[] = "hVP:C:"; + static const struct option long_opts [] = { + {"Version", 0, 0, 'V'}, + {"P", 1, 0, 'P'}, + {"C", 1, 0, 'C'}, + {"help", 0, 0, 'h'}, + { } + }; + + argv0 = argv[0]; + + while ((ch = getopt_long(argc, argv, str_opts, long_opts, NULL)) != -1) { + switch (ch) { + case 'V': + fprintf(stderr, "%s %s\n", argv0, get_build_version()); + exit(-1); + case 'C': + sa_hca_name = optarg; + break; + case 'P': + sa_port_num = strtoul(optarg, NULL, 0); + break; + case 'h': + default: + usage(); + } + } + argc -= optind; + argv += optind; + + if (!argv[0]) { + trap_name = traps[0].trap_name; + } else { + trap_name = argv[0]; + } + + madrpc_show_errors(1); + madrpc_init(NULL, 0, mgmt_classes, 2); + + return (send_trap(trap_name)); +} -- 1.5.1 From Golden-01 at Mail.ru Thu Mar 20 18:28:56 2008 From: Golden-01 at Mail.ru () Date: Fri, 21 Mar 2008 04:28:56 +0300 Subject: [ofa-general] =?iso-8859-1?q?=C2=E0=F8_=E4=EE=EF=EE=EB=ED=E8=F2?= =?iso-8859-1?q?=E5=EB=FC=ED=FB=E9_=E7=E0=F0=E0=E1=EE=F2=EE=EA=2C?= =?iso-8859-1?q?=E0_=E2=EE=E7=EC=EE=E6=ED=EE_=E8_=EE=E1=E5=F1=EF=E5?= =?iso-8859-1?q?=F7=E5=ED=ED=EE=E5_=E1=F3=E4=F3=F9=E5=E5=2E?= Message-ID: <200803210428.PTJMLPEFMC@82.138.52.149>                                                                                Добрый день !      Извините,что,возможно ,отнимаю у Вас время,но если Вы имеете  доступ к Интернету,умеете пользоваться  мышкой,   имеете 2-3 часа свободного времени,и Вас интересуют деньги,то всего один щелчок мышкой может отделять Вас от состояния !!! Еще раз простите за вторжение.Желаю Вам удачного дня . Жду Вашего ответа : Moskvich-V at Mail.ru  или  Golden-V at Yandex.ru Удачи Вам !!! Информация находиться в прикрепленном   файле  .... Не беспокойтесь, это не вирус,но если Вы ,тем не менее,  опасаетесь  его  открыть  ,не  спешите  его  удалять ,вовсе не сложно проверить почту при помощи антивируса. Это не реклама, а действительно выгодное предложение !!! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Golden Stream.doc Type: application/octet-stream Size: 160256 bytes Desc: not available URL: From a-2 at absopc.com Thu Mar 20 19:25:50 2008 From: a-2 at absopc.com (Winnie Gilliam) Date: Fri, 21 Mar 2008 10:25:50 +0800 Subject: [ofa-general] Long time, no talk Message-ID: <01c88b3d$ef0e0b00$cdeb8edb@a-2> Hello! I am tired this evening. I am nice girl that would like to chat with you. Email me at Victoria at BestGolova.com only, because I am using my friend's email to write this. Will send some of my pictures From info at pleshood.com Thu Mar 20 22:28:34 2008 From: info at pleshood.com (=?windows-1255?B?7vgg6eXh7CD44uE=?=) Date: Fri, 21 Mar 2008 00:28:34 -0500 Subject: [ofa-general] =?windows-1255?b?5ODtIOD0+fgg7OT45ennIOLtIOzs4CDk?= =?windows-1255?b?8enr5fDp7Q==?= Message-ID: <20080321052933.E0049E60974@openfabrics.org> An HTML attachment was scrubbed... URL: From info at oborksac.com Fri Mar 21 00:06:15 2008 From: info at oborksac.com (=?windows-1255?B?4/gnIPnpIPHl7O8=?=) Date: Fri, 21 Mar 2008 02:06:15 -0500 Subject: [ofa-general] =?windows-1255?b?7uQg5Pnp6OQg7OT56eIg+OXl5+ntIOHy?= =?windows-1255?b?8fc=?= ? Message-ID: <20080321070714.67623E60ACF@openfabrics.org> An HTML attachment was scrubbed... URL: From slacksoj99 at harryshouseofheat.com Fri Mar 21 00:45:13 2008 From: slacksoj99 at harryshouseofheat.com (Maria Vernon) Date: Fri, 21 Mar 2008 09:45:13 +0200 Subject: [ofa-general] Viagra 50mg x 10 pills $59.95 price Message-ID: <01c88b38$427d5a80$0c916050@slacksoj99> Viagra 50mg x 60 pills $119.95 http://ruthchristnerdr309.blogspot.com From manni at guestbook.com Fri Mar 21 01:21:11 2008 From: manni at guestbook.com (Alphonse Skinner) Date: Fri, 21 Mar 2008 17:21:11 +0900 Subject: [ofa-general] Proffesionale Mittel gegen Imp.// ()/.. tenz Message-ID: <069752974.04576189222521@guestbook.com> Sie leben nur einmal - warum dann nicht was neues ausprobieren? Pr. .. Eise die keine Konk... ..Urrenz kennen - Bequem und dis kret 0... . N-line! be... .Stellen. - Disk rete Verpackung und Zahlung - Kein peinlicher Arz t besuch erforderlich - Kos... Tenlose, arztliche Telefon-Beratung - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - keine versteckte Kos// Ten Originalme/ dikamente Ciia..aa^_^aaalis...... 10 Pack. 21,00 Euro Viia..aa^_^aaagra... 10 Pack. 11,00 Euro Vier Dosen gibt's bei jeder Bestellung umsonst http://tonethrow.com (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From aved at cslanet.calstatela.edu Fri Mar 21 01:36:57 2008 From: aved at cslanet.calstatela.edu (Jeannette Maher) Date: Fri, 21 Mar 2008 09:36:57 +0100 Subject: [ofa-general] Kaufen Sie Softwareloesungen billiger Message-ID: <840784043.13140399605917@cslanet.calstatela.edu> An HTML attachment was scrubbed... URL: From rmorris at onlinetickets.com Fri Mar 21 02:17:36 2008 From: rmorris at onlinetickets.com (Zane Felix) Date: Fri, 21 Mar 2008 14:17:36 +0500 Subject: [ofa-general] Kaufen Sie Softwareloesungen billiger Message-ID: <478429418.68492813883598@onlinetickets.com> An HTML attachment was scrubbed... URL: From cwainc at startribune.com Fri Mar 21 03:26:27 2008 From: cwainc at startribune.com (Mossie Arellano) Date: Fri, 21 Mar 2008 10:26:27 +0000 Subject: [ofa-general] Grosse Auswahl der Software zum runterladen Message-ID: <01c88b3e$051bcb80$fe1bd45a@cwainc> Industry standard software at nominal feeSie konnen unsere Software sofort runterladen. Zahlen Sie und die Downloads werden Ihnen sofort zur Verfugung stehen. Unsere Programme wurden fuer mehrere OS (Windows und Macintosh) programmiert und sind in allen europaeischen Sprachen verfuegbar. Wir verkaufen nur originale Vollversionen, haben aber die guenstigsten Preise. http://geocities.com/odomamie/* Office Enterprise 2007: $79.95 * Windows XP Professional With SP2 Full Version: $59.95 * Adobe Photoshop CS3 Extended: $79.95 * Adobe Acrobat 8.0 Professional: $69.95 http://geocities.com/odomamie/Unsere Kundenberater sind immer bereit Ihnen bei der Installation zu helfen. Wir antworten sehr schnell und geben Ihnen auch Geld-Zurueck-Garantie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fk3mtpulepaje at post.fukubiki.com Fri Mar 21 03:30:24 2008 From: fk3mtpulepaje at post.fukubiki.com (Fredric Moreno) Date: Fri, 21 Mar 2008 11:30:24 +0100 Subject: [ofa-general] Ihnen werden unsere Softwarepreise gefallen Message-ID: <782570110.40294602257593@post.fukubiki.com> Legal software salesBekommen Sie Ihre Software unverzueglich. Einfach zahlen und sofort runterladen. Hier sind Programme in allen europaeischen Sprachen verfuegbar, programmiert fuer Windows und Macintosh. Alle Softwaren sind sehr guenstig, es handelt sich dabei garantiert um originale, komplette und voellig funktionale Versionen. http://geocities.com/robynnorman70/* Windows XP Professional With SP2 Full Version: $59.95 * Adobe Acrobat 8.0 Professional: $69.95 * Office System Professional 2003 (5 Cds): $59.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/robynnorman70/Bei uns kaufen Sie sicher ein, denn unsere kompetenten Mitarbeiten vom Kundencenter werden Ihnen bei der Softwareinstallation weiterhelfen. Wir antworten unverzueglich und Sie bekommen von uns eine Geld-Zurueck-Garantie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-anitaz at abakomp.dk Fri Mar 21 03:51:49 2008 From: a-anitaz at abakomp.dk (Deloris Johnson) Date: Fri, 21 Mar 2008 18:51:49 +0800 Subject: [ofa-general] You told me that you will reply back Message-ID: <01c88b84$9e6dd080$10b54bda@a-anitaz> Hello! I am bored this evening. I am nice girl that would like to chat with you. Email me at Isabella at BestGolova.com only, because I am using my friend's email to write this. Hope you wanna see my pics. From bruton at atlconnect.com Fri Mar 21 04:25:46 2008 From: bruton at atlconnect.com (Lois Shaffer) Date: Fri, 21 Mar 2008 12:25:46 +0100 Subject: [ofa-general] Leich zu installierende Software Message-ID: <040070743.10179224258296@atlconnect.com> An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Fri Mar 21 04:39:43 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 21 Mar 2008 04:39:43 -0700 Subject: [ofa-general] [PATCH 2/3] New utility ibsendtrap In-Reply-To: <20080320181338.11498acb.weiny2@llnl.gov> References: <20080320181338.11498acb.weiny2@llnl.gov> Message-ID: <1206099583.8099.214.camel@hrosenstock-ws.xsigo.com> Ira, On Thu, 2008-03-20 at 18:13 -0700, Ira Weiny wrote: > >From bd6213e232cb3e86574fd21cb13fdc00ffcd0a02 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Mon, 17 Mar 2008 15:20:05 -0700 > Subject: [PATCH] New utility ibsendtrap This looks like a test tool to me rather than a diagnostic. IMO such functionality belongs in the SMA. -- Hal > Signed-off-by: Ira K. Weiny > --- > infiniband-diags/Makefile.am | 6 +- > infiniband-diags/src/ibsendtrap.c | 179 +++++++++++++++++++++++++++++++++++++ > 2 files changed, 184 insertions(+), 1 deletions(-) > create mode 100644 infiniband-diags/src/ibsendtrap.c > > diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am > index 07a6233..85411ea 100644 > --- a/infiniband-diags/Makefile.am > +++ b/infiniband-diags/Makefile.am > @@ -10,7 +10,7 @@ endif > sbin_PROGRAMS = src/ibaddr src/ibnetdiscover src/ibping src/ibportstate \ > src/ibroute src/ibstat src/ibsysstat src/ibtracert \ > src/perfquery src/sminfo src/smpdump src/smpquery \ > - src/saquery src/vendstat > + src/saquery src/vendstat src/ibsendtrap > > sbin_SCRIPTS = scripts/ibcheckerrs scripts/ibchecknet scripts/ibchecknode \ > scripts/ibcheckport scripts/ibhosts scripts/ibstatus \ > @@ -71,6 +71,10 @@ src_saquery_SOURCES = src/saquery.c src/ibdiag_common.c > src_saquery_CFLAGS = -Wall -DOSM_VENDOR_INTF_OPENIB -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) > src_saquery_LDFLAGS = -Wl,--rpath -Wl,$(libdir) > > +src_ibsendtrap_SOURCES = src/ibsendtrap.c src/ibdiag_common.c > +src_ibsendtrap_CFLAGS = -Wall $(DBGFLAGS) > +src_ibsendtrap_LDFLAGS = -Wl,--rpath -Wl,$(libdir) > + > src_vendstat_SOURCES = src/vendstat.c src/ibdiag_common.c > src_vendstat_CFLAGS = -Wall $(DBGFLAGS) > > diff --git a/infiniband-diags/src/ibsendtrap.c b/infiniband-diags/src/ibsendtrap.c > new file mode 100644 > index 0000000..ae5069b > --- /dev/null > +++ b/infiniband-diags/src/ibsendtrap.c > @@ -0,0 +1,179 @@ > +/* > + * Copyright (c) 2008 Lawrence Livermore National Security > + * > + * Produced at Lawrence Livermore National Laboratory. > + * Written by Ira Weiny . > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +#include > +#include > + > +#define _GNU_SOURCE > +#include > + > +#include > +#include > + > +#include "ibdiag_common.h" > + > +char *argv0 = ""; > +char *sa_hca_name = NULL; > +uint32_t sa_port_num = 0; > + > + > +static int > +send_144_node_desc_update(void) > +{ > + ib_portid_t sm_port; > + ib_portid_t selfportid; > + int selfport; > + ib_rpc_t trap_rpc; > + ib_mad_notice_attr_t notice; > + > + if (ib_resolve_self(&selfportid, &selfport, NULL)) > + IBERROR("can't resolve self"); > + > + if (ib_resolve_smlid(&sm_port, 0)) > + IBERROR("can't resolve SM destination port"); > + > + memset(&trap_rpc, 0, sizeof(trap_rpc)); > + trap_rpc.mgtclass = IB_SMI_CLASS; > + trap_rpc.method = IB_MAD_METHOD_TRAP; > + trap_rpc.trid = mad_trid(); > + trap_rpc.attr.id = NOTICE; > + trap_rpc.datasz = IB_SMP_DATA_SIZE; > + trap_rpc.dataoffs = IB_SMP_DATA_OFFS; > + > + memset(¬ice, 0, sizeof(notice)); > + notice.generic_type = 0x80 | IB_NOTICE_TYPE_INFO; > + notice.g_or_v.generic.prod_type_lsb = cl_hton16(IB_NODE_TYPE_CA); > + notice.g_or_v.generic.trap_num = cl_hton16(144); > + notice.issuer_lid = cl_hton16(selfportid.lid); > + notice.data_details.ntc_144.lid = cl_hton16(selfportid.lid); > + notice.data_details.ntc_144.local_changes = TRAP_144_MASK_OTHER_LOCAL_CHANGES; > + notice.data_details.ntc_144.change_flgs = TRAP_144_MASK_NODE_DESCRIPTION_CHANGE; > + > + return (mad_send(&trap_rpc, &sm_port, NULL, ¬ice)); > +} > + > +typedef struct _trap_def { > + char *trap_name; > + int (*send_func)(void); > +} trap_def_t; > + > +trap_def_t traps[2] = { > + { "node_desc_change", send_144_node_desc_update }, > + { NULL, NULL } > +}; > + > +static void > +usage(void) > +{ > + int i = 0; > + fprintf(stderr, "Usage: %s [-hV]" > + "[-C -P ] \n", > + argv0); > + fprintf(stderr, " Queries node records by default\n"); > + fprintf(stderr, " -C specify the SA query HCA\n"); > + fprintf(stderr, " -P specify the SA query port\n"); > + fprintf(stderr, " -V print version\n"); > + fprintf(stderr, " can be one of the following\n"); > + for (i=0; traps[i].trap_name; i++) { > + fprintf(stderr, " %s\n", traps[i].trap_name); > + } > + fprintf(stderr, " default behavior is to send \"%s\"\n", > + traps[0].trap_name); > + > + exit(-1); > +} > + > +int > +send_trap(char *trap_name) > +{ > + int i = 0; > + for (i=0; traps[i].trap_name; i++) { > + if (strcmp(traps[i].trap_name, trap_name) == 0) { > + return (traps[i].send_func()); > + } > + } > + usage(); > + exit(1); > +} > + > +int > +main(int argc, char **argv) > +{ > + int mgmt_classes[2] = {IB_SMI_CLASS, IB_SMI_DIRECT_CLASS}; > + int ch = 0; > + char *trap_name = NULL; > + > + static char const str_opts[] = "hVP:C:"; > + static const struct option long_opts [] = { > + {"Version", 0, 0, 'V'}, > + {"P", 1, 0, 'P'}, > + {"C", 1, 0, 'C'}, > + {"help", 0, 0, 'h'}, > + { } > + }; > + > + argv0 = argv[0]; > + > + while ((ch = getopt_long(argc, argv, str_opts, long_opts, NULL)) != -1) { > + switch (ch) { > + case 'V': > + fprintf(stderr, "%s %s\n", argv0, get_build_version()); > + exit(-1); > + case 'C': > + sa_hca_name = optarg; > + break; > + case 'P': > + sa_port_num = strtoul(optarg, NULL, 0); > + break; > + case 'h': > + default: > + usage(); > + } > + } > + argc -= optind; > + argv += optind; > + > + if (!argv[0]) { > + trap_name = traps[0].trap_name; > + } else { > + trap_name = argv[0]; > + } > + > + madrpc_show_errors(1); > + madrpc_init(NULL, 0, mgmt_classes, 2); > + > + return (send_trap(trap_name)); > +} From hrosenstock at xsigo.com Fri Mar 21 04:45:41 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 21 Mar 2008 04:45:41 -0700 Subject: [ofa-general] A couple of minor questions about ibutils building and git tree Message-ID: <1206099942.8099.220.camel@hrosenstock-ws.xsigo.com> Oren, I have a couple of minor questions about ibutils building and it's git tree. When I build and install I see the following modifications: # modified: ibdiag/doc/ibdiagui.1 # modified: ibis/COPYING Having these is a little dangerous as they might get checked in by mistake ? Is ibdiagui.1 a generated file ? If so, should it be in the tree ? diff --git a/ibdiag/doc/ibdiagui.1 b/ibdiag/doc/ibdiagui.1 index 3aa898f..0dfe8a1 100644 --- a/ibdiag/doc/ibdiagui.1 +++ b/ibdiag/doc/ibdiagui.1 @@ -1,4 +1,4 @@ -.\" Automatically generated by Pod::Man v1.34, Pod::Parser v1.13 +.\" Automatically generated by Pod::Man v1.37, Pod::Parser v1.14 .\" .\" Standard preamble: .\" ======================================================================== @@ -129,7 +129,7 @@ .\" ======================================================================== .\" .IX Title "IBDIAGUI 1" -.TH IBDIAGUI 1 "2006-11-17" "IBDIAG 1.0" "IB DIAGNOSTIC PACKAGE" +.TH IBDIAGUI 1 "2008-03-04" "IBDIAG 1.0" "IB DIAGNOSTIC PACKAGE" .SH "NAME" \&\fBibdiagui \- IB Diagnostic GUI\fR .SH "SYNOPSYS" The ibis/COPYING file in the git tree looks like it has the standard OpenFabrics license whereas after building it has GPL. Thanks. -- Hal From jforgette at spancrete.com Fri Mar 21 05:24:52 2008 From: jforgette at spancrete.com (Vida Thayer) Date: Fri, 21 Mar 2008 13:24:52 +0100 Subject: [ofa-general] Originale Softwareprodukte Message-ID: <045218367.61422943407087@spancrete.com> An HTML attachment was scrubbed... URL: From reparteed60 at shannonplumb.com Fri Mar 21 05:34:59 2008 From: reparteed60 at shannonplumb.com (Lionel Foster) Date: Fri, 21 Mar 2008 19:34:59 +0700 Subject: [ofa-general] Penis consists of three chambers, two of which are on the top (Corpora Cavernosa) Message-ID: <01c88b8a$a6307380$183a137b@reparteed60> A completely high-quality and safe natural herbal formula http://tammypringlenm166.blogspot.com From pluperfectmx93 at 10thchurch.com Fri Mar 21 05:49:05 2008 From: pluperfectmx93 at 10thchurch.com (Esther Ritchie) Date: Fri, 21 Mar 2008 04:49:05 -0800 Subject: [ofa-general] Viagra 100mg x 90 pills US $ 159.95 price Message-ID: <01c88b0e$e480fa00$220e0275@pluperfectmx93> 50mg x 60 pills US $ 2.00 Per Pill http://imogenecorneliusrm603.blogspot.com From Golden-01 at Mail.ru Fri Mar 21 05:50:23 2008 From: Golden-01 at Mail.ru (=?Windows-1251?b?wuvg5Ojs6PA=?=) Date: Fri, 21 Mar 2008 15:50:23 +0300 Subject: [ofa-general] =?windows-1251?b?wuD4IOTu7+7r7ejy5ev87fvpIOfg8ODh?= =?windows-1251?b?7vLu6g==?= , =?windows-1251?b?4CDi7ufs7ubt7iDoIO7h5fHv5ffl7e3u5SDh8+Tz+eXl?= . Message-ID: <20080321125123.DD784E60A0A@openfabrics.org> Добрый день ! Извините,что,возможно ,отнимаю у Вас время,но если Вы имеете доступ к Интернету,умеете пользоваться мышкой, имеете 2-3 часа свободного времени,и Вас интересуют деньги,то всего один щелчок мышкой может отделять Вас от состояния !!! Еще раз простите за вторжение . Желаю Вам удачного дня. Жду Вашего ответа : Moskvich-V at Mail.ru или Golden-V at Yandex.ru Удачи Вам !!! Информация находиться в прикрепленном файле .... Не беспокойтесь, это не вирус,но если Вы ,тем не менее, опасаетесь его открыть ,не спешите его удалять ,вовсе не сложно проверить почту при помощи антивируса. Это не реклама, а действительно выгодное предложение !!! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Golden Stream.doc Type: application/octet-stream Size: 160256 bytes Desc: not available URL: From er at novman.com Fri Mar 21 06:16:29 2008 From: er at novman.com (Jessenia Valenzuela) Date: Fri, 21 Mar 2008 14:16:29 +0100 Subject: [ofa-general] Sie koennen hier jede Software laden Message-ID: <995423769.23851651486141@novman.com> Cheap and excellent software - too good to be true? Read information belowSie bekommen unsere Software ohne Wartezeiten. Sofort nach der Bezahlung koennen Sie diese runterladen. Unsere Programme wurden nicht nur fuer Windows, sondern auch fuer Macintosh entwickelt und sind in allen europaeischen Sprachen verfuegbar. Wir haben sehr gunstige Preise und es handelt sich dabei nur um vollstandige Originalversionen. http://geocities.com/rosario_kirkland/* Office Enterprise 2007: $79.95 * Windows XP Professional With SP2 Full Version: $59.95 * Adobe Photoshop CS3 Extended: $79.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/rosario_kirkland/Sie werden nach dem Kauf nicht alleine gelassen. Unsere freundlichen Kundenberater werden Ihnen bei der Installation helfen, falls Sie es benotigen. Schnelle Antworten und Geld-Zurueck-Garantie sind bei uns selbstverstandlich. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dwsjmnetm at sjmnet.com Fri Mar 21 06:27:44 2008 From: dwsjmnetm at sjmnet.com (Shana Shelton) Date: Fri, 21 Mar 2008 16:27:44 +0300 Subject: [ofa-general] Medications that you need. Message-ID: <01c88b70$7d9b8000$18550cc3@dwsjmnetm> Buy Must Have medications at Canada based pharmacy. No prescription at all! Same quality! Save your money, buy pills immediately! http://brpqlf.roledbintel.net/?73107694 We provide confidential and secure purchase! From or.gerlitz at gmail.com Fri Mar 21 06:36:58 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Fri, 21 Mar 2008 15:36:58 +0200 Subject: [ofa-general] Re: [ewg] [RFC][0/2] IPoIB UD 4K MTU support In-Reply-To: <1206000222.8399.10.camel@localhost.localdomain> References: <1206000222.8399.10.camel@localhost.localdomain> Message-ID: <15ddcffd0803210636h6e97eecdy3e5403aa4d904957@mail.gmail.com> On 3/20/08, Shirley Ma wrote: > > Here is a patchset to enable IPoIB UD 4K MTU support for any IB fabric > where the max IPoIB payload can be up to 4K. This patchset uses two S/G > buffers when IPoIB payload + IB_GRH header size is greater than > PAGE_SIZE. The first buffer size is IB_GRH_HEAD + IPOIB_ENCAP_LEN. The > second buffer is the data. > Just wanted to make sure, this patchset is targeting the mainline kernel, correct? Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Fri Mar 21 06:52:19 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Fri, 21 Mar 2008 06:52:19 -0700 Subject: [ofa-general] [PATCH 3/3] opensm/opensm/osm_trap_rcv.c: respond to new trap 144 node descriptor update flag In-Reply-To: <20080320181339.33e0b62f.weiny2@llnl.gov> References: <20080320181339.33e0b62f.weiny2@llnl.gov> Message-ID: <1206107539.8099.248.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-20 at 18:13 -0700, Ira Weiny wrote: > >From 99c54c9d5dead4098a0f0660d235a845300b1868 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 20 Mar 2008 17:59:01 -0700 > Subject: [PATCH] opensm/opensm/osm_trap_rcv.c: respond to new trap 144 node descriptor update > flag Some minor stylistic comments below. > Signed-off-by: Ira K. Weiny > --- > opensm/opensm/osm_node_info_rcv.c | 40 +++++++++++++++++++++++++----------- > opensm/opensm/osm_trap_rcv.c | 22 ++++++++++++++++++++ > 2 files changed, 50 insertions(+), 12 deletions(-) > > diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c > index 9b2c74c..6818e05 100644 > --- a/opensm/opensm/osm_node_info_rcv.c > +++ b/opensm/opensm/osm_node_info_rcv.c > @@ -306,17 +306,41 @@ __osm_ni_rcv_process_new_node(IN osm_sm_t * sm, > /********************************************************************** > The plock must be held before calling this function. > **********************************************************************/ > +void > +osm_req_get_node_desc(IN osm_sm_t * sm, > + osm_physp_t *p_physp) > +{ > + ib_api_status_t status = IB_SUCCESS; > + osm_madw_context_t context; > + > + OSM_LOG_ENTER(sm->p_log); > + > + context.nd_context.node_guid = > + osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); > + > + status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), > + IB_MAD_ATTR_NODE_DESC, > + 0, CL_DISP_MSGID_NONE, &context); > + if (status != IB_SUCCESS) > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " > + "Failure initiating NodeDescription request (%s)\n", > + ib_get_err_str(status)); > + > + OSM_LOG_EXIT(sm->p_log); > +} > + > +/********************************************************************** > + The plock must be held before calling this function. > +**********************************************************************/ > static void > __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, > IN osm_node_t * const p_node, > IN const osm_madw_t * const p_madw) > { > - ib_api_status_t status = IB_SUCCESS; > - osm_madw_context_t context; > - osm_physp_t *p_physp; > ib_node_info_t *p_ni; > ib_smp_t *p_smp; > uint8_t port_num; > + osm_physp_t *p_physp = NULL; > > OSM_LOG_ENTER(sm->p_log); > > @@ -334,15 +358,7 @@ __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, > */ > p_physp = osm_node_get_physp_ptr(p_node, port_num); > > - context.nd_context.node_guid = osm_node_get_node_guid(p_node); > - > - status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), > - IB_MAD_ATTR_NODE_DESC, > - 0, CL_DISP_MSGID_NONE, &context); > - if (status != IB_SUCCESS) > - OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " > - "Failure initiating NodeDescription request (%s)\n", > - ib_get_err_str(status)); > + osm_req_get_node_desc(sm, p_physp); > > OSM_LOG_EXIT(sm->p_log); > } > diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c > index 5cf5a21..7453929 100644 > --- a/opensm/opensm/osm_trap_rcv.c > +++ b/opensm/opensm/osm_trap_rcv.c > @@ -61,6 +61,8 @@ > #include > #include > > +extern void osm_req_get_node_desc(IN osm_sm_t * sm, osm_physp_t *p_physp); Shouldn't this be added to the osm_node_desc_rcv.h header and that header included in osm_trap_rcv.c ? > + > /********************************************************************** > * > * TRAP HANDLING: > @@ -577,6 +579,26 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, > } > } > > + /* Check for node descriptor update. IB Spec v1.2.1 pg 823*/ > + if ((p_ntci->data_details.ntc_144.local_changes & TRAP_144_MASK_OTHER_LOCAL_CHANGES) && > + (p_ntci->data_details.ntc_144.change_flgs & TRAP_144_MASK_NODE_DESCRIPTION_CHANGE) > + ) { > + > + osm_node_t *p_node = NULL; > + > + OSM_LOG(sm->p_log, OSM_LOG_INFO, "Trap 144 Node descriptor update\n"); Node description r.t. Node descriptor > + if (p_physp) { > + CL_PLOCK_ACQUIRE(sm->p_lock); > + osm_req_get_node_desc(sm, p_physp); > + CL_PLOCK_RELEASE(sm->p_lock); > + } else { > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, > + "ERR 3812: No physical port found for " > + "trap 144: \"node descriptor update\"\n"); Node description r.t. node descriptor > + } > + } > + > /* do a sweep if we received a trap */ > if (sm->p_subn->opt.sweep_on_trap) { > /* if this is trap number 128 or run_heavy_sweep is TRUE - update the > -- > 1.5.1 > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From pvardu at bracherrawlins.com Fri Mar 21 07:14:02 2008 From: pvardu at bracherrawlins.com (Otto Wagner) Date: Fri, 21 Mar 2008 16:14:02 +0200 Subject: [ofa-general] Take her to love heaven Message-ID: <01c88b6e$93e1c790$5cbb7b59@pvardu> She wants you more now Look attached details and find MORE! -------------- next part -------------- A non-text attachment was scrubbed... Name: vo.zip Type: application/zip Size: 636 bytes Desc: not available URL: From Arkady.Kanevsky at netapp.com Fri Mar 21 07:51:00 2008 From: Arkady.Kanevsky at netapp.com (Kanevsky, Arkady) Date: Fri, 21 Mar 2008 10:51:00 -0400 Subject: [ofa-general] files preambule Message-ID: I noticed that all files in the preambule states that the file can be taken under GNU GPLv2 or OpenIB BSD licence. Should not it state OpenFabrics instead? Thanks, Arkady Kanevsky email: arkady at netapp.com Network Appliance Inc. phone: 781-768-5395 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 Waltham, MA 02451 central phone: 781-768-5300 -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.huang at la.foxconn.com Fri Mar 21 07:52:05 2008 From: eric.huang at la.foxconn.com (Ursula Franco) Date: Fri, 21 Mar 2008 06:52:05 -0800 Subject: [ofa-general] Weniger zahlen fuer perfekte Standardsoftware Message-ID: <651362076.78162985228178@la.foxconn.com> An HTML attachment was scrubbed... URL: From a-ando at aba-arch.com Fri Mar 21 07:57:34 2008 From: a-ando at aba-arch.com (Jody Jensen) Date: Fri, 21 Mar 2008 21:57:34 +0700 Subject: [ofa-general] I was looking for you Message-ID: <01c88b9e$915ddb00$4337167b@a-ando> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Lena at BestGolova.com only, because I am using my friend's email to write this. I will reply with my pics From mashirle at us.ibm.com Thu Mar 20 23:17:30 2008 From: mashirle at us.ibm.com (Shirley Ma) Date: Thu, 20 Mar 2008 23:17:30 -0700 Subject: [ofa-general] Re: [ewg] [RFC][0/2] IPoIB UD 4K MTU support In-Reply-To: <15ddcffd0803210636h6e97eecdy3e5403aa4d904957@mail.gmail.com> References: <1206000222.8399.10.camel@localhost.localdomain> <15ddcffd0803210636h6e97eecdy3e5403aa4d904957@mail.gmail.com> Message-ID: <1206080250.8399.33.camel@localhost.localdomain> Hello Or, > Just wanted to make sure, this patchset is targeting the mainline > kernel, correct? > > Or. Yes. I used Roland's git tree for-2.6.25 branch. I didn't see for-2.6.26 branch. thanks Shirley From breezedtsf8 at mcguirecorp.com Fri Mar 21 09:16:10 2008 From: breezedtsf8 at mcguirecorp.com (Parker Ervin) Date: Fri, 21 Mar 2008 17:16:10 +0100 Subject: [ofa-general] Increases blood flow in the penis and... Message-ID: <01c88b77$41b7e900$9cac1957@breezedtsf8> When man goes along the street and something big sticks out of his jeans... http://colettesligarcs493.blogspot.com From weiny2 at llnl.gov Fri Mar 21 09:49:59 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 21 Mar 2008 09:49:59 -0700 Subject: [ofa-general] [PATCH 2/3] New utility ibsendtrap In-Reply-To: <1206099583.8099.214.camel@hrosenstock-ws.xsigo.com> References: <20080320181338.11498acb.weiny2@llnl.gov> <1206099583.8099.214.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080321094959.5ce4a536.weiny2@llnl.gov> On Fri, 21 Mar 2008 04:39:43 -0700 Hal Rosenstock wrote: > Ira, > > On Thu, 2008-03-20 at 18:13 -0700, Ira Weiny wrote: > > >From bd6213e232cb3e86574fd21cb13fdc00ffcd0a02 Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Mon, 17 Mar 2008 15:20:05 -0700 > > Subject: [PATCH] New utility ibsendtrap > > This looks like a test tool to me rather than a diagnostic. IMO such > functionality belongs in the SMA. Agreed, new patch series to follow, Ira > > -- Hal > > > Signed-off-by: Ira K. Weiny > > --- > > infiniband-diags/Makefile.am | 6 +- > > infiniband-diags/src/ibsendtrap.c | 179 +++++++++++++++++++++++++++++++++++++ > > 2 files changed, 184 insertions(+), 1 deletions(-) > > create mode 100644 infiniband-diags/src/ibsendtrap.c > > > > diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am > > index 07a6233..85411ea 100644 > > --- a/infiniband-diags/Makefile.am > > +++ b/infiniband-diags/Makefile.am > > @@ -10,7 +10,7 @@ endif > > sbin_PROGRAMS = src/ibaddr src/ibnetdiscover src/ibping src/ibportstate \ > > src/ibroute src/ibstat src/ibsysstat src/ibtracert \ > > src/perfquery src/sminfo src/smpdump src/smpquery \ > > - src/saquery src/vendstat > > + src/saquery src/vendstat src/ibsendtrap > > > > sbin_SCRIPTS = scripts/ibcheckerrs scripts/ibchecknet scripts/ibchecknode \ > > scripts/ibcheckport scripts/ibhosts scripts/ibstatus \ > > @@ -71,6 +71,10 @@ src_saquery_SOURCES = src/saquery.c src/ibdiag_common.c > > src_saquery_CFLAGS = -Wall -DOSM_VENDOR_INTF_OPENIB -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) > > src_saquery_LDFLAGS = -Wl,--rpath -Wl,$(libdir) > > > > +src_ibsendtrap_SOURCES = src/ibsendtrap.c src/ibdiag_common.c > > +src_ibsendtrap_CFLAGS = -Wall $(DBGFLAGS) > > +src_ibsendtrap_LDFLAGS = -Wl,--rpath -Wl,$(libdir) > > + > > src_vendstat_SOURCES = src/vendstat.c src/ibdiag_common.c > > src_vendstat_CFLAGS = -Wall $(DBGFLAGS) > > > > diff --git a/infiniband-diags/src/ibsendtrap.c b/infiniband-diags/src/ibsendtrap.c > > new file mode 100644 > > index 0000000..ae5069b > > --- /dev/null > > +++ b/infiniband-diags/src/ibsendtrap.c > > @@ -0,0 +1,179 @@ > > +/* > > + * Copyright (c) 2008 Lawrence Livermore National Security > > + * > > + * Produced at Lawrence Livermore National Laboratory. > > + * Written by Ira Weiny . > > + * > > + * This software is available to you under a choice of one of two > > + * licenses. You may choose to be licensed under the terms of the GNU > > + * General Public License (GPL) Version 2, available from the file > > + * COPYING in the main directory of this source tree, or the > > + * OpenIB.org BSD license below: > > + * > > + * Redistribution and use in source and binary forms, with or > > + * without modification, are permitted provided that the following > > + * conditions are met: > > + * > > + * - Redistributions of source code must retain the above > > + * copyright notice, this list of conditions and the following > > + * disclaimer. > > + * > > + * - Redistributions in binary form must reproduce the above > > + * copyright notice, this list of conditions and the following > > + * disclaimer in the documentation and/or other materials > > + * provided with the distribution. > > + * > > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > > + * SOFTWARE. > > + * > > + */ > > + > > +#include > > +#include > > + > > +#define _GNU_SOURCE > > +#include > > + > > +#include > > +#include > > + > > +#include "ibdiag_common.h" > > + > > +char *argv0 = ""; > > +char *sa_hca_name = NULL; > > +uint32_t sa_port_num = 0; > > + > > + > > +static int > > +send_144_node_desc_update(void) > > +{ > > + ib_portid_t sm_port; > > + ib_portid_t selfportid; > > + int selfport; > > + ib_rpc_t trap_rpc; > > + ib_mad_notice_attr_t notice; > > + > > + if (ib_resolve_self(&selfportid, &selfport, NULL)) > > + IBERROR("can't resolve self"); > > + > > + if (ib_resolve_smlid(&sm_port, 0)) > > + IBERROR("can't resolve SM destination port"); > > + > > + memset(&trap_rpc, 0, sizeof(trap_rpc)); > > + trap_rpc.mgtclass = IB_SMI_CLASS; > > + trap_rpc.method = IB_MAD_METHOD_TRAP; > > + trap_rpc.trid = mad_trid(); > > + trap_rpc.attr.id = NOTICE; > > + trap_rpc.datasz = IB_SMP_DATA_SIZE; > > + trap_rpc.dataoffs = IB_SMP_DATA_OFFS; > > + > > + memset(¬ice, 0, sizeof(notice)); > > + notice.generic_type = 0x80 | IB_NOTICE_TYPE_INFO; > > + notice.g_or_v.generic.prod_type_lsb = cl_hton16(IB_NODE_TYPE_CA); > > + notice.g_or_v.generic.trap_num = cl_hton16(144); > > + notice.issuer_lid = cl_hton16(selfportid.lid); > > + notice.data_details.ntc_144.lid = cl_hton16(selfportid.lid); > > + notice.data_details.ntc_144.local_changes = TRAP_144_MASK_OTHER_LOCAL_CHANGES; > > + notice.data_details.ntc_144.change_flgs = TRAP_144_MASK_NODE_DESCRIPTION_CHANGE; > > + > > + return (mad_send(&trap_rpc, &sm_port, NULL, ¬ice)); > > +} > > + > > +typedef struct _trap_def { > > + char *trap_name; > > + int (*send_func)(void); > > +} trap_def_t; > > + > > +trap_def_t traps[2] = { > > + { "node_desc_change", send_144_node_desc_update }, > > + { NULL, NULL } > > +}; > > + > > +static void > > +usage(void) > > +{ > > + int i = 0; > > + fprintf(stderr, "Usage: %s [-hV]" > > + "[-C -P ] \n", > > + argv0); > > + fprintf(stderr, " Queries node records by default\n"); > > + fprintf(stderr, " -C specify the SA query HCA\n"); > > + fprintf(stderr, " -P specify the SA query port\n"); > > + fprintf(stderr, " -V print version\n"); > > + fprintf(stderr, " can be one of the following\n"); > > + for (i=0; traps[i].trap_name; i++) { > > + fprintf(stderr, " %s\n", traps[i].trap_name); > > + } > > + fprintf(stderr, " default behavior is to send \"%s\"\n", > > + traps[0].trap_name); > > + > > + exit(-1); > > +} > > + > > +int > > +send_trap(char *trap_name) > > +{ > > + int i = 0; > > + for (i=0; traps[i].trap_name; i++) { > > + if (strcmp(traps[i].trap_name, trap_name) == 0) { > > + return (traps[i].send_func()); > > + } > > + } > > + usage(); > > + exit(1); > > +} > > + > > +int > > +main(int argc, char **argv) > > +{ > > + int mgmt_classes[2] = {IB_SMI_CLASS, IB_SMI_DIRECT_CLASS}; > > + int ch = 0; > > + char *trap_name = NULL; > > + > > + static char const str_opts[] = "hVP:C:"; > > + static const struct option long_opts [] = { > > + {"Version", 0, 0, 'V'}, > > + {"P", 1, 0, 'P'}, > > + {"C", 1, 0, 'C'}, > > + {"help", 0, 0, 'h'}, > > + { } > > + }; > > + > > + argv0 = argv[0]; > > + > > + while ((ch = getopt_long(argc, argv, str_opts, long_opts, NULL)) != -1) { > > + switch (ch) { > > + case 'V': > > + fprintf(stderr, "%s %s\n", argv0, get_build_version()); > > + exit(-1); > > + case 'C': > > + sa_hca_name = optarg; > > + break; > > + case 'P': > > + sa_port_num = strtoul(optarg, NULL, 0); > > + break; > > + case 'h': > > + default: > > + usage(); > > + } > > + } > > + argc -= optind; > > + argv += optind; > > + > > + if (!argv[0]) { > > + trap_name = traps[0].trap_name; > > + } else { > > + trap_name = argv[0]; > > + } > > + > > + madrpc_show_errors(1); > > + madrpc_init(NULL, 0, mgmt_classes, 2); > > + > > + return (send_trap(trap_name)); > > +} From rdreier at cisco.com Fri Mar 21 09:54:43 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 09:54:43 -0700 Subject: [ofa-general] [PATCH 01/10] IB/ipath - fix a byte order compile check warning In-Reply-To: <20080321001740.13431.27631.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Thu, 20 Mar 2008 17:17:40 -0700") References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> <20080321001740.13431.27631.stgit@eng-46.mv.qlogic.com> Message-ID: > This patch fixes a compiler warning when the driver is compiled with: > make M=drivers/infiniband/hw/ipath C=2 CF=-D__CHECK_ENDIAN__ > - dd->ipath_pioavailregs_dma[i] = dd->ipath_pioavailshadow[i] > - = le64_to_cpu(val); > + dd->ipath_pioavailregs_dma[i] = cpu_to_le64(val); > + dd->ipath_pioavailshadow[i] = val; This is more than a warning fix, right? It seems it fixes an incorrect value being assigned to pioavailshadow on big endian systems. - R. From ralph.campbell at qlogic.com Fri Mar 21 10:05:27 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Fri, 21 Mar 2008 10:05:27 -0700 Subject: [ofa-general] [PATCH 01/10] IB/ipath - fix a byte order compile check warning In-Reply-To: References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> <20080321001740.13431.27631.stgit@eng-46.mv.qlogic.com> Message-ID: <1206119127.5109.1079.camel@brick.pathscale.com> On Fri, 2008-03-21 at 09:54 -0700, Roland Dreier wrote: > > This patch fixes a compiler warning when the driver is compiled with: > > make M=drivers/infiniband/hw/ipath C=2 CF=-D__CHECK_ENDIAN__ > > > - dd->ipath_pioavailregs_dma[i] = dd->ipath_pioavailshadow[i] > > - = le64_to_cpu(val); > > + dd->ipath_pioavailregs_dma[i] = cpu_to_le64(val); > > + dd->ipath_pioavailshadow[i] = val; > > This is more than a warning fix, right? It seems it fixes an > incorrect value being assigned to pioavailshadow on big endian > systems. > > - R. Correct. I guess I was fixating on the warning since you pointed it out. From rajouri.jammu at gmail.com Fri Mar 21 11:57:40 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Fri, 21 Mar 2008 11:57:40 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> Message-ID: <3307cdf90803211157l1c5b3febra0d4c89028b4523b@mail.gmail.com> > > > > You should be able to add parameters using modprobe.conf. See the > manpage for modprobe.conf for more details. > > > modprobe ib_mtcha num_mpt=262144 num_mtt=2097152 > > I tried that here and it works, and I get > > ib_mthca 0000:0b:00.0: HCA context memory: reserving 706672 KB > > in the debug output. > I burnt a mem-free f/w on the card and this time I'm getting a different error when trying to set num_mtt to 2097152. I did: modprobe ib_mtcha num_mpt=262144 num_mtt=2097152 logs: kernel: ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) kernel: ib_mthca: Initializing 0000:01:00.0 kernel: PCI: Enabling device 0000:01:00.0 (0000 -> 0002) kernel: ACPI: PCI Interrupt 0000:01:00.0[A] -> GSI 18 (level, low) -> IRQ 58 kernel: ib_mthca 0000:01:00.0: Failed to initialize memory region table, aborting. kernel: ACPI: PCI interrupt for device 0000:01:00.0 disabled kernel: ib_mthca: probe of 0000:01:00.0 failed with error -12 Roland, when your attempt succeeded were you using a mem-free HCA or the 256MB version? devinfo whit the mem-free f/w. fw_ver: 5.3.0 node_guid: 0006:6a00:9800:8403 sys_image_guid: 0006:6a00:9800:8403 vendor_id: 0x02c9 vendor_part_id: 25218 hw_ver: 0x20 board_id: MT_0150000002 > - R. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-340 at a-channel.com Fri Mar 21 12:29:32 2008 From: a-340 at a-channel.com (Saundra Childs) Date: Sat, 22 Mar 2008 03:29:32 +0800 Subject: [ofa-general] the girl is looking for you Message-ID: <01c88bcc$f16b7600$8b821879@a-340> Hello! I am bored today. I am nice girl that would like to chat with you. Email me at Malin at BestGolova.com only, because I am using my friend's email to write this. Don't miss some of my naughty pictures. From semiconductort at shirlymacclaine.com Fri Mar 21 13:33:54 2008 From: semiconductort at shirlymacclaine.com (Rolland Elliott) Date: Fri, 21 Mar 2008 14:33:54 -0600 Subject: [ofa-general] Excel 2007 Message-ID: <01c88b60$969c3500$44e95ec8@semiconductort> Microsoft Office Enterprise 2007 includes: � Access 2007 � Communicator 2007 � Excel 2007 � Groove 2007 � InfoPath 2007 � OneNote 2007 � Outlook 2007 � PowerPoint 2007 � Publisher 2007 � Word 2007 http://jocelyncorrelldk268.blogspot.com System Requirements � Intel� Pentium� or AMD� 500 MHz processor � Microsoft Windows� XP Professional or Home Edition with Service Pack 2, Windows Server� 2003 with SP1, Microsoft Windows Vista. � 256 Mb of RAM � 2GB of available hard-disk space. � 1024x768 or higher resolution monitor � Some features require Microsoft Windows Desktop Search 3.0, Microsoft Windows Media Player 9.0, Microsoft DirectX 9.0b, Microsoft Active Sync 4.1 From rdreier at cisco.com Fri Mar 21 13:57:21 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 13:57:21 -0700 Subject: [ofa-general] [PATCH/RFC 1/2] IB/mthca: Avoid integer overflow when dealing with profile size Message-ID: >From eede145caa8d4ef6c3f03b49a5ea097f222a0a5a Mon Sep 17 00:00:00 2001 From: Roland Dreier Date: Thu, 20 Mar 2008 13:38:47 -0700 mthca_make_profile() returns the size in bytes of the HCA context layout it creates, or a negative value if an error occurs. However, the return value is declared as u64 and the memfree initialization path casts this value to int to test if it is negative. This makes it think incorrectly than an error has occurred if the context size happens to be bigger than 2GB, since this turns into a negative int. Fix this by having mthca_make_profile() return an s64 and testing for an error by checking whether this 64-bit value itself is negative. Signed-off-by: Roland Dreier --- drivers/infiniband/hw/mthca/mthca_main.c | 11 +++++++---- drivers/infiniband/hw/mthca/mthca_profile.c | 4 ++-- drivers/infiniband/hw/mthca/mthca_profile.h | 2 +- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_main.c b/drivers/infiniband/hw/mthca/mthca_main.c index 3889ae8..9ebadd6 100644 --- a/drivers/infiniband/hw/mthca/mthca_main.c +++ b/drivers/infiniband/hw/mthca/mthca_main.c @@ -276,6 +276,7 @@ static int mthca_dev_lim(struct mthca_dev *mdev, struct mthca_dev_lim *dev_lim) static int mthca_init_tavor(struct mthca_dev *mdev) { + s64 size; u8 status; int err; struct mthca_dev_lim dev_lim; @@ -328,9 +329,11 @@ static int mthca_init_tavor(struct mthca_dev *mdev) if (mdev->mthca_flags & MTHCA_FLAG_SRQ) profile.num_srq = dev_lim.max_srqs; - err = mthca_make_profile(mdev, &profile, &dev_lim, &init_hca); - if (err < 0) + size = mthca_make_profile(mdev, &profile, &dev_lim, &init_hca); + if (size < 0) { + err = size; goto err_disable; + } err = mthca_INIT_HCA(mdev, &init_hca, &status); if (err) { @@ -609,7 +612,7 @@ static int mthca_init_arbel(struct mthca_dev *mdev) struct mthca_dev_lim dev_lim; struct mthca_profile profile; struct mthca_init_hca_param init_hca; - u64 icm_size; + s64 icm_size; u8 status; int err; @@ -657,7 +660,7 @@ static int mthca_init_arbel(struct mthca_dev *mdev) profile.num_srq = dev_lim.max_srqs; icm_size = mthca_make_profile(mdev, &profile, &dev_lim, &init_hca); - if ((int) icm_size < 0) { + if (icm_size < 0) { err = icm_size; goto err_stop_fw; } diff --git a/drivers/infiniband/hw/mthca/mthca_profile.c b/drivers/infiniband/hw/mthca/mthca_profile.c index 26bf86d..605a8d5 100644 --- a/drivers/infiniband/hw/mthca/mthca_profile.c +++ b/drivers/infiniband/hw/mthca/mthca_profile.c @@ -63,7 +63,7 @@ enum { MTHCA_NUM_PDS = 1 << 15 }; -u64 mthca_make_profile(struct mthca_dev *dev, +s64 mthca_make_profile(struct mthca_dev *dev, struct mthca_profile *request, struct mthca_dev_lim *dev_lim, struct mthca_init_hca_param *init_hca) @@ -77,7 +77,7 @@ u64 mthca_make_profile(struct mthca_dev *dev, }; u64 mem_base, mem_avail; - u64 total_size = 0; + s64 total_size = 0; struct mthca_resource *profile; struct mthca_resource tmp; int i, j; diff --git a/drivers/infiniband/hw/mthca/mthca_profile.h b/drivers/infiniband/hw/mthca/mthca_profile.h index 9464180..e76cb62 100644 --- a/drivers/infiniband/hw/mthca/mthca_profile.h +++ b/drivers/infiniband/hw/mthca/mthca_profile.h @@ -53,7 +53,7 @@ struct mthca_profile { int fmr_reserved_mtts; }; -u64 mthca_make_profile(struct mthca_dev *mdev, +s64 mthca_make_profile(struct mthca_dev *mdev, struct mthca_profile *request, struct mthca_dev_lim *dev_lim, struct mthca_init_hca_param *init_hca); -- 1.5.4.3 From rdreier at cisco.com Fri Mar 21 13:58:28 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 13:58:28 -0700 Subject: [ofa-general] [PATCH/RFC 2/2] IB/mthca: Avoid integer overflow when allocating huge ICM table In-Reply-To: (Roland Dreier's message of "Fri, 21 Mar 2008 13:57:21 -0700") References: Message-ID: In mthca_alloc_icm_table(), the number of entries to allocate for the table->icm array is computed by calculating obj_size * nobj and then dividing by MTHCA_TABLE_CHUNK_SIZE. If nobj is really large, then obj_size * nobj may overflow and the division may get the wrong value (even a negative value). Fix this by calculating the number of objects per chunk and then dividing nobj by this value instead. This patch allows crazy configurations such as loading ib_mthca with the module parameter num_mtt=33554432 to work properly. Signed-off-by: Roland Dreier --- drivers/infiniband/hw/mthca/mthca_memfree.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c b/drivers/infiniband/hw/mthca/mthca_memfree.c index d7d502d..b224079 100644 --- a/drivers/infiniband/hw/mthca/mthca_memfree.c +++ b/drivers/infiniband/hw/mthca/mthca_memfree.c @@ -359,12 +359,14 @@ struct mthca_icm_table *mthca_alloc_icm_table(struct mthca_dev *dev, int use_lowmem, int use_coherent) { struct mthca_icm_table *table; + int obj_per_chunk; int num_icm; unsigned chunk_size; int i; u8 status; - num_icm = (obj_size * nobj + MTHCA_TABLE_CHUNK_SIZE - 1) / MTHCA_TABLE_CHUNK_SIZE; + obj_per_chunk = MTHCA_TABLE_CHUNK_SIZE / obj_size; + num_icm = DIV_ROUND_UP(nobj, obj_per_chunk); table = kmalloc(sizeof *table + num_icm * sizeof *table->icm, GFP_KERNEL); if (!table) -- 1.5.4.3 From rdreier at cisco.com Fri Mar 21 14:02:21 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 14:02:21 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus Chien Tung (1): RDMA/nes: Fix MSS calculation on RDMA path drivers/infiniband/hw/nes/nes_cm.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c index 39adb26..0bef878 100644 --- a/drivers/infiniband/hw/nes/nes_cm.c +++ b/drivers/infiniband/hw/nes/nes_cm.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -1072,7 +1073,7 @@ static struct nes_cm_node *make_cm_node(struct nes_cm_core *cm_core, ts = current_kernel_time(); cm_node->tcp_cntxt.loc_seq_num = htonl(ts.tv_nsec); cm_node->tcp_cntxt.mss = nesvnic->max_frame_size - sizeof(struct iphdr) - - sizeof(struct tcphdr) - ETH_HLEN; + sizeof(struct tcphdr) - ETH_HLEN - VLAN_HLEN; cm_node->tcp_cntxt.rcv_nxt = 0; /* get a unique session ID , add thread_id to an upcounter to handle race */ atomic_inc(&cm_core->node_cnt); From rdreier at cisco.com Fri Mar 21 14:02:46 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 14:02:46 -0700 Subject: [ofa-general] Re: [PATCH] RDMA/nes: Fix MSS calculation on RDMA path In-Reply-To: <200803210055.m2L0tUjU031839@velma.neteffect.com> (gstreiff@neteffect.com's message of "Thu, 20 Mar 2008 19:55:30 -0500") References: <200803210055.m2L0tUjU031839@velma.neteffect.com> Message-ID: applied, thanks. From rdreier at cisco.com Fri Mar 21 14:08:56 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 14:08:56 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803211157l1c5b3febra0d4c89028b4523b@mail.gmail.com> (Rajouri Jammu's message of "Fri, 21 Mar 2008 11:57:40 -0700") References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> <3307cdf90803211157l1c5b3febra0d4c89028b4523b@mail.gmail.com> Message-ID: > I burnt a mem-free f/w on the card and this time I'm getting a different > error when trying to set num_mtt to 2097152. > I did: > modprobe ib_mtcha num_mpt=262144 num_mtt=2097152 > kernel: ib_mthca 0000:01:00.0: Failed to initialize memory region table, aborting. You may not have enough contiguous memory for the buddy allocator bitmaps for the MTT. What does /proc/buddyinfo show when this happens? > Roland, when your attempt succeeded were you using a mem-free HCA or the > 256MB version? mem-free. With the fixes I just posted, I am able to load the driver with num_mtt=33554432 and have it work in fact. It's not clear to me why you need such crazy values. Having 2M MTT segments means you can map 64GB of memory with 4KB pages... do you really need that much memory pinned for IB operations? - R. From rdreier at cisco.com Fri Mar 21 14:20:16 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 14:20:16 -0700 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: (Hoang-Nam Nguyen's message of "Thu, 20 Mar 2008 20:59:59 +0100") References: Message-ID: > Our ehca2 supports sort of low-latency QP for UD and RC. It would be great > if we can e.g. enhance above enum like this > QP_CREATE_LL = 1 << 1 OK, that actually convinces me that this is useful infrastructure. However, please use 1 << 2 for your flag since 0 is LSO and 1 is used by XRC already. - R. From rdreier at cisco.com Fri Mar 21 14:20:16 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 14:20:16 -0700 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: (Hoang-Nam Nguyen's message of "Thu, 20 Mar 2008 20:59:59 +0100") References: Message-ID: > Our ehca2 supports sort of low-latency QP for UD and RC. It would be great > if we can e.g. enhance above enum like this > QP_CREATE_LL = 1 << 1 OK, that actually convinces me that this is useful infrastructure. However, please use 1 << 2 for your flag since 0 is LSO and 1 is used by XRC already. - R. From shampartzoumian at qcindustries.com Fri Mar 21 14:21:41 2008 From: shampartzoumian at qcindustries.com (EuroSoftware) Date: Fri, 21 Mar 2008 22:21:41 +0100 Subject: [ofa-general] Software at the Lowest Prices Ever Message-ID: <674311948.11352271117991@qcindustries.com> Industry standard software at nominal feeOur main goal is to render low cost PC and Macintosh lawful soft and computer solutions for any budget. Whether you're a corporate buyer, a proprietor of small enterprise, or shopping for your home personal computer, we believe that we can assist you.VIEW WHAT WE GOT TO PROPOSE http://geocities.com/chance.holden/Most popular products:* Microsoft Office Enterprise 2007: Retail price today - $599.95; Our just - $79.95 * Adobe Creative Suite 3 Web Premium: Retail price for this time - $1599.95; Our only today - $219.95 * Adobe InCopy CS2: Retail price this day - $249.95; Our only for today - $59.95 * AutoCAD Architecture 2008: Retail price for now - $4999.95; Our only for today - $149.95 * Adobe Fireworks CS3: Retail price this day - $299.00; Our now just - $59.95 * Office System Professional 2003 (5 Cds): Retail price now - $469.95; Our now just - $59.95 * Adobe Acrobat 7 Professional: Retail price this day - $449.95; Our just - $59.95 CHECK WHAT WE HAVE TO PROPOSE http://geocities.com/chance.holden/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Fri Mar 21 14:34:16 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 14:34:16 -0700 Subject: [ofa-general] Re: [PATCH 10/10] IB/ipath - Fix sanity checks on QP number of WRs and SGEs In-Reply-To: <20080321001826.13431.64812.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Thu, 20 Mar 2008 17:18:26 -0700") References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> <20080321001826.13431.64812.stgit@eng-46.mv.qlogic.com> Message-ID: thanks, applied all 10 From info at paat.co.uk Fri Mar 21 15:24:31 2008 From: info at paat.co.uk (University Degree) Date: Sat, 22 Mar 2008 00:24:31 +0200 Subject: [ofa-general] University Degree in Short Terms Message-ID: <923691472.62142585852599@paat.co.uk> 1-410-230-1849 University Degree OBTAIN A PROSPEROUS FUTURE, MONEY-EARNING POWER, AND THE PRESTIGE THAT COMES WITH HAVING THE CAREER POSITION YOU’VE ALWAYS DREAMED OF. DIPLOMA FROM PRESTIGIOUS NON-ACCREDITED UNVERSITIES BASED ON YOUR PRESENT KNOWLEDGE AND PROFESSIONAL EXPERIENCE. If you qualify, no required tests, classes, books or examinations. Confidentiality Assured 1-410-230-1849 24 hours a day, 7 days a week including Sundays and Holidays Sat, 22 Mar 2008 00:24:31 +0200 -------------- next part -------------- An HTML attachment was scrubbed... URL: From akstcaltrolavoromnsdgs at altrolavoro.it Fri Mar 21 15:48:31 2008 From: akstcaltrolavoromnsdgs at altrolavoro.it (Armando Vinson) Date: Fri, 21 Mar 2008 23:48:31 +0100 Subject: [ofa-general] i love you Message-ID: <01c88bae$11ac1600$861db455@akstcaltrolavoromnsdgs> Instant worldwide shipping, friendly support guaranteed. http://nicholeconantsg.blogspot.com From weiny2 at llnl.gov Fri Mar 21 15:51:12 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 21 Mar 2008 15:51:12 -0700 Subject: [ofa-general] [PATCH 0/5] trap 144, node descriptor change handling. Message-ID: <20080321155112.289cf10f.weiny2@llnl.gov> I have altered these patches based on input from Hal. There are now 5 of them. 1) Enhance set_nodedesc.sh 2) Add optional ibsendtrap test tool and create --enable-test-utils configure option 3) Add mcm_rereg_test to optional test tool config option (This also updates the tool to use the new ibmad_gid_t since it would not compile otherwise.) 4) Update OpenSM to respond to node description change flag 5) Create osm_node_desc.h and move extern declarations into this common header. This includes Hal's minor stylistic fixes as well as makes ibsendtrap an optional testing util, rather than a default diag. Thanks, Ira From weiny2 at llnl.gov Fri Mar 21 15:51:15 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 21 Mar 2008 15:51:15 -0700 Subject: [ofa-general] [PATCH 1/5] infiniband-diags/scripts/set_nodedesc.sh: enhance to be able to set names other than hostname and to provide feedback on the names assigned Message-ID: <20080321155115.15e4d480.weiny2@llnl.gov> >From 9553461b42d8dde7418b9eb3c07a8e78b58a32f1 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Mon, 17 Mar 2008 14:46:45 -0700 Subject: [PATCH] infiniband-diags/scripts/set_nodedesc.sh: enhance to be able to set names other than hostname and to provide feedback on the names assigned. Signed-off-by: Ira K. Weiny --- infiniband-diags/scripts/set_nodedesc.sh | 47 ++++++++++++++++++++++++++++-- 1 files changed, 44 insertions(+), 3 deletions(-) diff --git a/infiniband-diags/scripts/set_nodedesc.sh b/infiniband-diags/scripts/set_nodedesc.sh index 7b817cc..55a2fd1 100755 --- a/infiniband-diags/scripts/set_nodedesc.sh +++ b/infiniband-diags/scripts/set_nodedesc.sh @@ -1,17 +1,58 @@ #!/bin/sh -# set the node_desc field of any hca found to the defined hostname - +if [ -f /etc/sysconfig/network ]; then . /etc/sysconfig/network +fi ib_sysfs="/sys/class/infiniband" +newname="$HOSTNAME" + + +function usage +{ + echo "Usage: `basename $0` [-hv] []" + echo " set the node_desc field of all hca's found in \"$ib_sysfs\"" + echo " -h this help" + echo " -v view all node descriptors" + echo " [] set name to name specified." + echo " Default is to use the hostname: \"$HOSTNAME\"" + exit 2 +} + +function viewall +{ + for hca in `ls $ib_sysfs`; do + if [ -f $ib_sysfs/$hca/node_desc ]; then + echo -n "$hca: " + cat $ib_sysfs/$hca/node_desc + else + logger -s "Failed to set node_desc for : $hca" + fi + done + exit 0 +} + +while getopts "hv" flag +do + case $flag in + "h") usage;; + "v") viewall;; + esac +done + +shift $(($OPTIND - 1)) + +if [ "$1" != "" ]; then + newname="$1" +fi for hca in `ls $ib_sysfs`; do if [ -f $ib_sysfs/$hca/node_desc ]; then - echo -n "$HOSTNAME" >> $ib_sysfs/$hca/node_desc + echo -n "$newname" >> $ib_sysfs/$hca/node_desc else logger -s "Failed to set node_desc for : $hca" fi done exit 0 + -- 1.5.1 From weiny2 at llnl.gov Fri Mar 21 15:51:17 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 21 Mar 2008 15:51:17 -0700 Subject: [ofa-general] [PATCH 2/5] Add an optional test utility 'ibsendtrap' Message-ID: <20080321155117.2b69452e.weiny2@llnl.gov> >From 9294bf0041e88a077918619d5a93be318075cd74 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Fri, 21 Mar 2008 14:26:05 -0700 Subject: [PATCH] Add an optional test utility 'ibsendtrap' use --enable-test-utils to build this tool Signed-off-by: Ira K. Weiny --- infiniband-diags/Makefile.am | 8 ++ infiniband-diags/configure.in | 12 +++ infiniband-diags/src/ibsendtrap.c | 179 +++++++++++++++++++++++++++++++++++++ 3 files changed, 199 insertions(+), 0 deletions(-) create mode 100644 infiniband-diags/src/ibsendtrap.c diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index 07a6233..d29d417 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -12,6 +12,10 @@ sbin_PROGRAMS = src/ibaddr src/ibnetdiscover src/ibping src/ibportstate \ src/perfquery src/sminfo src/smpdump src/smpquery \ src/saquery src/vendstat +if ENABLE_TEST_UTILS +sbin_PROGRAMS += src/ibsendtrap +endif + sbin_SCRIPTS = scripts/ibcheckerrs scripts/ibchecknet scripts/ibchecknode \ scripts/ibcheckport scripts/ibhosts scripts/ibstatus \ scripts/ibswitches scripts/ibnodes scripts/ibrouters \ @@ -71,6 +75,10 @@ src_saquery_SOURCES = src/saquery.c src/ibdiag_common.c src_saquery_CFLAGS = -Wall -DOSM_VENDOR_INTF_OPENIB -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) src_saquery_LDFLAGS = -Wl,--rpath -Wl,$(libdir) +src_ibsendtrap_SOURCES = src/ibsendtrap.c src/ibdiag_common.c +src_ibsendtrap_CFLAGS = -Wall $(DBGFLAGS) +src_ibsendtrap_LDFLAGS = -Wl,--rpath -Wl,$(libdir) + src_vendstat_SOURCES = src/vendstat.c src/ibdiag_common.c src_vendstat_CFLAGS = -Wall $(DBGFLAGS) diff --git a/infiniband-diags/configure.in b/infiniband-diags/configure.in index 7468935..33eece1 100644 --- a/infiniband-diags/configure.in +++ b/infiniband-diags/configure.in @@ -72,6 +72,18 @@ AC_CHECK_FUNCS([strchr strrchr strtol strtoul memset]) dnl Checks for typedefs, structures, and compiler characteristics. AC_C_CONST +dnl Check if we should include test utilities +AC_MSG_CHECKING(for --enable-test-utils) +AC_ARG_ENABLE(test-utils, +[ --enable-test-utils build additional test utilities], +[case "${enableval}" in + yes) tutils=yes ;; + no) tutils=no ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-test-utils) ;; +esac],[tutils=no]) +AM_CONDITIONAL(ENABLE_TEST_UTILS, test x$tutils = xyes) +AC_MSG_RESULT(${tutils=no}) + dnl Check for perl and perl install location AC_MSG_CHECKING(for --with-perl-path ) AC_ARG_WITH(perl-path, diff --git a/infiniband-diags/src/ibsendtrap.c b/infiniband-diags/src/ibsendtrap.c new file mode 100644 index 0000000..ae5069b --- /dev/null +++ b/infiniband-diags/src/ibsendtrap.c @@ -0,0 +1,179 @@ +/* + * Copyright (c) 2008 Lawrence Livermore National Security + * + * Produced at Lawrence Livermore National Laboratory. + * Written by Ira Weiny . + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +#include +#include + +#define _GNU_SOURCE +#include + +#include +#include + +#include "ibdiag_common.h" + +char *argv0 = ""; +char *sa_hca_name = NULL; +uint32_t sa_port_num = 0; + + +static int +send_144_node_desc_update(void) +{ + ib_portid_t sm_port; + ib_portid_t selfportid; + int selfport; + ib_rpc_t trap_rpc; + ib_mad_notice_attr_t notice; + + if (ib_resolve_self(&selfportid, &selfport, NULL)) + IBERROR("can't resolve self"); + + if (ib_resolve_smlid(&sm_port, 0)) + IBERROR("can't resolve SM destination port"); + + memset(&trap_rpc, 0, sizeof(trap_rpc)); + trap_rpc.mgtclass = IB_SMI_CLASS; + trap_rpc.method = IB_MAD_METHOD_TRAP; + trap_rpc.trid = mad_trid(); + trap_rpc.attr.id = NOTICE; + trap_rpc.datasz = IB_SMP_DATA_SIZE; + trap_rpc.dataoffs = IB_SMP_DATA_OFFS; + + memset(¬ice, 0, sizeof(notice)); + notice.generic_type = 0x80 | IB_NOTICE_TYPE_INFO; + notice.g_or_v.generic.prod_type_lsb = cl_hton16(IB_NODE_TYPE_CA); + notice.g_or_v.generic.trap_num = cl_hton16(144); + notice.issuer_lid = cl_hton16(selfportid.lid); + notice.data_details.ntc_144.lid = cl_hton16(selfportid.lid); + notice.data_details.ntc_144.local_changes = TRAP_144_MASK_OTHER_LOCAL_CHANGES; + notice.data_details.ntc_144.change_flgs = TRAP_144_MASK_NODE_DESCRIPTION_CHANGE; + + return (mad_send(&trap_rpc, &sm_port, NULL, ¬ice)); +} + +typedef struct _trap_def { + char *trap_name; + int (*send_func)(void); +} trap_def_t; + +trap_def_t traps[2] = { + { "node_desc_change", send_144_node_desc_update }, + { NULL, NULL } +}; + +static void +usage(void) +{ + int i = 0; + fprintf(stderr, "Usage: %s [-hV]" + "[-C -P ] \n", + argv0); + fprintf(stderr, " Queries node records by default\n"); + fprintf(stderr, " -C specify the SA query HCA\n"); + fprintf(stderr, " -P specify the SA query port\n"); + fprintf(stderr, " -V print version\n"); + fprintf(stderr, " can be one of the following\n"); + for (i=0; traps[i].trap_name; i++) { + fprintf(stderr, " %s\n", traps[i].trap_name); + } + fprintf(stderr, " default behavior is to send \"%s\"\n", + traps[0].trap_name); + + exit(-1); +} + +int +send_trap(char *trap_name) +{ + int i = 0; + for (i=0; traps[i].trap_name; i++) { + if (strcmp(traps[i].trap_name, trap_name) == 0) { + return (traps[i].send_func()); + } + } + usage(); + exit(1); +} + +int +main(int argc, char **argv) +{ + int mgmt_classes[2] = {IB_SMI_CLASS, IB_SMI_DIRECT_CLASS}; + int ch = 0; + char *trap_name = NULL; + + static char const str_opts[] = "hVP:C:"; + static const struct option long_opts [] = { + {"Version", 0, 0, 'V'}, + {"P", 1, 0, 'P'}, + {"C", 1, 0, 'C'}, + {"help", 0, 0, 'h'}, + { } + }; + + argv0 = argv[0]; + + while ((ch = getopt_long(argc, argv, str_opts, long_opts, NULL)) != -1) { + switch (ch) { + case 'V': + fprintf(stderr, "%s %s\n", argv0, get_build_version()); + exit(-1); + case 'C': + sa_hca_name = optarg; + break; + case 'P': + sa_port_num = strtoul(optarg, NULL, 0); + break; + case 'h': + default: + usage(); + } + } + argc -= optind; + argv += optind; + + if (!argv[0]) { + trap_name = traps[0].trap_name; + } else { + trap_name = argv[0]; + } + + madrpc_show_errors(1); + madrpc_init(NULL, 0, mgmt_classes, 2); + + return (send_trap(trap_name)); +} -- 1.5.1 From weiny2 at llnl.gov Fri Mar 21 15:51:19 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 21 Mar 2008 15:51:19 -0700 Subject: [ofa-general] [PATCH 3/5] Add mcm_rereg_test to test-utils option. Message-ID: <20080321155119.651d99f7.weiny2@llnl.gov> >From 60a6563c1f726137a188a024481f25cbab9b140b Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Fri, 21 Mar 2008 14:44:31 -0700 Subject: [PATCH] Add mcm_rereg_test to test-utils option. Fix mcm_rereg_test.c to use ibmad_gid_t Signed-off-by: Ira K. Weiny --- infiniband-diags/Makefile.am | 7 +++---- infiniband-diags/src/mcm_rereg_test.c | 14 +++++++------- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index d29d417..e502a06 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -13,7 +13,7 @@ sbin_PROGRAMS = src/ibaddr src/ibnetdiscover src/ibping src/ibportstate \ src/saquery src/vendstat if ENABLE_TEST_UTILS -sbin_PROGRAMS += src/ibsendtrap +sbin_PROGRAMS += src/ibsendtrap src/mcm_rereg_test endif sbin_SCRIPTS = scripts/ibcheckerrs scripts/ibchecknet scripts/ibchecknode \ @@ -82,9 +82,8 @@ src_ibsendtrap_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_vendstat_SOURCES = src/vendstat.c src/ibdiag_common.c src_vendstat_CFLAGS = -Wall $(DBGFLAGS) -#src_mcm_rereg_test_SOURCES = src/mcm_rereg_test.c -#src_mcm_rereg_test_CFLAGS = -Wall $(DBGFLAGS) -#sbin_PROGRAMS += src/mcm_rereg_test +src_mcm_rereg_test_SOURCES = src/mcm_rereg_test.c +src_mcm_rereg_test_CFLAGS = -Wall $(DBGFLAGS) man_MANS = man/ibaddr.8 man/ibcheckerrors.8 man/ibcheckerrs.8 \ man/ibchecknet.8 man/ibchecknode.8 man/ibcheckport.8 \ diff --git a/infiniband-diags/src/mcm_rereg_test.c b/infiniband-diags/src/mcm_rereg_test.c index 57c8db6..4f3ccc1 100644 --- a/infiniband-diags/src/mcm_rereg_test.c +++ b/infiniband-diags/src/mcm_rereg_test.c @@ -69,12 +69,12 @@ #define IB_MCR_COMPMASK_JOIN_STATE (1ULL<<16) #define IB_MCR_COMPMASK_PROXY (1ULL<<17) -static ib_gid_t mgid_ipoib = { +static ibmad_gid_t mgid_ipoib = { 0xff, 0x12, 0x40, 0x1b, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff }; -uint64_t build_mcm_rec(uint8_t *data, ib_gid_t mgid, ib_gid_t port_gid) +uint64_t build_mcm_rec(uint8_t *data, ibmad_gid_t mgid, ibmad_gid_t port_gid) { memset(data, 0, IB_SA_DATA_SIZE); mad_set_array(data, 0, IB_SA_MCM_MGID_F, mgid); @@ -104,7 +104,7 @@ static void build_mcm_rec_umad(void *umad, ib_portid_t *dport, int method, } static int rereg_send(int port, int agent, ib_portid_t *dport, - uint8_t *umad, int len, int method, ib_gid_t port_gid) + uint8_t *umad, int len, int method, ibmad_gid_t port_gid) { uint8_t data[IB_SA_DATA_SIZE]; uint64_t comp_mask; @@ -123,7 +123,7 @@ static int rereg_send(int port, int agent, ib_portid_t *dport, } static int rereg_port_gid(int port, int agent, ib_portid_t *dport, - uint8_t *umad, int len, ib_gid_t port_gid) + uint8_t *umad, int len, ibmad_gid_t port_gid) { uint8_t data[IB_SA_DATA_SIZE]; uint64_t comp_mask; @@ -152,7 +152,7 @@ static int rereg_port_gid(int port, int agent, ib_portid_t *dport, } struct guid_trid { - ib_gid_t gid; + ibmad_gid_t gid; uint64_t guid; uint64_t trid; }; @@ -194,7 +194,7 @@ static int rereg_send_all(int port, int agent, ib_portid_t *dport, static int rereg_mcm_rec_send(int port, int agent, ib_portid_t *dport, int cnt) { ib_portid_t portid; - ib_gid_t port_gid; + ibmad_gid_t port_gid; uint8_t *umad; int len, ret = 0; @@ -384,7 +384,7 @@ static int rereg_and_test_port(char *guid_file, int port, int agent, ib_portid_t { char line[256]; FILE *f; - ib_gid_t port_gid; + ibmad_gid_t port_gid; uint64_t prefix = htonll(0xfe80000000000000llu); uint64_t guid = htonll(0x0002c90200223825llu); struct guid_trid *list; -- 1.5.1 From weiny2 at llnl.gov Fri Mar 21 15:51:22 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 21 Mar 2008 15:51:22 -0700 Subject: [ofa-general] [PATCH 4/5] opensm/opensm/osm_trap_rcv.c: respond to new trap 144 node description update flag Message-ID: <20080321155122.53262251.weiny2@llnl.gov> >From 72f64c6eb0b82ea7dc8980733eb41660e2f40b1a Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 20 Mar 2008 17:59:01 -0700 Subject: [PATCH] opensm/opensm/osm_trap_rcv.c: respond to new trap 144 node description update flag Signed-off-by: Ira K. Weiny --- opensm/opensm/osm_node_info_rcv.c | 40 +++++++++++++++++++++++++----------- opensm/opensm/osm_trap_rcv.c | 22 ++++++++++++++++++++ 2 files changed, 50 insertions(+), 12 deletions(-) diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 9b2c74c..6818e05 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -306,17 +306,41 @@ __osm_ni_rcv_process_new_node(IN osm_sm_t * sm, /********************************************************************** The plock must be held before calling this function. **********************************************************************/ +void +osm_req_get_node_desc(IN osm_sm_t * sm, + osm_physp_t *p_physp) +{ + ib_api_status_t status = IB_SUCCESS; + osm_madw_context_t context; + + OSM_LOG_ENTER(sm->p_log); + + context.nd_context.node_guid = + osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); + + status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), + IB_MAD_ATTR_NODE_DESC, + 0, CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) + OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " + "Failure initiating NodeDescription request (%s)\n", + ib_get_err_str(status)); + + OSM_LOG_EXIT(sm->p_log); +} + +/********************************************************************** + The plock must be held before calling this function. +**********************************************************************/ static void __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, IN osm_node_t * const p_node, IN const osm_madw_t * const p_madw) { - ib_api_status_t status = IB_SUCCESS; - osm_madw_context_t context; - osm_physp_t *p_physp; ib_node_info_t *p_ni; ib_smp_t *p_smp; uint8_t port_num; + osm_physp_t *p_physp = NULL; OSM_LOG_ENTER(sm->p_log); @@ -334,15 +358,7 @@ __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, */ p_physp = osm_node_get_physp_ptr(p_node, port_num); - context.nd_context.node_guid = osm_node_get_node_guid(p_node); - - status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), - IB_MAD_ATTR_NODE_DESC, - 0, CL_DISP_MSGID_NONE, &context); - if (status != IB_SUCCESS) - OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " - "Failure initiating NodeDescription request (%s)\n", - ib_get_err_str(status)); + osm_req_get_node_desc(sm, p_physp); OSM_LOG_EXIT(sm->p_log); } diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c index 5cf5a21..bc8311d 100644 --- a/opensm/opensm/osm_trap_rcv.c +++ b/opensm/opensm/osm_trap_rcv.c @@ -61,6 +61,8 @@ #include #include +extern void osm_req_get_node_desc(IN osm_sm_t * sm, osm_physp_t *p_physp); + /********************************************************************** * * TRAP HANDLING: @@ -577,6 +579,26 @@ __osm_trap_rcv_process_request(IN osm_sm_t * sm, } } + /* Check for node descriptor update. IB Spec v1.2.1 pg 823*/ + if ((p_ntci->data_details.ntc_144.local_changes & TRAP_144_MASK_OTHER_LOCAL_CHANGES) && + (p_ntci->data_details.ntc_144.change_flgs & TRAP_144_MASK_NODE_DESCRIPTION_CHANGE) + ) { + + osm_node_t *p_node = NULL; + + OSM_LOG(sm->p_log, OSM_LOG_INFO, "Trap 144 Node description update\n"); + + if (p_physp) { + CL_PLOCK_ACQUIRE(sm->p_lock); + osm_req_get_node_desc(sm, p_physp); + CL_PLOCK_RELEASE(sm->p_lock); + } else { + OSM_LOG(sm->p_log, OSM_LOG_ERROR, + "ERR 3812: No physical port found for " + "trap 144: \"node description update\"\n"); + } + } + /* do a sweep if we received a trap */ if (sm->p_subn->opt.sweep_on_trap) { /* if this is trap number 128 or run_heavy_sweep is TRUE - update the -- 1.5.1 From weiny2 at llnl.gov Fri Mar 21 15:51:25 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 21 Mar 2008 15:51:25 -0700 Subject: [ofa-general] [PATCH 5/5] Move extern declarations for node description processing to a new header file. Message-ID: <20080321155125.170f629c.weiny2@llnl.gov> >From 93479e6cac589167f79f09d42900994e947c90f4 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Fri, 21 Mar 2008 15:23:16 -0700 Subject: [PATCH] Move extern declarations for node description processing to a new header file. Signed-off-by: Ira K. Weiny --- opensm/include/opensm/osm_node_desc.h | 54 +++++++++++++++++++++++++++++++++ opensm/opensm/osm_node_desc_rcv.c | 27 ++++++++++++++++ opensm/opensm/osm_node_info_rcv.c | 26 ---------------- opensm/opensm/osm_sm.c | 2 +- opensm/opensm/osm_trap_rcv.c | 3 +- 5 files changed, 83 insertions(+), 29 deletions(-) create mode 100644 opensm/include/opensm/osm_node_desc.h diff --git a/opensm/include/opensm/osm_node_desc.h b/opensm/include/opensm/osm_node_desc.h new file mode 100644 index 0000000..d86f6ba --- /dev/null +++ b/opensm/include/opensm/osm_node_desc.h @@ -0,0 +1,54 @@ +/* + * Copyright (c) 2008 Lawrence Livermore National Security + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +#ifndef _OSM_NODE_DESC_H_ +#define _OSM_NODE_DESC_H_ + +#include +#include + +#ifdef __cplusplus +# define BEGIN_C_DECLS extern "C" { +# define END_C_DECLS } +#else /* !__cplusplus */ +# define BEGIN_C_DECLS +# define END_C_DECLS +#endif /* __cplusplus */ + +BEGIN_C_DECLS + +void osm_nd_rcv_process(void *context, void *data); +void osm_req_get_node_desc(osm_sm_t * sm, osm_physp_t *p_physp); + +END_C_DECLS +#endif /* _OSM_NODE_DESC_H_ */ diff --git a/opensm/opensm/osm_node_desc_rcv.c b/opensm/opensm/osm_node_desc_rcv.c index 4a22aab..a7266d5 100644 --- a/opensm/opensm/osm_node_desc_rcv.c +++ b/opensm/opensm/osm_node_desc_rcv.c @@ -134,3 +134,30 @@ void osm_nd_rcv_process(IN void *context, IN void *data) CL_PLOCK_RELEASE(sm->p_lock); OSM_LOG_EXIT(sm->p_log); } + +/********************************************************************** + The plock must be held before calling this function. +**********************************************************************/ +void +osm_req_get_node_desc(IN osm_sm_t * sm, + osm_physp_t *p_physp) +{ + ib_api_status_t status = IB_SUCCESS; + osm_madw_context_t context; + + OSM_LOG_ENTER(sm->p_log); + + context.nd_context.node_guid = + osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); + + status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), + IB_MAD_ATTR_NODE_DESC, + 0, CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) + OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " + "Failure initiating NodeDescription request (%s)\n", + ib_get_err_str(status)); + + OSM_LOG_EXIT(sm->p_log); +} + diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 6818e05..9a3eb7b 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -306,32 +306,6 @@ __osm_ni_rcv_process_new_node(IN osm_sm_t * sm, /********************************************************************** The plock must be held before calling this function. **********************************************************************/ -void -osm_req_get_node_desc(IN osm_sm_t * sm, - osm_physp_t *p_physp) -{ - ib_api_status_t status = IB_SUCCESS; - osm_madw_context_t context; - - OSM_LOG_ENTER(sm->p_log); - - context.nd_context.node_guid = - osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); - - status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), - IB_MAD_ATTR_NODE_DESC, - 0, CL_DISP_MSGID_NONE, &context); - if (status != IB_SUCCESS) - OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " - "Failure initiating NodeDescription request (%s)\n", - ib_get_err_str(status)); - - OSM_LOG_EXIT(sm->p_log); -} - -/********************************************************************** - The plock must be held before calling this function. -**********************************************************************/ static void __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, IN osm_node_t * const p_node, diff --git a/opensm/opensm/osm_sm.c b/opensm/opensm/osm_sm.c index 32525ba..69aafb4 100644 --- a/opensm/opensm/osm_sm.c +++ b/opensm/opensm/osm_sm.c @@ -64,12 +64,12 @@ #include #include #include +#include #define OSM_SM_INITIAL_TID_VALUE 0x1233 extern void osm_lft_rcv_process(IN void *context, IN void *data); extern void osm_mft_rcv_process(IN void *context, IN void *data); -extern void osm_nd_rcv_process(IN void *context, IN void *data); extern void osm_ni_rcv_process(IN void *context, IN void *data); extern void osm_pkey_rcv_process(IN void *context, IN void *data); extern void osm_pi_rcv_process(IN void *context, IN void *data); diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c index bc8311d..6fcf406 100644 --- a/opensm/opensm/osm_trap_rcv.c +++ b/opensm/opensm/osm_trap_rcv.c @@ -60,8 +60,7 @@ #include #include #include - -extern void osm_req_get_node_desc(IN osm_sm_t * sm, osm_physp_t *p_physp); +#include /********************************************************************** * -- 1.5.1 From dillowda at ornl.gov Fri Mar 21 16:03:40 2008 From: dillowda at ornl.gov (David Dillow) Date: Fri, 21 Mar 2008 19:03:40 -0400 Subject: [ofa-general] Re: [PATCH] IB/srp: enforce protocol limit on srp_sg_tablesize In-Reply-To: References: <1205777880.10010.26.camel@lap75545.ornl.gov> <1205782965.10010.30.camel@lap75545.ornl.gov> Message-ID: <1206140620.4685.4.camel@obelisk.thedillows.org> On Wed, 2008-03-19 at 13:54 -0700, Roland Dreier wrote: > thanks, applied I see you queued this for 2.6.26 -- I'd have thought that this would go to 2.6.25 as it is a bug fix. What do you think? -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office From a-anthw at acci.asn.au Fri Mar 21 16:34:59 2008 From: a-anthw at acci.asn.au (Dirk Mcintyre) Date: Fri, 21 Mar 2008 19:34:59 -0400 Subject: [ofa-general] looking for someone? Message-ID: <01c88b8a$a6307380$824e53c8@a-anthw> Hello! I am tired this afternoon. I am nice girl that would like to chat with you. Email me at Sara at BestGolova.com only, because I am using my friend's email to write this. Mind me sending some of my pictures to you? From fondazon at web.de Fri Mar 21 17:06:44 2008 From: fondazon at web.de (Fondazione Vittorio) Date: Sat, 22 Mar 2008 01:06:44 +0100 Subject: [ofa-general] Fondazione Di Vittorio, Message-ID: <1143351210@web.de> Fondazione Di Vittorio, ITALY. http://www.fondazionedivittorio.it [http://www.fondazionedivittorio.it/] Attention:Recipient, The Fondazione Di Vittorio, would like to notify you that you have been chosen by the board of trustees as one of the final recipients of a Grant/Donation cash aid of USD$1,000,000.00 (One Million United States Dollars) for your own personal education, and business development. The Fondazione Di Vittorio, established 1977 by the Multi-Million groups and now supported by United Nations Organization (UNO) and the European Union (EU),is conceived with the objective of human growth, educational, and community development thereby uplifting the standard of living of people. Based on the random selection exercise of inter net websites and millions of supermarket cash invoices worldwide, you were selected among the lucky recipients to receive the award sum of US$1,000,000.00 (One Million United States Dollars) as grant/donations cash aid from the Vittorio Foundation in accordance with the enabling act of Parliament. (Note that all beneficiaries email addresses were selected randomly from over 100,000 internet websites or a shop's cash invoice around your area in which you might have purchased something from). You are required to contact our office annex in MADRID SPAIN for the processing of your grant/donation cash aid as this will ensure efficiency in our services, meanwhile you will be given your donation pin number/payment-release-order-form, which you will use in collecting the funds. Please endeavor to quote your Qualification numbers (S-222-6747, C-900-56) in all discussions. CONTACT DETIALS BELOW: =========================================== Fondazione Di Vittorio ESPAÑOLA CONTACT NAME: DR.MANUEL PABLO E-MAIL CONTACT: fdvitorrio.es at terra.es [mailto:fdvitorrio.es at terra.es] =========================================== All information is strictly confidential and will only be used for the purpose to which it is been requested inorder to avoid unwarranted taking advantage of this programme by non-participant or unofficial personnel. Please note that these grant/donation cash aid are strictly administered by Foundazion di Vittorio, under delegated powers from the United Nations Organization (UNO). Finally, all funds should be claimed by their respective beneficiaries, no later than 15 days after notification. On behalf of the Board kindly, accept our warmest congratulations. Regards. Mr. Andrea Gianfagna (Foundation officer) Bis 50 MB Dateianhänge? Kein Problem! *http://freemail.web.de/club/landingpage.htm/?mc=025556* [http://freemail.web.de/club/landingpage.htm/?mc=025556] -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at pleshood.com Fri Mar 21 17:30:42 2008 From: info at pleshood.com (=?windows-1255?B?4/jqIOT59PI=?=) Date: Fri, 21 Mar 2008 19:30:42 -0500 Subject: [ofa-general] =?windows-1255?b?9/jp6fjkIO74+vf6IPLtIPfl4yDk9+nl?= =?windows-1255?q?=ED?= Message-ID: <20080322003142.5DFAAE60A0A@openfabrics.org> An HTML attachment was scrubbed... URL: From dwspacifym at spacify.com Fri Mar 21 18:52:58 2008 From: dwspacifym at spacify.com (Gino Santiago) Date: Sat, 22 Mar 2008 04:52:58 +0300 Subject: [ofa-general] Make significant savings buying medications in Canada. Message-ID: <01c88bd8$993a7100$8a9ccc54@dwspacifym> Speaking about e-shops, Canadian Pharmacy drugstore is the most trustworthy online drugstore that has gained a reputation of providing with the top quality meds at the absolutely low cost. We understand that terms of delivery are extremely important when it comes to health products, so we deliver all the orders really fast. Make a secure and confidential purchase with Canadian Pharmacy. Check our prices and you will definitely make the order. http://schoolsudden.com «CanadianPharmacy» will satisfy all you pharmaceutical needs. Mike Lee From rdreier at cisco.com Fri Mar 21 19:47:00 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 19:47:00 -0700 Subject: [ofa-general] [PATCH 0/10] IB/ipath -- patches in for-roland for 2.6.26 In-Reply-To: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Thu, 20 Mar 2008 17:17:35 -0700") References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: By the way... I got tired of seeing the warning, so I added the patch below to my tree. Please let me know if this is not the right fix and I will drop it... IB/ipath: Fix PCI config write size used to clear linkctrl error bits In slave_or_pri_blk(), pci_write_config_byte() is used to write a 16-bit quantity to clear linkctrl CRC error bits. This is clearly a bug and also causes the warning drivers/infiniband/hw/ipath/ipath_iba6110.c: In function 'slave_or_pri_blk': drivers/infiniband/hw/ipath/ipath_iba6110.c:849: warning: overflow in implicit constant conversion Fix this by using pci_write_config_word() instead. Signed-off-by: Roland Dreier --- drivers/infiniband/hw/ipath/ipath_iba6110.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_iba6110.c b/drivers/infiniband/hw/ipath/ipath_iba6110.c index 19e3955..d241f1c 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6110.c +++ b/drivers/infiniband/hw/ipath/ipath_iba6110.c @@ -845,7 +845,7 @@ static void slave_or_pri_blk(struct ipath_devdata *dd, struct pci_dev *pdev, /* * now write them back to clear the error. */ - pci_write_config_byte(pdev, link_off, + pci_write_config_word(pdev, link_off, linkctrl & (0xf << 8)); } } -- 1.5.4.3 From rdreier at cisco.com Fri Mar 21 19:51:52 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 19:51:52 -0700 Subject: [ofa-general] Re: [PATCH] IB/srp: enforce protocol limit on srp_sg_tablesize In-Reply-To: <1206140620.4685.4.camel@obelisk.thedillows.org> (David Dillow's message of "Fri, 21 Mar 2008 19:03:40 -0400") References: <1205777880.10010.26.camel@lap75545.ornl.gov> <1205782965.10010.30.camel@lap75545.ornl.gov> <1206140620.4685.4.camel@obelisk.thedillows.org> Message-ID: > I see you queued this for 2.6.26 -- I'd have thought that this would go > to 2.6.25 as it is a bug fix. What do you think? I didn't think it was serious enough to go in at the rc5/rc6 stage. It's borderline even to call it a bug, and the workaround is pretty trivial -- "don't do that then." - R. From rdreier at cisco.com Fri Mar 21 20:07:40 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 21 Mar 2008 20:07:40 -0700 Subject: [ofa-general] RE: [PATCH 3/3] Infiniband: don't use task_struct.tgid in make_cm_node() In-Reply-To: <5E701717F2B2ED4EA60F87C8AA57B7CC0795008A@venom2> (Glenn Streiff's message of "Thu, 20 Mar 2008 16:51:31 -0500") References: <5E701717F2B2ED4EA60F87C8AA57B7CC0795008A@venom2> Message-ID: Thanks, I added this to my tree: commit 0f9d8de35f49df534b79d656155d3cb1ff70f726 Author: Roland Dreier Date: Fri Mar 21 20:06:49 2008 -0700 RDMA/nes: Remove session_id from nes_cm stuff The session_id members of struct nes_cm_listener and struct nes_cm_node are write-only, so remove them. This allows the session_id member of struct nes_cm_core to be removed as well, since it is only used to write those other session_id values. This removes the use of current->tgid (which will be deprecated) pointed out by Pavel Emelyanov . Acked-by: Glenn Streiff Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c index c1a85f3..31a28e9 100644 --- a/drivers/infiniband/hw/nes/nes_cm.c +++ b/drivers/infiniband/hw/nes/nes_cm.c @@ -365,7 +365,6 @@ static void print_core(struct nes_cm_core *core) if (!core) return; nes_debug(NES_DBG_CM, "---------------------------------------------\n"); - nes_debug(NES_DBG_CM, "Session ID : %u \n", atomic_read(&core->session_id)); nes_debug(NES_DBG_CM, "State : %u \n", core->state); @@ -1100,8 +1099,6 @@ static struct nes_cm_node *make_cm_node(struct nes_cm_core *cm_core, cm_node->tcp_cntxt.rcv_nxt = 0; /* get a unique session ID , add thread_id to an upcounter to handle race */ atomic_inc(&cm_core->node_cnt); - atomic_inc(&cm_core->session_id); - cm_node->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); cm_node->conn_type = cm_info->conn_type; cm_node->apbvt_set = 0; cm_node->accept_pend = 0; @@ -1628,9 +1625,7 @@ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core, listener->cm_core = cm_core; listener->nesvnic = nesvnic; atomic_inc(&cm_core->node_cnt); - atomic_inc(&cm_core->session_id); - listener->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); listener->conn_type = cm_info->conn_type; listener->backlog = cm_info->backlog; listener->listener_state = NES_CM_LISTENER_ACTIVE_STATE; @@ -1943,7 +1938,6 @@ static struct nes_cm_core *nes_cm_alloc_core(void) cm_core->state = NES_CM_STATE_INITED; cm_core->free_tx_pkt_max = NES_CM_DEFAULT_FREE_PKTS; - atomic_set(&cm_core->session_id, 0); atomic_set(&cm_core->events_posted, 0); /* init the packet lists */ diff --git a/drivers/infiniband/hw/nes/nes_cm.h b/drivers/infiniband/hw/nes/nes_cm.h index 980fb67..7717cb2 100644 --- a/drivers/infiniband/hw/nes/nes_cm.h +++ b/drivers/infiniband/hw/nes/nes_cm.h @@ -225,7 +225,6 @@ enum nes_cm_listener_state { struct nes_cm_listener { struct list_head list; - u64 session_id; struct nes_cm_core *cm_core; u8 loc_mac[ETH_ALEN]; nes_addr_t loc_addr; @@ -242,7 +241,6 @@ struct nes_cm_listener { /* per connection node and node state information */ struct nes_cm_node { - u64 session_id; u32 hashkey; nes_addr_t loc_addr, rem_addr; @@ -327,7 +325,6 @@ struct nes_cm_event { struct nes_cm_core { enum nes_cm_node_state state; - atomic_t session_id; atomic_t listen_node_cnt; struct nes_cm_node listen_list; From kcpuwhyd at bodymath.com Fri Mar 21 20:58:17 2008 From: kcpuwhyd at bodymath.com (Chang Willis) Date: Sat, 22 Mar 2008 12:58:17 +0900 Subject: [ofa-general] Melt away fat easily Message-ID: <01c88c1c$65813a80$1ddd8c7d@kcpuwhyd> Want to shave a few pounds? You must know this secret! Look attached details or visit out site (LINK IS IN ATTACHED DETAILS) -------------- next part -------------- A non-text attachment was scrubbed... Name: as.zip Type: application/zip Size: 578 bytes Desc: not available URL: From rajouri.jammu at gmail.com Fri Mar 21 21:02:09 2008 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Fri, 21 Mar 2008 21:02:09 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> <3307cdf90803211157l1c5b3febra0d4c89028b4523b@mail.gmail.com> Message-ID: <3307cdf90803212102x1d2500dfv22edd0e26175e7f6@mail.gmail.com> On Fri, Mar 21, 2008 at 2:08 PM, Roland Dreier wrote: > > I burnt a mem-free f/w on the card and this time I'm getting a > different > > error when trying to set num_mtt to 2097152. > > > I did: > > > modprobe ib_mtcha num_mpt=262144 num_mtt=2097152 > > > kernel: ib_mthca 0000:01:00.0: Failed to initialize memory region > table, aborting. > > You may not have enough contiguous memory for the buddy allocator > bitmaps for the MTT. What does /proc/buddyinfo show when this > happens? > I have 32GB of physical memory in the system. cat /proc/buddyinfo Node 0, zone DMA 3 5 3 2 4 0 2 0 2 0 2 Node 0, zone DMA32 1 0 0 2 5 4 2 2 3 3 486 Node 0, zone Normal 80 38 8 189 335 46 40 3 3 1 7366 No idea how to interpret that. Can I change some system parameter so that it gets the required memory? > > Roland, when your attempt succeeded were you using a mem-free HCA or > the > > 256MB version? > > mem-free. With the fixes I just posted, I am able to load the driver > with num_mtt=33554432 and have it work in fact. > > It's not clear to me why you need such crazy values. Having 2M MTT > segments means you can map 64GB of memory with 4KB pages... do you > really need that much memory pinned for IB operations? > I thought 2M MTTs will allow to map up to 8GB of memory? 2^21*4K. Is that not correct? > > - R. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dwsolarfishm at solarfish.com Sat Mar 22 01:38:23 2008 From: dwsolarfishm at solarfish.com (Sandra Bialas) Date: Sat, 22 Mar 2008 13:38:23 +0500 Subject: [ofa-general] You have no need to look for a reliable online drugstore anymore. Message-ID: <01c88c21$ff97b180$6cd5ef4d@dwsolarfishm> «CanadianPharmacy» is committed to provide their customers with top quality medications at absolutely low prices. The major part of our customers is Americans as medications in Canada are cheaper than in America. We guarantee that your private information is strictly protected and your purchase will be confidential. We ship internationally and deliver fast with tracking possibility. Medications from all popular categories! http://underbrother.com The cheapest place to buy your meds! Sandra Bialas From a-4c at abandapart.com Sat Mar 22 02:05:38 2008 From: a-4c at abandapart.com (Arturo Abraham) Date: Sat, 22 Mar 2008 17:05:38 +0800 Subject: [ofa-general] Where have you been? Message-ID: <01c88c3e$f36e1d00$949fd43c@a-4c> Hello! I am tired today. I am nice girl that would like to chat with you. Email me at Berit at BestGolova.com only, because I am using my friend's email to write this. Wanna see some pictures of me? From rdreier at cisco.com Fri Mar 21 20:07:40 2008 From: rdreier at cisco.com (ext Roland Dreier) Date: Fri, 21 Mar 2008 20:07:40 -0700 Subject: [ofa-general] RE: [PATCH 3/3] Infiniband: don't use task_struct.tgid in make_cm_node() In-Reply-To: <5E701717F2B2ED4EA60F87C8AA57B7CC0795008A@venom2> (Glenn Streiff's message of "Thu, 20 Mar 2008 16:51:31 -0500") References: <5E701717F2B2ED4EA60F87C8AA57B7CC0795008A@venom2> Message-ID: Thanks, I added this to my tree: commit 0f9d8de35f49df534b79d656155d3cb1ff70f726 Author: Roland Dreier Date: Fri Mar 21 20:06:49 2008 -0700 RDMA/nes: Remove session_id from nes_cm stuff The session_id members of struct nes_cm_listener and struct nes_cm_node are write-only, so remove them. This allows the session_id member of struct nes_cm_core to be removed as well, since it is only used to write those other session_id values. This removes the use of current->tgid (which will be deprecated) pointed out by Pavel Emelyanov . Acked-by: Glenn Streiff Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c index c1a85f3..31a28e9 100644 --- a/drivers/infiniband/hw/nes/nes_cm.c +++ b/drivers/infiniband/hw/nes/nes_cm.c @@ -365,7 +365,6 @@ static void print_core(struct nes_cm_core *core) if (!core) return; nes_debug(NES_DBG_CM, "---------------------------------------------\n"); - nes_debug(NES_DBG_CM, "Session ID : %u \n", atomic_read(&core->session_id)); nes_debug(NES_DBG_CM, "State : %u \n", core->state); @@ -1100,8 +1099,6 @@ static struct nes_cm_node *make_cm_node(struct nes_cm_core *cm_core, cm_node->tcp_cntxt.rcv_nxt = 0; /* get a unique session ID , add thread_id to an upcounter to handle race */ atomic_inc(&cm_core->node_cnt); - atomic_inc(&cm_core->session_id); - cm_node->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); cm_node->conn_type = cm_info->conn_type; cm_node->apbvt_set = 0; cm_node->accept_pend = 0; @@ -1628,9 +1625,7 @@ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core, listener->cm_core = cm_core; listener->nesvnic = nesvnic; atomic_inc(&cm_core->node_cnt); - atomic_inc(&cm_core->session_id); - listener->session_id = (u32)(atomic_read(&cm_core->session_id) + current->tgid); listener->conn_type = cm_info->conn_type; listener->backlog = cm_info->backlog; listener->listener_state = NES_CM_LISTENER_ACTIVE_STATE; @@ -1943,7 +1938,6 @@ static struct nes_cm_core *nes_cm_alloc_core(void) cm_core->state = NES_CM_STATE_INITED; cm_core->free_tx_pkt_max = NES_CM_DEFAULT_FREE_PKTS; - atomic_set(&cm_core->session_id, 0); atomic_set(&cm_core->events_posted, 0); /* init the packet lists */ diff --git a/drivers/infiniband/hw/nes/nes_cm.h b/drivers/infiniband/hw/nes/nes_cm.h index 980fb67..7717cb2 100644 --- a/drivers/infiniband/hw/nes/nes_cm.h +++ b/drivers/infiniband/hw/nes/nes_cm.h @@ -225,7 +225,6 @@ enum nes_cm_listener_state { struct nes_cm_listener { struct list_head list; - u64 session_id; struct nes_cm_core *cm_core; u8 loc_mac[ETH_ALEN]; nes_addr_t loc_addr; @@ -242,7 +241,6 @@ struct nes_cm_listener { /* per connection node and node state information */ struct nes_cm_node { - u64 session_id; u32 hashkey; nes_addr_t loc_addr, rem_addr; @@ -327,7 +325,6 @@ struct nes_cm_event { struct nes_cm_core { enum nes_cm_node_state state; - atomic_t session_id; atomic_t listen_node_cnt; struct nes_cm_node listen_list; -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ From giustizia.itsimonavitali at giustizia.it Sat Mar 22 02:54:44 2008 From: giustizia.itsimonavitali at giustizia.it (EuroSoftware) Date: Sat, 22 Mar 2008 10:54:44 +0100 Subject: [ofa-general] MS Office XP, Adobe Acrobat 8, MS Office 2007 Message-ID: <490870658.33482907075942@giustizia.it> An HTML attachment was scrubbed... URL: From lfv at mb.uni-dortmund.de Sat Mar 22 05:47:57 2008 From: lfv at mb.uni-dortmund.de (Jules Lyon) Date: Sat, 22 Mar 2008 14:47:57 +0200 Subject: [ofa-general] Energy fur ihren Schw. Anz, kaufen und 85% sparen Message-ID: <01c88c2b$b7e2de00$0fd1fc58@lfv> Haben Sie endlich wieder Spass am Leben! Pr. .. Eise die keine Konk... ..Urrenz kennen - Disk rete Verpackung und Zahlung - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - keine versteckte Kos// Ten - Bequem und dis kret 0... . N-line! be... .Stellen. - Kein peinlicher Arz t besuch erforderlich - Kos... Tenlose, arztliche Telefon-Beratung Originalme/ dikamente Ciia..aa^_^aaalis...... 10 Pack. 21,00 Euro Viia..aa^_^aaagra... 10 Pack. 11,00 Euro Jetzt bestellen - und vier Pil. .. len umsonst erhalten http://speeddress.com (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From heather.fleece at ipaper.com Sat Mar 22 06:27:08 2008 From: heather.fleece at ipaper.com (EuroSoftware) Date: Sat, 22 Mar 2008 14:27:08 +0100 Subject: [ofa-general] Weniger zahlen fuer perfekte Standardsoftware Message-ID: <209776720.16649592496664@ipaper.com> Software full line in cheap&quick OEM e-shopWir freuen uns darauf, Ihnen lokalisierte Versionen bekannter Programme anbieten zu können: Englisch, Deutsch, Französisch, Italienisch, Spanisch und viele andere Sprachen! Sofort nach dem Kauf können Sie jedes Programm herunterladen und installieren.http://geocities.com/macdonaldmilo/* Windows XP Professional With SP2 Full Version: $59.95 * Office Enterprise 2007: $79.95 * Adobe Photoshop CS3 Extended: $79.95 * Adobe Acrobat 8.0 Professional: $69.95 http://geocities.com/macdonaldmilo/Wir haben mehr 300 verschiedener Programmes für PC und Macintosh! Kaufen jetzt, warten Sie nicht! -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Sat Mar 22 06:33:34 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Sat, 22 Mar 2008 06:33:34 -0700 Subject: [ofa-general] [PATCH 0/5] trap 144, node descriptor change handling. In-Reply-To: <20080321155112.289cf10f.weiny2@llnl.gov> References: <20080321155112.289cf10f.weiny2@llnl.gov> Message-ID: <1206192814.8099.375.camel@hrosenstock-ws.xsigo.com> On Fri, 2008-03-21 at 15:51 -0700, Ira Weiny wrote: > I have altered these patches based on input from Hal. There are now 5 of them. > > 1) Enhance set_nodedesc.sh > 2) Add optional ibsendtrap test tool and create --enable-test-utils > configure option > 3) Add mcm_rereg_test to optional test tool config option > (This also updates the tool to use the new ibmad_gid_t since it would not > compile otherwise.) > 4) Update OpenSM to respond to node description change flag > 5) Create osm_node_desc.h and move extern declarations into this common > header. Oh, I was looking at an older OpenSM when I suggested adding these to osm_node_desc_rcv.h. OFED 1.3 has this but master doesn't. -- Hal > This includes Hal's minor stylistic fixes as well as makes ibsendtrap an > optional testing util, rather than a default diag. > > Thanks, > Ira > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From a-amybro at a3sc.co.kr Sat Mar 22 07:11:56 2008 From: a-amybro at a3sc.co.kr (Jenny Warren) Date: Sat, 22 Mar 2008 22:11:56 +0800 Subject: [ofa-general] You told me we can chat Message-ID: <365974556.48179169065010@a3sc.co.kr> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Anita at BestGolova.com only, because I am using my friend's email to write this. Hope you like my pictures. From jfrankling at msigusa.com Sat Mar 22 07:27:33 2008 From: jfrankling at msigusa.com (Alvaro Greenwood) Date: Sat, 22 Mar 2008 15:27:33 +0100 Subject: [ofa-general] Gro?er Softwareverkauf! Niedrigste Preise! Message-ID: <669969077.38822931599845@msigusa.com> An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Sat Mar 22 07:35:52 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Sat, 22 Mar 2008 07:35:52 -0700 Subject: [ofa-general] [PATCH] OpenSM release notes: Add in new QLogic HCAs Message-ID: <1206196552.8099.394.camel@hrosenstock-ws.xsigo.com> OpenSM release notes: Add in new QLogic HCAs Please apply to both master and OFED 1.3 Signed-off-by: Hal Rosenstock diff --git a/opensm/doc/opensm_release_notes-3.1.10.txt b/opensm/doc/opensm_release_notes-3.1.10.txt index 732ff59..8e552d5 100644 --- a/opensm/doc/opensm_release_notes-3.1.10.txt +++ b/opensm/doc/opensm_release_notes-3.1.10.txt @@ -474,6 +474,8 @@ Device | Note iPath | QHT6040 (PathScale InfiniPath HT-460) iPath | QHT6140 (PathScale InfiniPath HT-465) iPath | QLE6140 (PathScale InfiniPath PE-880) +iPath | QLE7240 +iPath | QLE7280 Note 1: OpenSM does not run on an IBM Galaxy (eHCA) as it does not expose QP0 and QP1. However, it does support it as a device on the subnet. From syntan7 at pcbinc.com Sat Mar 22 08:13:10 2008 From: syntan7 at pcbinc.com (EuroSoftware) Date: Sat, 22 Mar 2008 16:13:10 +0100 Subject: [ofa-general] MS Office 2007, AutoCAD 2008, Adobe Acrobat 8 Message-ID: <521180081.04086965500242@pcbinc.com> Um die echte und vollige Software in kurzer Zeit zu bekommen, braucht man nur zu bezahlen und auszulasten. Sie haben dann die Programmen auf allen europaischen Sprachen uberlassen, die fur Windows und Macintosh vorherbestimmt sind. Die Aufstellung des Programms ist jetzt kein Problem fur Ihnen. Die professionelle Konsultation des Anwenderdienstes hilft dabei. Garantiert sind die schnelle Antwort und die Moglichkeit der Ruckzahlung. Wenn Sie die Software kaufen, kaufen Sie nur die vollkommen funktionierende Software http://geocities.com/gregory_kip/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sh04008 at kralen.com Sat Mar 22 09:27:26 2008 From: sh04008 at kralen.com (Imani Marin) Date: Sat, 22 Mar 2008 13:27:26 -0300 Subject: [ofa-general] AutoCAD 2008, Adobe Acrobat 8, Photoshop CS3 Message-ID: <171440646.84983363735329@kralen.com> Purchase software at surprisingly low pricesBekommen Sie Ihre Software unverzueglich. Einfach zahlen und sofort runterladen. Hier sind Programme in allen europaeischen Sprachen verfuegbar, programmiert fuer Windows und Macintosh. Alle Softwaren sind sehr guenstig, es handelt sich dabei garantiert um originale, komplette und voellig funktionale Versionen. http://geocities.com/lesliepruitt56/* Office Enterprise 2007: $79.95 * Windows XP Professional With SP2 Full Version: $59.95 * Adobe Photoshop CS3 Extended: $79.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/lesliepruitt56/Sie werden nach dem Kauf nicht alleine gelassen. Unsere freundlichen Kundenberater werden Ihnen bei der Installation helfen, falls Sie es benotigen. Schnelle Antworten und Geld-Zurueck-Garantie sind bei uns selbstverstandlich. -------------- next part -------------- An HTML attachment was scrubbed... URL: From budhrani at gibtelecom.net Sat Mar 22 09:50:57 2008 From: budhrani at gibtelecom.net (KIA MOTORS PROMOTION) Date: Sat, 22 Mar 2008 17:50:57 +0100 Subject: [ofa-general] YOU JUST WON! Message-ID: <1206204657-5591.00011.00728-smmsdV2.1.6@mta1.mail.gibtelecom.net> Attn: You have been selected for a cash prize of 1.5 MillionGBP.and a brand new KIA Picanto from the Kia company contact walter jones, Email: kia_annualpromo at yahoo.co.uk, secret pin code KIA/001/11234 and your referencenumber TYT: 12058006/06 1;NAMES ........... 2;ADDRESS ................ 3;COUNTRY............... 4;OCCUPATION ............... 5;SEX ................. 6;PHONE ................. 7;AGE ................ 8;ATTACHED PHOTOGRAPH............ Congratulations once again ! From estate at pair.com Sat Mar 22 10:27:30 2008 From: estate at pair.com (Frankie Dolan) Date: Sat, 22 Mar 2008 18:27:30 +0100 Subject: [ofa-general] =?iso-8859-1?q?Sie_bekommen=2C_was_Sie_wollen_=96_g?= =?iso-8859-1?q?ute_Pillen?= Message-ID: <196188008.45346694483353@pair.com> Wir sind froh, Ihnen die Apotheke EuropeanPharmacy vorstellen zu koennen. Es ist eine vertrauliche und ganz professionelle Quelle, um die hochwertigsten Medikamente ganz guenstig zu besorgen. Es wird auch sehr viel zur Auswahl angeboten. 100%-tige Sicherheit zu den niedrigsten Preisen. http://ksyeeb.winterdid.cn/?083417368532Produkte nur mit Lizenz. Vertraulicher und sicherer Kauf. Blitzschnelle Lieferung. -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-alanb at adasoft.ch Sat Mar 22 11:25:11 2008 From: a-alanb at adasoft.ch (Cheryl Blackburn) Date: Sat, 22 Mar 2008 15:25:11 -0300 Subject: [ofa-general] did you miss me Message-ID: <01c88c30$eb0ee580$94894ebd@a-alanb> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Sofie at BestGolova.com only, because I am using my friend's email to write this. I will reply with my pics From dillowda at ornl.gov Sat Mar 22 12:53:40 2008 From: dillowda at ornl.gov (David Dillow) Date: Sat, 22 Mar 2008 15:53:40 -0400 Subject: [ofa-general] Re: [PATCH] IB/srp: enforce protocol limit on srp_sg_tablesize In-Reply-To: References: <1205777880.10010.26.camel@lap75545.ornl.gov> <1205782965.10010.30.camel@lap75545.ornl.gov> <1206140620.4685.4.camel@obelisk.thedillows.org> Message-ID: <1206215620.4685.19.camel@obelisk.thedillows.org> On Fri, 2008-03-21 at 19:51 -0700, Roland Dreier wrote: > > I see you queued this for 2.6.26 -- I'd have thought that this would go > > to 2.6.25 as it is a bug fix. What do you think? > > I didn't think it was serious enough to go in at the rc5/rc6 stage. > It's borderline even to call it a bug, and the workaround is pretty > trivial -- "don't do that then." While certainly we'd want to avoid large changes for minor bugs this late in the -rc cycle, a small, obvious change to ensure protocol correctness would be fine in my opinion. The problem with the workaround is that you won't hit the exception case until you have memory fragmentation, and then you get a dropped/hung SRP connection or in the hopefully unlikely case, data corruption. And even then you're not likely to realize that you made the sg table too large -- it does not have an obvious link to the symptoms. That said, I'm not vehement about it waiting an extra cycle. I know to keep my sg table entries down, and hopefully others that don't know there is an issue will be reading the list. Since it won't be in an OFED release for a while, I'd bet they're unlikely to be upgrading to 2.6.25 in any event. -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office From jijiherex at aoc.com Sat Mar 22 12:52:28 2008 From: jijiherex at aoc.com (boyd darcy) Date: Sat, 22 Mar 2008 19:52:28 +0000 Subject: [ofa-general] New Britney P*ssy shot Message-ID: <000901c88c65$038edc1b$3f0d8097@uftonrf> * Reaktionen: "Solche Taten lassen sich nicht verhindern" Download and Watch -------------- next part -------------- An HTML attachment was scrubbed... URL: From sibley at 2spi.com Sat Mar 22 14:25:10 2008 From: sibley at 2spi.com (eugenio jenny) Date: Sat, 22 Mar 2008 21:25:10 +0000 Subject: [ofa-general] New video with a naked celebrity Avril Lavigne Message-ID: <000901c88c72$021e0329$90e84999@nraosx> #ccvLBBChristina Aguilera Kick-up photo. #ZDnwmNThe presentation is Full! #cvLBBlOnly 1 day trial - get this Interesting cd now! #ZDnwmN Download it now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefano.ricciarelli at 2erremme.it Sat Mar 22 20:18:08 2008 From: stefano.ricciarelli at 2erremme.it (fairfax danna) Date: Sun, 23 Mar 2008 03:18:08 +0000 Subject: [ofa-general] Rihanna exposed Message-ID: <000801c88ca3$04d83778$3a4aa898@gtluvjwt> Download and Watch -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Sat Mar 22 22:48:57 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 23 Mar 2008 07:48:57 +0200 Subject: [ofa-general] files preambule In-Reply-To: References: Message-ID: <47E5EF49.9080506@voltaire.com> Kanevsky, Arkady wrote: > I noticed that all files in the preambule states that the file can be > taken under GNU GPLv2 or > OpenIB BSD licence. > Should not it state OpenFabrics instead? My understanding is that when ofa was founded it was decided to go on this --specific-- dual license, what reasoning you see to change that? Or. From ogerlitz at voltaire.com Sat Mar 22 23:36:29 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 23 Mar 2008 08:36:29 +0200 Subject: [ofa-general] [PATCH 3/10] IB/core: Add LSO support In-Reply-To: <1205767431.25950.138.camel@mtls03> References: <1205767431.25950.138.camel@mtls03> Message-ID: <47E5FA6D.7070307@voltaire.com> Eli Cohen wrote: > LSO allows the networikng stack to pass pass to the network driver SKBs > with data size larger then MTU and let the HW fragment the data to mss > sized packets. > Hi Eli, please see some comments below > --- a/include/rdma/ib_verbs.h > +++ b/include/rdma/ib_verbs.h > @@ -411,6 +412,7 @@ enum ib_wc_opcode { > IB_WC_COMP_SWAP, > IB_WC_FETCH_ADD, > IB_WC_BIND_MW, > + IB_WC_LSO, > with IB_WC_LSO never being used over this patchset, can we just remove it? > @@ -622,7 +624,8 @@ enum ib_wr_opcode { > IB_WR_SEND_WITH_IMM, > IB_WR_RDMA_READ, > IB_WR_ATOMIC_CMP_AND_SWP, > - IB_WR_ATOMIC_FETCH_AND_ADD > + IB_WR_ATOMIC_FETCH_AND_ADD, > + IB_WR_LSO > }; > > enum ib_send_flags { > @@ -630,7 +633,8 @@ enum ib_send_flags { > IB_SEND_SIGNALED = (1<<1), > IB_SEND_SOLICITED = (1<<2), > IB_SEND_INLINE = (1<<3), > - IB_SEND_IP_CSUM = (1<<4) > + IB_SEND_IP_CSUM = (1<<4), > + IB_SEND_UDP_LSO = (1<<5) > }; > IB_SEND_UDP_LSO is never used in this patchset, I guess you wanted to call it IB_SEND_TCP_LSO. Also how about using it in ipoib at the same manner as the IB_SEND_IP_CSUM bit is? (ie OR it into the send flags of the UD WR). With this in mind, I suggest that you remove the IB_WR_LSO. > > struct ib_sge { > @@ -660,6 +664,9 @@ struct ib_send_wr { > } atomic; > struct { > struct ib_ah *ah; > + void *header; > + int hlen; > + int mss; > Can you add shorting documentation for the new fields? Or. From ogerlitz at voltaire.com Sat Mar 22 23:43:32 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 23 Mar 2008 08:43:32 +0200 Subject: [ofa-general] [PATCH 6/10] IB/mlx4: Add LSO support In-Reply-To: <1205767443.25950.141.camel@mtls03> References: <1205767443.25950.141.camel@mtls03> Message-ID: <47E5FC14.6070802@voltaire.com> Eli Cohen wrote: > Add LSO support to mlx4 driver such that it will be able > to send SKBs passed from the driver which publish NETIF_TSO. > The patch is full with printk calls, please remove them. Or. From ogerlitz at voltaire.com Sun Mar 23 00:33:38 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 23 Mar 2008 09:33:38 +0200 Subject: [ofa-general] [PATCH 4/10] IB/ipoib: Add LSO support to ipoib In-Reply-To: <1205767435.25950.139.camel@mtls03> References: <1205767435.25950.139.camel@mtls03> Message-ID: <47E607D2.5000207@voltaire.com> Eli Cohen wrote: > --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c > @@ -418,14 +449,35 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, > { > + int hlen; > + void *phead; > + > + if (!skb_is_gso(skb)) { > + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { > + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", > + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); > + ++dev->stats.tx_dropped; > + ++dev->stats.tx_errors; > + ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); > + return; > + } > + phead = 0; > + hlen = 0; > + } else { > + /* > + * LSO header is limited to max 60 bytes > + */ > + if (unlikely((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) > 15)) { > + ipoib_warn(priv, "ip(%d) and tcp(%d) headers too long, dropping skb\n", > + ip_hdr(skb)->ihl << 2, tcp_hdr(skb)->doff << 2); > + goto drop; > + } > is the 60 bytes being a limitation of the connectX HW, the Linux kernel stack or some "lso spec"? > + hlen = ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) + IPOIB_ENCAP_LEN; > + phead = skb->data; > Looking in e1000 tso code (eg http://lxr.linux.no/linux/drivers/net/e1000/e1000_main.c#L2884) I see that the header len is gotten by skb_transport_offset(skb) + tcp_hdrlen(skb), is that what ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) gives? > @@ -470,6 +521,12 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, > netif_stop_queue(dev); > } > } > + return; > + > +drop: > + ++dev->stats.tx_errors; > + dev_kfree_skb_any(skb); > + return; > shouldn't the tx_dropped counter be incremented here? Or. From ogerlitz at voltaire.com Sun Mar 23 00:33:38 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 23 Mar 2008 09:33:38 +0200 Subject: [ofa-general] [PATCH 4/10] IB/ipoib: Add LSO support to ipoib In-Reply-To: <1205767435.25950.139.camel@mtls03> References: <1205767435.25950.139.camel@mtls03> Message-ID: <47E607D2.5000207@voltaire.com> Eli Cohen wrote: > --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c > @@ -418,14 +449,35 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, > { > + int hlen; > + void *phead; > + > + if (!skb_is_gso(skb)) { > + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { > + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", > + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); > + ++dev->stats.tx_dropped; > + ++dev->stats.tx_errors; > + ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); > + return; > + } > + phead = 0; > + hlen = 0; > + } else { > + /* > + * LSO header is limited to max 60 bytes > + */ > + if (unlikely((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) > 15)) { > + ipoib_warn(priv, "ip(%d) and tcp(%d) headers too long, dropping skb\n", > + ip_hdr(skb)->ihl << 2, tcp_hdr(skb)->doff << 2); > + goto drop; > + } > is the 60 bytes being a limitation of the connectX HW, the Linux kernel stack or some "lso spec"? > + hlen = ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) + IPOIB_ENCAP_LEN; > + phead = skb->data; > Looking in e1000 tso code (eg http://lxr.linux.no/linux/drivers/net/e1000/e1000_main.c#L2884) I see that the header len is gotten by skb_transport_offset(skb) + tcp_hdrlen(skb), is that what ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) gives? > @@ -470,6 +521,12 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, > netif_stop_queue(dev); > } > } > + return; > + > +drop: > + ++dev->stats.tx_errors; > + dev_kfree_skb_any(skb); > + return; > shouldn't the tx_dropped counter be incremented here? Or. From jackm at dev.mellanox.co.il Sun Mar 23 00:52:40 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Sun, 23 Mar 2008 09:52:40 +0200 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: References: Message-ID: <200803230952.41260.jackm@dev.mellanox.co.il> On Thursday 20 March 2008 21:59, Hoang-Nam Nguyen wrote: > Hi Roland and Eli! > >+enum qp_create_flags { > >+ QP_CREATE_LSO = 1 << 0, > >+}; > Our ehca2 supports sort of low-latency QP for UD and RC. It would be great > if we can e.g. enhance above enum like this > QP_CREATE_LL = 1 << 1 > If you agree with, I'll create a patch. For kernel space the changes within > ehca should not be that much. > Thx > Nam > Please use QP_CREATE_LL 1 << 2 I'm already using 1<<1 for XRC receive QPs: enum qp_create_flags { - QP_CREATE_LSO = 1 << 0, + QP_CREATE_LSO = 1 << 0, + QP_CREATE_XRC_RCV = 1 << 1, }; (from OFA 1.3, file kernel_patches/fixes/core_0110_xrc_rcv.patch) Thanks! - Jack > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From jackm at dev.mellanox.co.il Sun Mar 23 00:52:40 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Sun, 23 Mar 2008 09:52:40 +0200 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: References: Message-ID: <200803230952.41260.jackm@dev.mellanox.co.il> On Thursday 20 March 2008 21:59, Hoang-Nam Nguyen wrote: > Hi Roland and Eli! > >+enum qp_create_flags { > >+ QP_CREATE_LSO = 1 << 0, > >+}; > Our ehca2 supports sort of low-latency QP for UD and RC. It would be great > if we can e.g. enhance above enum like this > QP_CREATE_LL = 1 << 1 > If you agree with, I'll create a patch. For kernel space the changes within > ehca should not be that much. > Thx > Nam > Please use QP_CREATE_LL 1 << 2 I'm already using 1<<1 for XRC receive QPs: enum qp_create_flags { - QP_CREATE_LSO = 1 << 0, + QP_CREATE_LSO = 1 << 0, + QP_CREATE_XRC_RCV = 1 << 1, }; (from OFA 1.3, file kernel_patches/fixes/core_0110_xrc_rcv.patch) Thanks! - Jack > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From ogerlitz at voltaire.com Sun Mar 23 01:55:33 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 23 Mar 2008 10:55:33 +0200 Subject: [ofa-general] Re: [RFC][1/2] IPoIB UD 4K MTU support In-Reply-To: <1206005880.8399.20.camel@localhost.localdomain> References: <1206005880.8399.20.camel@localhost.localdomain> Message-ID: <47E61B05.2020003@voltaire.com> Shirley Ma wrote: > --- a/drivers/infiniband/ulp/ipoib/ipoib.h > +++ b/drivers/infiniband/ulp/ipoib/ipoib.h > @@ -61,6 +61,10 @@ enum { > > IPOIB_ENCAP_LEN = 4, > > + IPOIB_UD_MAX_PAYLOAD = 4096, > + IPOIB_UD_HEAD_SIZE = IB_GRH_BYTES + IPOIB_ENCAP_LEN, > + IPOIB_UD_RX_SG = (IPOIB_UD_MAX_PAYLOAD + IB_GRH_BYTES) / PAGE_SIZE, > Reading ipoib_ud_skb_put_frags below and its usage in the patch that follows, its unclear to me if IPOIB_UD_MAX_PAYLOAD is being made of (4K - IPOIB_ENCAP_LEN) + IPOIB_ENCAP_LEN or from adjustment to some IP header alignment constraint. Specifically, the design I'd like to see here is that the IPoIB header telling the type of the frame (ARP, IPv4, IPv6, etc) is provided up to the stack as part of the packet in the skb (eg its very useful with tcpdump/etc filters). Reading earlier threads I see that Roland suggested to allow for upto 4K-4 mtu towards the stack and use some internal buffer for the GRH where this buffer can be allocated and dma mapped once and being forget from till the driver cleans up, etc. Was there any problem with this approach? > +static inline void ipoib_ud_skb_put_frags(struct ipoib_dev_priv *priv, > + struct sk_buff *skb, > + unsigned int length) > +{ > + if (ipoib_ud_need_sg(priv->max_ib_mtu)) { > + skb_frag_t *frag = &skb_shinfo(skb)->frags[0]; > + /* > + * There is only two buffers needed for max_payload = 4K, > + * first buf size is IPOIB_UD_HEAD_SIZE > + */ > + skb->tail += IPOIB_UD_HEAD_SIZE; > + frag->size = length - IPOIB_UD_HEAD_SIZE; > + skb->data_len += frag->size; > + skb->truesize += frag->size; > + skb->len += length; > + } else > + skb_put(skb, length); > + > +} > I fail to follow what this code really wants to do and how it does it. Is there a must to touch "by hand" all the internal skb fields? also this function is called once by ipoib_ib_handle_rx_wc in the patch that follows, any reason not to make it static over there? Or. From erd at uphs.upenn.edu Sun Mar 23 07:07:44 2008 From: erd at uphs.upenn.edu (EuroSoftware) Date: , 23 Mar 2008 15:07:44 +0100 Subject: [ofa-general] MS Office XP, MS Office 2007, AutoCAD 2008 Message-ID: <306112742.29214387522680@uphs.upenn.edu> An HTML attachment was scrubbed... URL: From kliteyn at dev.mellanox.co.il Sun Mar 23 08:45:35 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Sun, 23 Mar 2008 17:45:35 +0200 Subject: [ofa-general] [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups Message-ID: <47E67B1F.8050102@dev.mellanox.co.il> Hi Sasha, QoS manager should set the right SL in the IPoIB broadcast groups to enforce the right SL for IPoIB traffic. At start it seemed to me that it won't work in the following scenario: - IPoIB partition created - IPoIB mcast group created - IPoIB connectivity was established between two ports - SL in the QoS policy file was changed - SM is doing heavy sweep, and setting new SL in the partition and mcast group, but the nodes already have connectivity, so they will continue to transmit on the previous SL But then I realized that this scenario will be a problem for any communication (not only IPoIB), because as long as clients are not issuing new PR request, they will have the old parameters that won't be affected by the changes in QoS policy. This patch is for ofed_1_3 branch. Please review and let me know what you think. Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/osm_qos_policy.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 39 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c index bde1e7e..4586a77 100644 --- a/opensm/opensm/osm_qos_policy.c +++ b/opensm/opensm/osm_qos_policy.c @@ -920,6 +920,13 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, if (p_qos_match_rule->p_qos_level->sl_set && p_prtn->sl != p_qos_match_rule->p_qos_level->sl) { + uint8_t sl; + uint32_t flow; + uint8_t hop; + cl_qmap_t * p_mlid_tbl; + osm_mgrp_t * p_mgrp; + osm_mgrp_t * p_next_mgrp; + /* overriding partition's SL */ osm_log(p_log, OSM_LOG_ERROR, @@ -931,6 +938,38 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, p_prtn->sl, p_qos_match_rule->p_qos_level->sl); p_prtn->sl = p_qos_match_rule->p_qos_level->sl; + + /* If this partition is an IPoIB partition, + there should be a matching MCast group. + Fix this group's SL too */ + + p_mlid_tbl = &p_qos_policy->p_subn->mgrp_mlid_tbl; + p_next_mgrp = (osm_mgrp_t *) cl_qmap_head(p_mlid_tbl); + while (p_next_mgrp != (osm_mgrp_t *) + cl_qmap_end(p_mlid_tbl)) { + p_mgrp = p_next_mgrp; + p_next_mgrp = (osm_mgrp_t *) + cl_qmap_next(&p_mgrp->map_item); + if (!p_mgrp->well_known) + continue; + if ((cl_ntoh16(p_mgrp->mcmember_rec.pkey) & 0x7fff) != + (cl_ntoh16(pkey) & 0x7fff)) + continue; + ib_member_get_sl_flow_hop( + p_mgrp->mcmember_rec.sl_flow_hop, + &sl, &flow, &hop); + if (sl != p_prtn->sl) { + osm_log(p_log, OSM_LOG_DEBUG, + "osm_qos_policy_validate: " + "Updating MCGroup (MLID 0x%04x) SL to " + "match partition SL %u\n", + cl_hton16(p_mgrp->mcmember_rec.mlid), + p_prtn->sl); + p_mgrp->mcmember_rec.sl_flow_hop = + ib_member_set_sl_flow_hop( + p_prtn->sl, flow, hop); + } + } } } } -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Sun Mar 23 16:01:01 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 24 Mar 2008 01:01:01 +0200 Subject: [ofa-general] [PATCH] opensm/osm_partition.h: trivial - fixing pkey order in struct Message-ID: <47E6E12D.6060104@dev.mellanox.co.il> Pkey in partition struct should be in network order. Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_partition.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/opensm/include/opensm/osm_partition.h b/opensm/include/opensm/osm_partition.h index 539b1aa..70da27d 100644 --- a/opensm/include/opensm/osm_partition.h +++ b/opensm/include/opensm/osm_partition.h @@ -96,7 +96,7 @@ BEGIN_C_DECLS */ typedef struct _osm_prtn { cl_map_item_t map_item; - uint16_t pkey; + ib_net16_t pkey; uint8_t sl; cl_map_t full_guid_tbl; cl_map_t part_guid_tbl; -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Sun Mar 23 16:38:05 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 24 Mar 2008 01:38:05 +0200 Subject: [ofa-general] [PATCH] OpenSM release notes Message-ID: <47E6E9DD.6040009@dev.mellanox.co.il> Added ConnectX to list of supported devices, fixed FW versions to match OFED release notes, and removed device IDs (no reason for them to be there, OFED release notes doesn't have them) Please apply both to ofed_1_3 and master Signed-off-by: Yevgeny Kliteynik --- opensm/doc/opensm_release_notes-3.1.10.txt | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) diff --git a/opensm/doc/opensm_release_notes-3.1.10.txt b/opensm/doc/opensm_release_notes-3.1.10.txt index 9ebae1d..f5540cb 100644 --- a/opensm/doc/opensm_release_notes-3.1.10.txt +++ b/opensm/doc/opensm_release_notes-3.1.10.txt @@ -456,14 +456,15 @@ Table 3 - Qualified Devices and Corresponding Firmware ====================================================== Mellanox -Device | FW versions ---------|----------------------------------------------------------- -MT43132 | InfiniScale - fw-43132 5.2.0 (and later) -MT47396 | InfiniScale III - fw-47396 0.5.0 (and later) -MT23108 | InfiniHost - fw-23108 3.3.2 (and later) -MT25204 | InfiniHost III Lx - fw-25204 1.0.1i (and later) -MT25208 | InfiniHost III Ex (InfiniHost Mode) - fw-25208 4.6.2 (and later) -MT25208 | InfiniHost III Ex (MemFree Mode) - fw-25218 5.0.1 (and later) +Device | FW versions +------------------------------------|------------------------------- +InfiniScale | fw-43132 5.2.000 (and later) +InfiniScale III | fw-47396 0.5.000 (and later) +InfiniHost | fw-23108 3.5.000 (and later) +InfiniHost III Lx | fw-25204 1.2.000 (and later) +InfiniHost III Ex (InfiniHost Mode) | fw-25208 4.8.200 (and later) +InfiniHost III Ex (MemFree Mode) | fw-25218 5.3.000 (and later) +ConnectX IB | fw-25408 2.3.000 (and later) QLogic/PathScale Device | Note -- 1.5.1.4 From sashak at voltaire.com Sun Mar 23 18:59:48 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 01:59:48 +0000 Subject: [ofa-general] Re: [PATCH 1/5] infiniband-diags/scripts/set_nodedesc.sh: enhance to be able to set names other than hostname and to provide feedback on the names assigned In-Reply-To: <20080321155115.15e4d480.weiny2@llnl.gov> References: <20080321155115.15e4d480.weiny2@llnl.gov> Message-ID: <20080324015948.GB2102@sashak.voltaire.com> On 15:51 Fri 21 Mar , Ira Weiny wrote: > From 9553461b42d8dde7418b9eb3c07a8e78b58a32f1 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Mon, 17 Mar 2008 14:46:45 -0700 > Subject: [PATCH] infiniband-diags/scripts/set_nodedesc.sh: enhance to be able to set names other > > than hostname and to provide feedback on the names assigned. > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Sun Mar 23 19:01:49 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 02:01:49 +0000 Subject: [ofa-general] Re: [PATCH 2/5] Add an optional test utility 'ibsendtrap' In-Reply-To: <20080321155117.2b69452e.weiny2@llnl.gov> References: <20080321155117.2b69452e.weiny2@llnl.gov> Message-ID: <20080324020149.GC2102@sashak.voltaire.com> On 15:51 Fri 21 Mar , Ira Weiny wrote: > From 9294bf0041e88a077918619d5a93be318075cd74 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Fri, 21 Mar 2008 14:26:05 -0700 > Subject: [PATCH] Add an optional test utility 'ibsendtrap' > use --enable-test-utils to build this tool > > Signed-off-by: Ira K. Weiny Applied. Thanks. The comment is below. [snip...] > diff --git a/infiniband-diags/src/ibsendtrap.c b/infiniband-diags/src/ibsendtrap.c > new file mode 100644 > index 0000000..ae5069b > --- /dev/null > +++ b/infiniband-diags/src/ibsendtrap.c > @@ -0,0 +1,179 @@ > +/* > + * Copyright (c) 2008 Lawrence Livermore National Security > + * > + * Produced at Lawrence Livermore National Laboratory. > + * Written by Ira Weiny . > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +#include > +#include There should be also stdlib.h for exit() and string.h for strtoul(). I added this. Sasha From sashak at voltaire.com Sun Mar 23 19:03:22 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 02:03:22 +0000 Subject: [ofa-general] Re: [PATCH 3/5] Add mcm_rereg_test to test-utils option. In-Reply-To: <20080321155119.651d99f7.weiny2@llnl.gov> References: <20080321155119.651d99f7.weiny2@llnl.gov> Message-ID: <20080324020322.GD2102@sashak.voltaire.com> On 15:51 Fri 21 Mar , Ira Weiny wrote: > From 60a6563c1f726137a188a024481f25cbab9b140b Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Fri, 21 Mar 2008 14:44:31 -0700 > Subject: [PATCH] Add mcm_rereg_test to test-utils option. > Fix mcm_rereg_test.c to use ibmad_gid_t > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Sun Mar 23 19:08:09 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 02:08:09 +0000 Subject: [ofa-general] Re: [PATCH 4/5] opensm/opensm/osm_trap_rcv.c: respond to new trap 144 node description update flag In-Reply-To: <20080321155122.53262251.weiny2@llnl.gov> References: <20080321155122.53262251.weiny2@llnl.gov> Message-ID: <20080324020809.GE2102@sashak.voltaire.com> On 15:51 Fri 21 Mar , Ira Weiny wrote: > From 72f64c6eb0b82ea7dc8980733eb41660e2f40b1a Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 20 Mar 2008 17:59:01 -0700 > Subject: [PATCH] opensm/opensm/osm_trap_rcv.c: respond to new trap 144 node description update flag > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Sun Mar 23 19:14:13 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 02:14:13 +0000 Subject: [ofa-general] Re: [PATCH 5/5] Move extern declarations for node description processing to a new header file. In-Reply-To: <20080321155125.170f629c.weiny2@llnl.gov> References: <20080321155125.170f629c.weiny2@llnl.gov> Message-ID: <20080324021413.GF2102@sashak.voltaire.com> Hi Ira, On 15:51 Fri 21 Mar , Ira Weiny wrote: > From 93479e6cac589167f79f09d42900994e947c90f4 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Fri, 21 Mar 2008 15:23:16 -0700 > Subject: [PATCH] Move extern declarations for node description processing to a new header file. > > > Signed-off-by: Ira K. Weiny > --- > opensm/include/opensm/osm_node_desc.h | 54 +++++++++++++++++++++++++++++++++ > opensm/opensm/osm_node_desc_rcv.c | 27 ++++++++++++++++ > opensm/opensm/osm_node_info_rcv.c | 26 ---------------- > opensm/opensm/osm_sm.c | 2 +- > opensm/opensm/osm_trap_rcv.c | 3 +- > 5 files changed, 83 insertions(+), 29 deletions(-) > create mode 100644 opensm/include/opensm/osm_node_desc.h > > diff --git a/opensm/include/opensm/osm_node_desc.h b/opensm/include/opensm/osm_node_desc.h > new file mode 100644 > index 0000000..d86f6ba > --- /dev/null > +++ b/opensm/include/opensm/osm_node_desc.h When new file is added it should be also added to EXTRA_DIST list in include/Makefile.am so 'make dist' will work. > @@ -0,0 +1,54 @@ > +/* > + * Copyright (c) 2008 Lawrence Livermore National Security > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +#ifndef _OSM_NODE_DESC_H_ > +#define _OSM_NODE_DESC_H_ > + > +#include > +#include > + > +#ifdef __cplusplus > +# define BEGIN_C_DECLS extern "C" { > +# define END_C_DECLS } > +#else /* !__cplusplus */ > +# define BEGIN_C_DECLS > +# define END_C_DECLS > +#endif /* __cplusplus */ > + > +BEGIN_C_DECLS > + > +void osm_nd_rcv_process(void *context, void *data); > +void osm_req_get_node_desc(osm_sm_t * sm, osm_physp_t *p_physp); > + > +END_C_DECLS > +#endif /* _OSM_NODE_DESC_H_ */ > diff --git a/opensm/opensm/osm_node_desc_rcv.c b/opensm/opensm/osm_node_desc_rcv.c > index 4a22aab..a7266d5 100644 > --- a/opensm/opensm/osm_node_desc_rcv.c > +++ b/opensm/opensm/osm_node_desc_rcv.c > @@ -134,3 +134,30 @@ void osm_nd_rcv_process(IN void *context, IN void *data) > CL_PLOCK_RELEASE(sm->p_lock); > OSM_LOG_EXIT(sm->p_log); > } > + > +/********************************************************************** > + The plock must be held before calling this function. > +**********************************************************************/ > +void > +osm_req_get_node_desc(IN osm_sm_t * sm, > + osm_physp_t *p_physp) > +{ > + ib_api_status_t status = IB_SUCCESS; > + osm_madw_context_t context; > + > + OSM_LOG_ENTER(sm->p_log); > + > + context.nd_context.node_guid = > + osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); > + > + status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), > + IB_MAD_ATTR_NODE_DESC, > + 0, CL_DISP_MSGID_NONE, &context); > + if (status != IB_SUCCESS) > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " > + "Failure initiating NodeDescription request (%s)\n", > + ib_get_err_str(status)); > + > + OSM_LOG_EXIT(sm->p_log); > +} > + > diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c > index 6818e05..9a3eb7b 100644 > --- a/opensm/opensm/osm_node_info_rcv.c > +++ b/opensm/opensm/osm_node_info_rcv.c > @@ -306,32 +306,6 @@ __osm_ni_rcv_process_new_node(IN osm_sm_t * sm, > /********************************************************************** > The plock must be held before calling this function. > **********************************************************************/ > -void > -osm_req_get_node_desc(IN osm_sm_t * sm, > - osm_physp_t *p_physp) > -{ > - ib_api_status_t status = IB_SUCCESS; > - osm_madw_context_t context; > - > - OSM_LOG_ENTER(sm->p_log); > - > - context.nd_context.node_guid = > - osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); > - > - status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), > - IB_MAD_ATTR_NODE_DESC, > - 0, CL_DISP_MSGID_NONE, &context); > - if (status != IB_SUCCESS) > - OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " > - "Failure initiating NodeDescription request (%s)\n", > - ib_get_err_str(status)); > - > - OSM_LOG_EXIT(sm->p_log); > -} > - > -/********************************************************************** > - The plock must be held before calling this function. > -**********************************************************************/ Do we need this move? > static void > __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, > IN osm_node_t * const p_node, > diff --git a/opensm/opensm/osm_sm.c b/opensm/opensm/osm_sm.c > index 32525ba..69aafb4 100644 > --- a/opensm/opensm/osm_sm.c > +++ b/opensm/opensm/osm_sm.c > @@ -64,12 +64,12 @@ > #include > #include > #include > +#include > > #define OSM_SM_INITIAL_TID_VALUE 0x1233 > > extern void osm_lft_rcv_process(IN void *context, IN void *data); > extern void osm_mft_rcv_process(IN void *context, IN void *data); > -extern void osm_nd_rcv_process(IN void *context, IN void *data); > extern void osm_ni_rcv_process(IN void *context, IN void *data); > extern void osm_pkey_rcv_process(IN void *context, IN void *data); > extern void osm_pi_rcv_process(IN void *context, IN void *data); Isn't it would be better to put all those declarations to n one place (let's say osm_sm.h) instead of creating new - one function prototype header file? Sasha From sashak at voltaire.com Sun Mar 23 19:15:04 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 02:15:04 +0000 Subject: [ofa-general] Re: [PATCH] OpenSM release notes: Add in new QLogic HCAs In-Reply-To: <1206196552.8099.394.camel@hrosenstock-ws.xsigo.com> References: <1206196552.8099.394.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080324021504.GG2102@sashak.voltaire.com> On 07:35 Sat 22 Mar , Hal Rosenstock wrote: > OpenSM release notes: Add in new QLogic HCAs > > Please apply to both master and OFED 1.3 > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From nacidoutdoorszur at idoutdoors.com Sun Mar 23 20:58:07 2008 From: nacidoutdoorszur at idoutdoors.com (Gino Beasley) Date: Mon, 24 Mar 2008 09:28:07 +0530 Subject: [ofa-general] Legal software sales Message-ID: <125947006.29347505842689@idoutdoors.com> Our aim is to guarantee PC and Mac lawful software and computer solutions of low price for anyone. Whether you are a corporate customer, a proprietor of small business, or shopping for your own home personal computer, we think that we'll assist you. TAKE A LOOK AT ALL OUR PRODUCTS! http://elizabethchamblissks743.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From cnyunblku at cani.every1.net Sun Mar 23 23:26:45 2008 From: cnyunblku at cani.every1.net (amanda) Date: Sun, 23 Mar 2008 22:26:45 -0800 Subject: [ofa-general] hello from amanda Message-ID: <2555734_@TLZ3908669_@TLZ> Hi My name is amanda. I found your email on that dating site. I also love sex on the side. I have a loving partner but he is working 16 hours a day and we have sex only once a week :( If you are interested and wanna see my pictures just email me at camanda81 at thegolovaworld.com Don`t reply, use the email above (my boyfriend doesn`t know about that email!) From cnyunblku at cani.every1.net Sun Mar 23 23:26:45 2008 From: cnyunblku at cani.every1.net (amanda) Date: Sun, 23 Mar 2008 22:26:45 -0800 Subject: [ofa-general] hello from amanda Message-ID: <2555734_@TLZ3908669_@TLZ> Hi My name is amanda. I found your email on that dating site. I also love sex on the side. I have a loving partner but he is working 16 hours a day and we have sex only once a week :( If you are interested and wanna see my pictures just email me at camanda81 at thegolovaworld.com Don`t reply, use the email above (my boyfriend doesn`t know about that email!) From tripp at stata-press.com Sun Mar 23 22:55:03 2008 From: tripp at stata-press.com (EuroSoftware) Date: Mon, 24 Mar 2008 11:25:03 +0530 Subject: [ofa-general] Die laecherlichen Preise fuer legale Software Message-ID: <495575732.33156971533414@stata-press.com> Suchen Sie nach der Software? Mochten sie momentan bekommen? Das ist das! Nur bezahlen und auslasten. Die Programmen sind auf allen europaischen Sprachen uberlassen und fur Windows und Macintosh vorherbestimmthttp://geocities.com/lindacherry71/Sie stellen jedes Programm leicht auf mit der Hilfe der professionellen Konsultation des Anwenderdienstes. Wenn Sie Fragen haben, bekommen Sie schnelle Antworte. Die Ruckzahlung ist moglich. Sie kaufen die Software, sie funktionieren ausgezeichnethttp://geocities.com/lindacherry71/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Sun Mar 23 23:14:21 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 06:14:21 +0000 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <47E67B1F.8050102@dev.mellanox.co.il> References: <47E67B1F.8050102@dev.mellanox.co.il> Message-ID: <20080324061421.GH2102@sashak.voltaire.com> Hi Yevgeny, On 17:45 Sun 23 Mar , Yevgeny Kliteynik wrote: > > QoS manager should set the right SL in the IPoIB broadcast groups > to enforce the right SL for IPoIB traffic. > > At start it seemed to me that it won't work in the following scenario: > - IPoIB partition created > - IPoIB mcast group created > - IPoIB connectivity was established between two ports > - SL in the QoS policy file was changed > - SM is doing heavy sweep, and setting new SL in the > partition and mcast group, but the nodes already have > connectivity, so they will continue to transmit on the > previous SL > But then I realized that this scenario will be a problem for any > communication (not only IPoIB), because as long as clients are not > issuing new PR request, they will have the old parameters that won't > be affected by the changes in QoS policy. > > This patch is for ofed_1_3 branch. Is it really OFED 1.3 materials and it is not enough (for already stabilized version) to have just a warning about SL value mismatch between partition and QoS managers? > Please review and let me know what you think. > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/opensm/osm_qos_policy.c | 39 +++++++++++++++++++++++++++++++++++++++ > 1 files changed, 39 insertions(+), 0 deletions(-) > > diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c > index bde1e7e..4586a77 100644 > --- a/opensm/opensm/osm_qos_policy.c > +++ b/opensm/opensm/osm_qos_policy.c > @@ -920,6 +920,13 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, > > if (p_qos_match_rule->p_qos_level->sl_set && > p_prtn->sl != p_qos_match_rule->p_qos_level->sl) { > + uint8_t sl; > + uint32_t flow; > + uint8_t hop; > + cl_qmap_t * p_mlid_tbl; > + osm_mgrp_t * p_mgrp; > + osm_mgrp_t * p_next_mgrp; > + > /* overriding partition's SL */ > osm_log(p_log, > OSM_LOG_ERROR, > @@ -931,6 +938,38 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, > p_prtn->sl, > p_qos_match_rule->p_qos_level->sl); > p_prtn->sl = p_qos_match_rule->p_qos_level->sl; > + > + /* If this partition is an IPoIB partition, > + there should be a matching MCast group. > + Fix this group's SL too */ > + > + p_mlid_tbl = &p_qos_policy->p_subn->mgrp_mlid_tbl; > + p_next_mgrp = (osm_mgrp_t *) cl_qmap_head(p_mlid_tbl); > + while (p_next_mgrp != (osm_mgrp_t *) > + cl_qmap_end(p_mlid_tbl)) { Instead of looping here you could just keep osm_mgrp_t reference as field in partition structure. > + p_mgrp = p_next_mgrp; > + p_next_mgrp = (osm_mgrp_t *) > + cl_qmap_next(&p_mgrp->map_item); > + if (!p_mgrp->well_known) > + continue; > + if ((cl_ntoh16(p_mgrp->mcmember_rec.pkey) & 0x7fff) != > + (cl_ntoh16(pkey) & 0x7fff)) > + continue; > + ib_member_get_sl_flow_hop( > + p_mgrp->mcmember_rec.sl_flow_hop, > + &sl, &flow, &hop); > + if (sl != p_prtn->sl) { > + osm_log(p_log, OSM_LOG_DEBUG, > + "osm_qos_policy_validate: " > + "Updating MCGroup (MLID 0x%04x) SL to " > + "match partition SL %u\n", > + cl_hton16(p_mgrp->mcmember_rec.mlid), > + p_prtn->sl); > + p_mgrp->mcmember_rec.sl_flow_hop = > + ib_member_set_sl_flow_hop( > + p_prtn->sl, flow, hop); > + } > + } > } > } > } What about to reduce a number of flow nesting? We have a functions in C. Sasha From sashak at voltaire.com Sun Mar 23 23:17:31 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 06:17:31 +0000 Subject: [ofa-general] Re: [PATCH] opensm/osm_partition.h: trivial - fixing pkey order in struct In-Reply-To: <47E6E12D.6060104@dev.mellanox.co.il> References: <47E6E12D.6060104@dev.mellanox.co.il> Message-ID: <20080324061731.GJ2102@sashak.voltaire.com> On 01:01 Mon 24 Mar , Yevgeny Kliteynik wrote: > Pkey in partition struct should be in network order. > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From sashak at voltaire.com Sun Mar 23 23:18:01 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 06:18:01 +0000 Subject: [ofa-general] Re: [PATCH] OpenSM release notes In-Reply-To: <47E6E9DD.6040009@dev.mellanox.co.il> References: <47E6E9DD.6040009@dev.mellanox.co.il> Message-ID: <20080324061801.GK2102@sashak.voltaire.com> On 01:38 Mon 24 Mar , Yevgeny Kliteynik wrote: > Added ConnectX to list of supported devices, fixed FW versions > to match OFED release notes, and removed device IDs (no reason > for them to be there, OFED release notes doesn't have them) > > Please apply both to ofed_1_3 and master > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From rdreier at cisco.com Sun Mar 23 23:33:18 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 23 Mar 2008 23:33:18 -0700 Subject: [ofa-general] [PATCH] libnes: Fix "make dist" Message-ID: Need to include src/nes_umain.h in the generated tar file to create a buildable distribution. Otherwise "make distcheck" fails with configure: error: cannot find sources (src/nes_umain.h) in .. Signed-off-by: Roland Dreier --- diff --git a/Makefile.am b/Makefile.am index 73aea71..a5fdfc2 100644 --- a/Makefile.am +++ b/Makefile.am @@ -19,7 +19,7 @@ nesconf_DATA = nes.driver #DEBIAN = debian/changelog debian/compat debian/control debian/copyright \ # debian/libnes1.install debian/libnes-dev.install debian/rules -EXTRA_DIST = src/nes-abi.h \ +EXTRA_DIST = src/nes-abi.h src/nes_umain.h \ src/nes.map libnes.spec.in nes.driver $(DEBIAN) dist-hook: libnes.spec From kliteyn at dev.mellanox.co.il Mon Mar 24 00:09:51 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 24 Mar 2008 09:09:51 +0200 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <20080324061421.GH2102@sashak.voltaire.com> References: <47E67B1F.8050102@dev.mellanox.co.il> <20080324061421.GH2102@sashak.voltaire.com> Message-ID: <47E753BF.9070505@dev.mellanox.co.il> Hi Sasha, Sasha Khapyorsky wrote: > Hi Yevgeny, > > On 17:45 Sun 23 Mar , Yevgeny Kliteynik wrote: >> QoS manager should set the right SL in the IPoIB broadcast groups >> to enforce the right SL for IPoIB traffic. >> >> At start it seemed to me that it won't work in the following scenario: >> - IPoIB partition created >> - IPoIB mcast group created >> - IPoIB connectivity was established between two ports >> - SL in the QoS policy file was changed >> - SM is doing heavy sweep, and setting new SL in the >> partition and mcast group, but the nodes already have >> connectivity, so they will continue to transmit on the >> previous SL >> But then I realized that this scenario will be a problem for any >> communication (not only IPoIB), because as long as clients are not >> issuing new PR request, they will have the old parameters that won't >> be affected by the changes in QoS policy. >> >> This patch is for ofed_1_3 branch. > > Is it really OFED 1.3 materials and it is not enough (for already > stabilized version) to have just a warning about SL value mismatch > between partition and QoS managers? QoS annex support in OFED 1.3 is defined as "experimental", or "technology preview", if you wish. I really can't say that QoS annex support is "stabilized" - the first user ever tried it (besides be) found a bug :) In light of the above, I do think that it's very much relevant for OFED 1.3. W/o this fix one of the main features (SL for IPoIB) in the QoS policy is broken. >> Please review and let me know what you think. >> >> Signed-off-by: Yevgeny Kliteynik >> --- >> opensm/opensm/osm_qos_policy.c | 39 +++++++++++++++++++++++++++++++++++++++ >> 1 files changed, 39 insertions(+), 0 deletions(-) >> >> diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c >> index bde1e7e..4586a77 100644 >> --- a/opensm/opensm/osm_qos_policy.c >> +++ b/opensm/opensm/osm_qos_policy.c >> @@ -920,6 +920,13 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, >> >> if (p_qos_match_rule->p_qos_level->sl_set && >> p_prtn->sl != p_qos_match_rule->p_qos_level->sl) { >> + uint8_t sl; >> + uint32_t flow; >> + uint8_t hop; >> + cl_qmap_t * p_mlid_tbl; >> + osm_mgrp_t * p_mgrp; >> + osm_mgrp_t * p_next_mgrp; >> + >> /* overriding partition's SL */ >> osm_log(p_log, >> OSM_LOG_ERROR, >> @@ -931,6 +938,38 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, >> p_prtn->sl, >> p_qos_match_rule->p_qos_level->sl); >> p_prtn->sl = p_qos_match_rule->p_qos_level->sl; >> + >> + /* If this partition is an IPoIB partition, >> + there should be a matching MCast group. >> + Fix this group's SL too */ >> + >> + p_mlid_tbl = &p_qos_policy->p_subn->mgrp_mlid_tbl; >> + p_next_mgrp = (osm_mgrp_t *) cl_qmap_head(p_mlid_tbl); >> + while (p_next_mgrp != (osm_mgrp_t *) >> + cl_qmap_end(p_mlid_tbl)) { > > Instead of looping here you could just keep osm_mgrp_t reference as > field in partition structure. Sure, that would be better. This is what I did initially, but then reworked the change to be as "local" as possible, because as I mentioned, this patch was for OFED 1.3 only - for master I intended to do it with reference in partition structure. >> + p_mgrp = p_next_mgrp; >> + p_next_mgrp = (osm_mgrp_t *) >> + cl_qmap_next(&p_mgrp->map_item); >> + if (!p_mgrp->well_known) >> + continue; >> + if ((cl_ntoh16(p_mgrp->mcmember_rec.pkey) & 0x7fff) != >> + (cl_ntoh16(pkey) & 0x7fff)) >> + continue; >> + ib_member_get_sl_flow_hop( >> + p_mgrp->mcmember_rec.sl_flow_hop, >> + &sl, &flow, &hop); >> + if (sl != p_prtn->sl) { >> + osm_log(p_log, OSM_LOG_DEBUG, >> + "osm_qos_policy_validate: " >> + "Updating MCGroup (MLID 0x%04x) SL to " >> + "match partition SL %u\n", >> + cl_hton16(p_mgrp->mcmember_rec.mlid), >> + p_prtn->sl); >> + p_mgrp->mcmember_rec.sl_flow_hop = >> + ib_member_set_sl_flow_hop( >> + p_prtn->sl, flow, hop); >> + } >> + } >> } >> } >> } > > What about to reduce a number of flow nesting? We have a functions in C. Gee, man.. Completely forgot about that... Like I said, I reworked the change to be as local as possible (although it turned out to be quite ugly). Anyway, I'll repost the improved patch. -- Yevgeny > Sasha > From ogerlitz at voltaire.com Mon Mar 24 00:21:02 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 24 Mar 2008 09:21:02 +0200 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos Message-ID: <47E7565E.1040206@voltaire.com> Hi Yevgeny, I wanted to clarify to/with you some issues re QoS/IPoIB in openSM. Looking in the documents provided by the ofed-docs package: from QoS_in_OFED.txt > 147 > ============================================================================== > 148 5. IPoIB > 149 > ============================================================================== > 151 IPoIB queries the SA for its broadcast group information. > 152 It provides the broadcast group SL, MTU, and RATE in every following > 153 PathRecord query performed when a new UDAV is needed by IPoIB. This is almost all wrong, sorry... the way it works in the Linux ipoib driver, is the following: address vectors for unicast traffic, both datagram and connected modes, are created through the result of an SA path query with this comp mask where numb_path is set to one, and the traffic class is the one returned by the SA to the broadcast group, see below. address vectors for multicast senders only (clients) are created through the result of an SA MC member query (join) with this comp mask address vectors for multicast receivers (that might send as well) are created through the result of an SA MC member query (join) with this comp mask For all but the broadcast group, the following bits are also set in the comp mask where they are all derived from the broadcast group, where in the broadcast case, the SA has assigned values for them. Bottom line, for path queries the --requested-- SL is not provided to the SM and there's a difference between the info provided for sender joins to server joins where only for the latter the driver asks for specific . from QoS_management_in_opensm.txt > 353 6.1 IPoIB > 354 IPoIB query is matched by PKey. Default PKey for IPoIB partition > is 0x7fff, so > 355 the following three match rules are equivalent: > 356 > 357 ipoib : > 358 ipoib, 0x7fff : > 359 any, pkey 0x7fff : I see, so ipoib is just an acronym for a path query with the default pkey. This creates confusion and I am not sure its well defined, at least in my case when I put my hands on QoS testing, I couldn't guess that "ipoib" == "pkey 0x7fff". For example, is it correct that "ipoib, pkey 0x8001 : " is not valid rule? Or. From ogerlitz at voltaire.com Mon Mar 24 00:31:20 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 24 Mar 2008 09:31:20 +0200 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <47E67B1F.8050102@dev.mellanox.co.il> References: <47E67B1F.8050102@dev.mellanox.co.il> Message-ID: <47E758C8.3060307@voltaire.com> Yevgeny Kliteynik wrote: > Hi Sasha, > > QoS manager should set the right SL in the IPoIB broadcast groups > to enforce the right SL for IPoIB traffic. > > At start it seemed to me that it won't work in the following scenario: > - IPoIB partition created > - IPoIB mcast group created > - IPoIB connectivity was established between two ports > - SL in the QoS policy file was changed > - SM is doing heavy sweep, and setting new SL in the > partition and mcast group, but the nodes already have > connectivity, so they will continue to transmit on the > previous SL > But then I realized that this scenario will be a problem for any > communication (not only IPoIB), because as long as clients are not > issuing new PR request, they will have the old parameters that won't > be affected by the changes in QoS policy. > Let me understand, when the user changes the QoS policy file, openSM reads it on the fly and creates new rules? > This patch is for ofed_1_3 branch. > > Please review and let me know what you think. > > Signed-off-by: Yevgeny Kliteynik > Is this the change log comment? please explain what the patch is doing, I understand that the text above is the problem statement, not the solution..., correct? Or. From kliteyn at dev.mellanox.co.il Mon Mar 24 00:58:00 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 24 Mar 2008 09:58:00 +0200 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <47E758C8.3060307@voltaire.com> References: <47E67B1F.8050102@dev.mellanox.co.il> <47E758C8.3060307@voltaire.com> Message-ID: <47E75F08.3040007@dev.mellanox.co.il> Or Gerlitz wrote: > Yevgeny Kliteynik wrote: >> Hi Sasha, >> >> QoS manager should set the right SL in the IPoIB broadcast groups >> to enforce the right SL for IPoIB traffic. >> >> At start it seemed to me that it won't work in the following scenario: >> - IPoIB partition created >> - IPoIB mcast group created >> - IPoIB connectivity was established between two ports >> - SL in the QoS policy file was changed >> - SM is doing heavy sweep, and setting new SL in the >> partition and mcast group, but the nodes already have >> connectivity, so they will continue to transmit on the >> previous SL >> But then I realized that this scenario will be a problem for any >> communication (not only IPoIB), because as long as clients are not >> issuing new PR request, they will have the old parameters that won't >> be affected by the changes in QoS policy. >> > Let me understand, when the user changes the QoS policy file, openSM > reads it on the fly and creates new rules? OpenSM re-reads the policy file every heavy sweep and creates new rules. >> This patch is for ofed_1_3 branch. >> >> Please review and let me know what you think. >> >> Signed-off-by: Yevgeny Kliteynik >> > Is this the change log comment? please explain what the patch is doing, > I understand that the text above is the problem statement, not the > solution..., correct? The explanation of what the patch is in the beginning of the mail: "QoS manager should set the right SL in the IPoIB broadcast groups to enforce the right SL for IPoIB traffic." QoS manager finds the IPoIB broadcast group that was created for pkey that is specified in the qos policy file, and updates the broadcast (multicast) group's SL to match the SL that was specified for this partition in the qos policy file. -- Yevgeny > Or. > > From kliteyn at mellanox.co.il Mon Mar 24 01:31:08 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 24 Mar 2008 10:31:08 +0200 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos In-Reply-To: <47E7565E.1040206@voltaire.com> References: <47E7565E.1040206@voltaire.com> Message-ID: <47E766CC.9070905@mellanox.co.il> Hi Or, Or Gerlitz wrote: > Hi Yevgeny, > > I wanted to clarify to/with you some issues re QoS/IPoIB in openSM. > Looking in the documents provided by the ofed-docs package: > > from QoS_in_OFED.txt >> 147 >> ============================================================================== >> >> 148 5. IPoIB >> 149 >> ============================================================================== >> >> 151 IPoIB queries the SA for its broadcast group information. >> 152 It provides the broadcast group SL, MTU, and RATE in every >> following >> 153 PathRecord query performed when a new UDAV is needed by IPoIB. > This is almost all wrong, sorry... the way it works in the Linux ipoib > driver, is the following: > > address vectors for unicast traffic, both datagram and connected > modes, are created through the result of an SA path query with this > comp mask where numb_path > is set to one, and the traffic class is the one returned by the SA to > the broadcast group, see below. > > address vectors for multicast senders only (clients) are created > through the result of an SA MC member query (join) with this comp mask > > address vectors for multicast receivers (that might send as well) are > created through the result of an SA MC member query (join) with this > comp mask > > For all but the broadcast group, the following bits are also set in > the comp mask > FLOW_LABEL, HOP_LIMIT> where they are all derived from the broadcast > group, where in the broadcast case, the SA has assigned values for them. > > Bottom line, for path queries the --requested-- SL is not provided to > the SM and there's a difference between the info provided for sender > joins to server joins where only for the latter the driver asks for > specific . OK. Can you rephrase what you said in two sentences? Something short to replace the existing (and wrong, as you pointed out) three lines: 151 IPoIB queries the SA for its broadcast group information. 152 It provides the broadcast group SL, MTU, and RATE in every following 153 PathRecord query performed when a new UDAV is needed by IPoIB. > > from QoS_management_in_opensm.txt >> 353 6.1 IPoIB >> 354 IPoIB query is matched by PKey. Default PKey for IPoIB partition >> is 0x7fff, so >> 355 the following three match rules are equivalent: >> 356 >> 357 ipoib : >> 358 ipoib, 0x7fff : >> 359 any, pkey 0x7fff : > I see, so ipoib is just an acronym for a path query with the default > pkey. "ipoib" w/o any pkey is an acronym for a path query with the default pkey, but this rule also tells OpenSM to set the partition's SL to , and to set the IPoIB mcast group's SL to . Actually, line 358 is wrong (I'll update it). It should be as follows: 357 ipoib : 358 ipoib, pkey 0x7fff : 359 any, pkey 0x7fff : These three lines are equivalent. > > This creates confusion and I am not sure its well defined, at least in > my case when I put my hands on QoS testing, I couldn't guess that > "ipoib" == "pkey 0x7fff". For example, is it correct that "ipoib, pkey > 0x8001 : " is not valid rule? This rule is also valid, providing that you actually have a partition with pkey 0x8001 configured in the partition cfg file. -- Yevgeny > > Or. > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From sashak at voltaire.com Mon Mar 24 01:54:07 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 08:54:07 +0000 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <47E753BF.9070505@dev.mellanox.co.il> References: <47E67B1F.8050102@dev.mellanox.co.il> <20080324061421.GH2102@sashak.voltaire.com> <47E753BF.9070505@dev.mellanox.co.il> Message-ID: <20080324085407.GA595@sashak.voltaire.com> On 09:09 Mon 24 Mar , Yevgeny Kliteynik wrote: > > QoS annex support in OFED 1.3 is defined as "experimental", > or "technology preview", if you wish. Correct. Obviously this doesn't mean that development should continue on OFED 1.3 branch. > I really can't say that QoS annex support is "stabilized" - "stabilized" is about OFED 1.3 > the first user ever tried it (besides be) found a bug :) > In light of the above, I do think that it's very much relevant > for OFED 1.3. W/o this fix one of the main features (SL for IPoIB) > in the QoS policy is broken. An user will need to synchronize partition and QoS configs (the same still be true for QoS low level parameters). This is downside of course. But I don't think it so critical assuming QoS annex support is experimental anyway. Sasha From sashak at voltaire.com Mon Mar 24 02:36:02 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 09:36:02 +0000 Subject: [ofa-general] [PATCH] opensm: updn/connect_roots: preserve connectivity to root nodes Message-ID: <20080324093602.GC595@sashak.voltaire.com> Option '--connect_roots' was added in order to preserve connectivity between root nodes when Up/Down routing algorithm is used (by default Up/Down leaves root nodes not connected). The problems with this option could appear with some topologies when some root's (root A) neighbor switch has valid up/down path (which could not be shortest path due to up/down restriction) to another root node (root B) in a fabric which cross this root A node and at the same time root A has shortest path (preserved with --connect_roots option) to the root B which crosses its neighbor. In this case we will have single routing loop which is not good. (actually it causes OpenSM to crash during multicast spanning trees building). Solution is to preserve to root nodes shortest paths (on when --connect_root is specified) for all switches, not only other roots as it is today. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_updn.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/opensm/opensm/osm_ucast_updn.c b/opensm/opensm/osm_ucast_updn.c index 95bd946..0dc9f1f 100644 --- a/opensm/opensm/osm_ucast_updn.c +++ b/opensm/opensm/osm_ucast_updn.c @@ -454,8 +454,7 @@ static int __osm_subn_set_up_down_min_hop_table(IN updn_t * p_updn) p_sw = p_next_sw; p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); /* Clear Min Hop Table */ - if (p_subn->opt.connect_roots - && !((struct updn_node *)p_sw->priv)->rank) + if (p_subn->opt.connect_roots) updn_clear_root_hops(p_updn, p_sw); else osm_switch_clear_hops(p_sw); -- 1.5.4.rc2.60.gb2e62 From sashak at voltaire.com Mon Mar 24 02:43:06 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 09:43:06 +0000 Subject: [ofa-general] [PATCH] opensm/osm_mcast_mgr: limit spanning tree creation recursion to max hops (64) In-Reply-To: <20080324093602.GC595@sashak.voltaire.com> References: <20080324093602.GC595@sashak.voltaire.com> Message-ID: <20080324094306.GD595@sashak.voltaire.com> This limits spanning tree building recursion to max hops enabled in IB (64). Without it OpenSM crashes in endless recursion (__osm_mcast_mgr_branch()) when lid matrices (min hop tables) are not configured properly or broken for some reason (as example - using old version of '--connect_roots' option with up/down on some topologies). Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_mcast_mgr.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_mcast_mgr.c b/opensm/opensm/osm_mcast_mgr.c index fe4cfbf..683a16d 100644 --- a/opensm/opensm/osm_mcast_mgr.c +++ b/opensm/opensm/osm_mcast_mgr.c @@ -569,6 +569,14 @@ static osm_mtree_node_t *__osm_mcast_mgr_branch(osm_sm_t * sm, depth++; + if (depth >= 64) { + OSM_LOG(sm->p_log, OSM_LOG_ERROR, + "Maximal hops number is reached MLID 0x%x." + " Break processing.", mlid_ho); + __osm_mcast_mgr_purge_list(sm, p_list); + goto Exit; + } + if (depth > *p_max_depth) { CL_ASSERT(depth == *p_max_depth + 1); *p_max_depth = depth; -- 1.5.4.rc2.60.gb2e62 From leeds at ata-engineering.co.uk Mon Mar 24 00:56:57 2008 From: leeds at ata-engineering.co.uk (dimitrou lotfi) Date: Mon, 24 Mar 2008 07:56:57 +0000 Subject: [ofa-general] Shocking porno Britney Spears Message-ID: <000901c88d93$023da815$34798eac@xbdyb> #wXgFvLNicole Kidman Full presentation. #tEyjeLThe mpeg4 is Full! #XgFvLvOnly 1 day trial - get this Kick-up mpeg4 now! #tEyjeL Download it now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at dev.mellanox.co.il Mon Mar 24 03:33:57 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 24 Mar 2008 12:33:57 +0200 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <20080324085407.GA595@sashak.voltaire.com> References: <47E67B1F.8050102@dev.mellanox.co.il> <20080324061421.GH2102@sashak.voltaire.com> <47E753BF.9070505@dev.mellanox.co.il> <20080324085407.GA595@sashak.voltaire.com> Message-ID: <47E78395.9020604@dev.mellanox.co.il> OK, I'll repost the patch for master only. For OFED 1.3 the partition SL configuration should be done through partition config file only. -- Yevgeny Sasha Khapyorsky wrote: > On 09:09 Mon 24 Mar , Yevgeny Kliteynik wrote: >> QoS annex support in OFED 1.3 is defined as "experimental", >> or "technology preview", if you wish. > > Correct. Obviously this doesn't mean that development should continue on > OFED 1.3 branch. > >> I really can't say that QoS annex support is "stabilized" - > > "stabilized" is about OFED 1.3 > >> the first user ever tried it (besides be) found a bug :) >> In light of the above, I do think that it's very much relevant >> for OFED 1.3. W/o this fix one of the main features (SL for IPoIB) >> in the QoS policy is broken. > > An user will need to synchronize partition and QoS configs (the same > still be true for QoS low level parameters). This is downside of course. > But I don't think it so critical assuming QoS annex support is > experimental anyway. > > Sasha > From ogerlitz at voltaire.com Mon Mar 24 03:45:43 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 24 Mar 2008 12:45:43 +0200 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos In-Reply-To: <47E766CC.9070905@mellanox.co.il> References: <47E7565E.1040206@voltaire.com> <47E766CC.9070905@mellanox.co.il> Message-ID: <47E78657.60105@voltaire.com> Yevgeny Kliteynik wrote: > OK. Can you rephrase what you said in two sentences? Something short > to replace the existing (and wrong, as you pointed out) three lines: > > 151 IPoIB queries the SA for its broadcast group information. > 152 It provides the broadcast group SL, MTU, and RATE in every > following > 153 PathRecord query performed when a new UDAV is needed by IPoIB. Will try to send something later this week. This document describes the QoS related architecture of Linux IB stack, hence do you agree that the file name would be changed to reflect that? ie that the word "ofed" will not appear there? >> This creates confusion and I am not sure its well defined, at least >> in my case when I put my hands on QoS testing, I couldn't guess that >> "ipoib" == "pkey 0x7fff". For example, is it correct that "ipoib, >> pkey 0x8001 : " is not valid rule? > > This rule is also valid, providing that you actually have a partition > with pkey 0x8001 configured in the partition cfg file. OK, so the rule is valid, but would you agree that the word "ipoib" adds nothing to the rule? Or. From nny at bcs.co.il Mon Mar 24 03:22:11 2008 From: nny at bcs.co.il (angeli ebenezer) Date: Mon, 24 Mar 2008 10:22:11 +0000 Subject: [ofa-general] 95% discount. Code #hBwD Message-ID: <000801c88da7$052b1e1c$7b19419d@ppkupjfl> Hi openib-general, be smart, buy your meds from the best supplier since 1997. http://www.google.com/pagead/iclk?sa=l&ai=fsYHYe&num=84875&adurl=http://nVKW.sugaronly.com albert peebles From sashak at voltaire.com Mon Mar 24 05:32:26 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 12:32:26 +0000 Subject: [ofa-general] OFED March 10 teleconfrance meeting summary In-Reply-To: <6C2C79E72C305246B504CBA17B5500C90380CC85@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C90380CC85@mtlexch01.mtl.com> Message-ID: <20080324123226.GE595@sashak.voltaire.com> On 16:28 Tue 11 Mar , Tziporet Koren wrote: > > OFED meeting summary: > ===================== > 1. Decide on fix release criteria and process (see below) > 2. OFED 1.4: > - Target date: Sep 2008 > - Will start with kernel 2.6.25 once it released > - Next meeting will be devoted to define the features set. In addition to this I would suggest to switch to tarball based distribution for userspace components with OFED 1.4 (as was discussed many times - maintainers will release tarballs, no ofed_1_4/package any more, etc). Sasha From bryan at ims-van.com Mon Mar 24 06:09:54 2008 From: bryan at ims-van.com (Jazmyne Davenport) Date: Mon, 24 Mar 2008 14:09:54 +0100 Subject: [ofa-general] Kaufen Sie Softwareloesungen billiger Message-ID: <01c88db8$bb8ae500$945a3897@bryan> Software range expansion-price downfallUnsere Software konnen Sie sofort downloaden. Einfach bezahlen und schon kann das runterladen beginnen. Unsere Programme funktionieren unter Windows und mit Macintosh und sind in allen europaeischen Sprachen verfugbar. Wir haben die guenstigsten Preise, aber es handelt sich natuerlich nur um Originalversionen, die voellig legal und im vollen Umfang sind. http://geocities.com/laurabarnes99/* Office Enterprise 2007: $79.95 * Windows XP Professional With SP2 Full Version: $59.95 * AutoCAD 2008: $129.95 * Office System Professional 2003 (5 Cds): $59.95 http://geocities.com/laurabarnes99/Bestellen Sie bei uns ohne Sorgen. Wir haben kompetente Supportmitarbeiter, die Ihnen bei der Installation weiterhelfen, wenn Sie Hilfe brauchen. Schnell und unverzueglich werden von uns Ihre Fragen beantwortet. Bei uns gibt es natuerlich auch Geld-Zurueck-Garantie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli at dev.mellanox.co.il Mon Mar 24 07:38:06 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 24 Mar 2008 16:38:06 +0200 Subject: [ofa-general] [PATCH 3/10] IB/core: Add LSO support In-Reply-To: <47E5FA6D.7070307@voltaire.com> References: <1205767431.25950.138.camel@mtls03> <47E5FA6D.7070307@voltaire.com> Message-ID: <1206369486.25950.289.camel@mtls03> On Sun, 2008-03-23 at 08:36 +0200, Or Gerlitz wrote: > > --- a/include/rdma/ib_verbs.h > > +++ b/include/rdma/ib_verbs.h > > @@ -411,6 +412,7 @@ enum ib_wc_opcode { > > IB_WC_COMP_SWAP, > > IB_WC_FETCH_ADD, > > IB_WC_BIND_MW, > > + IB_WC_LSO, > > > with IB_WC_LSO never being used over this patchset, can we just remove it? It is used in cq.c. I don't think we can remove it even though IPoIB does not use this completion type - other clients might want to. > > @@ -622,7 +624,8 @@ enum ib_wr_opcode { > > IB_WR_SEND_WITH_IMM, > > IB_WR_RDMA_READ, > > IB_WR_ATOMIC_CMP_AND_SWP, > > - IB_WR_ATOMIC_FETCH_AND_ADD > > + IB_WR_ATOMIC_FETCH_AND_ADD, > > + IB_WR_LSO > > }; > > > > enum ib_send_flags { > > @@ -630,7 +633,8 @@ enum ib_send_flags { > > IB_SEND_SIGNALED = (1<<1), > > IB_SEND_SOLICITED = (1<<2), > > IB_SEND_INLINE = (1<<3), > > - IB_SEND_IP_CSUM = (1<<4) > > + IB_SEND_IP_CSUM = (1<<4), > > + IB_SEND_UDP_LSO = (1<<5) > > }; > > > IB_SEND_UDP_LSO is never used in this patchset, I guess you wanted to > call it IB_SEND_TCP_LSO. Also how about using it in ipoib at the same > manner as the IB_SEND_IP_CSUM bit is? (ie OR it into the send flags of > the UD WR). With this in mind, I suggest that you remove the IB_WR_LSO. Actually I think of IB_SEND_UDP_LSO as a leftover from some other patch and IB_WR_LSO is just a WR that does LSO. It does not have "TCP" in its name in the same way as IB_WR_RDMA_READ does not have "RC" in it. > > > > struct ib_sge { > > @@ -660,6 +664,9 @@ struct ib_send_wr { > > } atomic; > > struct { > > struct ib_ah *ah; > > + void *header; > > + int hlen; > > + int mss; > > > Can you add shorting documentation for the new fields? Of course I can add - I just followed the "spirit" of other parts of the code where there is not description. Is this an exception in this regard? > > Or. > From eli at dev.mellanox.co.il Mon Mar 24 07:38:43 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 24 Mar 2008 16:38:43 +0200 Subject: [ofa-general] [PATCH 6/10] IB/mlx4: Add LSO support In-Reply-To: <47E5FC14.6070802@voltaire.com> References: <1205767443.25950.141.camel@mtls03> <47E5FC14.6070802@voltaire.com> Message-ID: <1206369523.25950.291.camel@mtls03> On Sun, 2008-03-23 at 08:43 +0200, Or Gerlitz wrote: > Eli Cohen wrote: > > Add LSO support to mlx4 driver such that it will be able > > to send SKBs passed from the driver which publish NETIF_TSO. > > > The patch is full with printk calls, please remove them. > Thanks, I'll remove them. From kliteyn at dev.mellanox.co.il Mon Mar 24 07:49:12 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 24 Mar 2008 16:49:12 +0200 Subject: [ofa-general] [PATCH] opensm/QoS: setting SL in the IPoIB MCast groups Message-ID: <47E7BF68.7050104@dev.mellanox.co.il> Hi Sasha, This is a reworked patch that sets SL in the IPoIB mcast groups Note that I'm using functions here! :) Added mcast mlid to the partition structure. This mlid is used later by the QoS manager for quick access to the mcast group. The patch is for master only. Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_partition.h | 5 ++ opensm/opensm/osm_prtn.c | 9 +++- opensm/opensm/osm_qos_policy.c | 84 ++++++++++++++++++++++++++------ 3 files changed, 80 insertions(+), 18 deletions(-) diff --git a/opensm/include/opensm/osm_partition.h b/opensm/include/opensm/osm_partition.h index 70da27d..326aeb6 100644 --- a/opensm/include/opensm/osm_partition.h +++ b/opensm/include/opensm/osm_partition.h @@ -97,6 +97,7 @@ BEGIN_C_DECLS typedef struct _osm_prtn { cl_map_item_t map_item; ib_net16_t pkey; + ib_net16_t mlid; uint8_t sl; cl_map_t full_guid_tbl; cl_map_t part_guid_tbl; @@ -110,6 +111,10 @@ typedef struct _osm_prtn { * pkey * The IBA defined P_KEY of this Partition. * +* mlid +* The network ordered LID of the well known Multicast Group +* that was created for this partition. +* * sl * The Service Level (SL) associated with this Partiton. * diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c index 2d0b313..187cff6 100644 --- a/opensm/opensm/osm_prtn.c +++ b/opensm/opensm/osm_prtn.c @@ -230,8 +230,10 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, OSM_LOG(p_log, OSM_LOG_ERROR, "Failed to create MC group with pkey 0x%04x\n", cl_ntoh16(pkey)); - if (p_mgrp) + if (p_mgrp) { p_mgrp->well_known = TRUE; + p->mlid = p_mgrp->mlid; + } /* workaround for TS */ /* FIXME: remove this upon TS fixes */ @@ -243,8 +245,11 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, status = osm_mcmr_rcv_find_or_create_new_mgrp(p_sa, comp_mask, &mc_rec, &p_mgrp); - if (p_mgrp) + if (p_mgrp) { p_mgrp->well_known = TRUE; + if (!p->mlid) + p->mlid = p_mgrp->mlid; + } return status; } diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c index aef1856..1c428e5 100644 --- a/opensm/opensm/osm_qos_policy.c +++ b/opensm/opensm/osm_qos_policy.c @@ -763,6 +763,69 @@ static osm_qos_port_group_t *__qos_policy_get_port_group_by_name( /*************************************************** ***************************************************/ +static void __qos_policy_validate_pkey( + osm_qos_policy_t * p_qos_policy, + osm_qos_match_rule_t * p_qos_match_rule, + osm_prtn_t * p_prtn) +{ + uint8_t sl; + uint32_t flow; + uint8_t hop; + osm_mgrp_t * p_mgrp; + + if (!p_qos_policy || !p_qos_match_rule || !p_prtn) + return; + + if (!p_qos_match_rule->p_qos_level->sl_set || + p_prtn->sl == p_qos_match_rule->p_qos_level->sl) + return; + + /* overriding partition's SL */ + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_ERROR, + "ERR AC15: pkey 0x%04X in match rule - " + "overriding partition SL (%u) with QoS Level SL (%u)\n", + cl_ntoh16(p_prtn->pkey), p_prtn->sl, + p_qos_match_rule->p_qos_level->sl); + p_prtn->sl = p_qos_match_rule->p_qos_level->sl; + + + /* If this partition is an IPoIB partition, there should + be a matching MCast group. Fix this group's SL too */ + + if (!p_prtn->mlid) + return; + + p_mgrp = (osm_mgrp_t *) cl_qmap_get( + &p_qos_policy->p_subn->mgrp_mlid_tbl, + p_prtn->mlid); + if (p_mgrp == (osm_mgrp_t *) + cl_qmap_end(&p_qos_policy->p_subn->mgrp_mlid_tbl)) { + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_ERROR, + "ERR AC16: MCast group for partition with " + "pkey 0x%04X not found\n", + cl_ntoh16(p_prtn->pkey)); + return; + } + + CL_ASSERT((cl_ntoh16(p_mgrp->mcmember_rec.pkey) & 0x7fff) == + (cl_ntoh16(p_prtn->pkey) & 0x7fff)); + + ib_member_get_sl_flow_hop(p_mgrp->mcmember_rec.sl_flow_hop, + &sl, &flow, &hop); + if (sl != p_prtn->sl) { + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_DEBUG, + "Updating MCGroup (MLID 0x%04x) SL to " + "match partition SL (%u)\n", + cl_hton16(p_mgrp->mcmember_rec.mlid), + p_prtn->sl); + p_mgrp->mcmember_rec.sl_flow_hop = + ib_member_set_sl_flow_hop(p_prtn->sl, flow, hop); + } +} + +/*************************************************** + ***************************************************/ + int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, osm_log_t *p_log) { @@ -901,27 +964,16 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, &p_qos_policy->p_subn->prtn_pkey_tbl, pkey); if (p_prtn == (osm_prtn_t *)cl_qmap_end( - &p_qos_policy->p_subn->prtn_pkey_tbl)) { + &p_qos_policy->p_subn->prtn_pkey_tbl)) /* partition for this pkey not found */ OSM_LOG(p_log, OSM_LOG_ERROR, "ERR AC14: " "pkey 0x%04X in match rule - " "partition doesn't exist\n", cl_ntoh16(pkey)); - continue; - } - - if (p_qos_match_rule->p_qos_level->sl_set && - p_prtn->sl != p_qos_match_rule->p_qos_level->sl) { - /* overriding partition's SL */ - OSM_LOG(p_log, OSM_LOG_ERROR, "ERR AC15: " - "pkey 0x%04X in match rule - " - "overriding partition SL (%u) " - "with QoS Level SL (%u)\n", - cl_ntoh16(pkey), - p_prtn->sl, - p_qos_match_rule->p_qos_level->sl); - p_prtn->sl = p_qos_match_rule->p_qos_level->sl; - } + else + __qos_policy_validate_pkey(p_qos_policy, + p_qos_match_rule, + p_prtn); } } -- 1.5.1.4 From eli at dev.mellanox.co.il Mon Mar 24 07:58:38 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 24 Mar 2008 16:58:38 +0200 Subject: [ofa-general] [PATCH 4/10] IB/ipoib: Add LSO support to ipoib In-Reply-To: <47E607D2.5000207@voltaire.com> References: <1205767435.25950.139.camel@mtls03> <47E607D2.5000207@voltaire.com> Message-ID: <1206370718.25950.302.camel@mtls03> On Sun, 2008-03-23 at 09:33 +0200, Or Gerlitz wrote: > Eli Cohen wrote: > > --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c > > +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c > > @@ -418,14 +449,35 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, > > { > > + int hlen; > > + void *phead; > > + > > + if (!skb_is_gso(skb)) { > > + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { > > + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", > > + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); > > + ++dev->stats.tx_dropped; > > + ++dev->stats.tx_errors; > > + ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); > > + return; > > + } > > + phead = 0; > > + hlen = 0; > > + } else { > > + /* > > + * LSO header is limited to max 60 bytes > > + */ > > + if (unlikely((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) > 15)) { > > + ipoib_warn(priv, "ip(%d) and tcp(%d) headers too long, dropping skb\n", > > + ip_hdr(skb)->ihl << 2, tcp_hdr(skb)->doff << 2); > > + goto drop; > > + } > > > is the 60 bytes being a limitation of the connectX HW, the Linux kernel > stack or some "lso spec"? > > + hlen = ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) + IPOIB_ENCAP_LEN; > > + phead = skb->data; It's an implementation decision - I assume that I will never get TSO SKBs where the headers exceed 60 bytes. > > > Looking in e1000 tso code (eg > http://lxr.linux.no/linux/drivers/net/e1000/e1000_main.c#L2884) I see > that the header len is gotten by > skb_transport_offset(skb) + tcp_hdrlen(skb), is that what > ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) gives? It looks like I would get the same effect if I'd used e1000 style though I'm not sure which approach is faster. > > @@ -470,6 +521,12 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, > > netif_stop_queue(dev); > > } > > } > > + return; > > + > > +drop: > > + ++dev->stats.tx_errors; > > + dev_kfree_skb_any(skb); > > + return; > > > shouldn't the tx_dropped counter be incremented here? I am not sure. Does every erroneous tx packets imply incrementing the drop counter too? From eli at dev.mellanox.co.il Mon Mar 24 07:58:38 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 24 Mar 2008 16:58:38 +0200 Subject: [ofa-general] [PATCH 4/10] IB/ipoib: Add LSO support to ipoib In-Reply-To: <47E607D2.5000207@voltaire.com> References: <1205767435.25950.139.camel@mtls03> <47E607D2.5000207@voltaire.com> Message-ID: <1206370718.25950.302.camel@mtls03> On Sun, 2008-03-23 at 09:33 +0200, Or Gerlitz wrote: > Eli Cohen wrote: > > --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c > > +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c > > @@ -418,14 +449,35 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, > > { > > + int hlen; > > + void *phead; > > + > > + if (!skb_is_gso(skb)) { > > + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { > > + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", > > + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); > > + ++dev->stats.tx_dropped; > > + ++dev->stats.tx_errors; > > + ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); > > + return; > > + } > > + phead = 0; > > + hlen = 0; > > + } else { > > + /* > > + * LSO header is limited to max 60 bytes > > + */ > > + if (unlikely((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) > 15)) { > > + ipoib_warn(priv, "ip(%d) and tcp(%d) headers too long, dropping skb\n", > > + ip_hdr(skb)->ihl << 2, tcp_hdr(skb)->doff << 2); > > + goto drop; > > + } > > > is the 60 bytes being a limitation of the connectX HW, the Linux kernel > stack or some "lso spec"? > > + hlen = ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) + IPOIB_ENCAP_LEN; > > + phead = skb->data; It's an implementation decision - I assume that I will never get TSO SKBs where the headers exceed 60 bytes. > > > Looking in e1000 tso code (eg > http://lxr.linux.no/linux/drivers/net/e1000/e1000_main.c#L2884) I see > that the header len is gotten by > skb_transport_offset(skb) + tcp_hdrlen(skb), is that what > ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2) gives? It looks like I would get the same effect if I'd used e1000 style though I'm not sure which approach is faster. > > @@ -470,6 +521,12 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, > > netif_stop_queue(dev); > > } > > } > > + return; > > + > > +drop: > > + ++dev->stats.tx_errors; > > + dev_kfree_skb_any(skb); > > + return; > > > shouldn't the tx_dropped counter be incremented here? I am not sure. Does every erroneous tx packets imply incrementing the drop counter too? From gstreiff at NetEffect.com Mon Mar 24 08:37:19 2008 From: gstreiff at NetEffect.com (Glenn Streiff) Date: Mon, 24 Mar 2008 10:37:19 -0500 Subject: [ofa-general] [PATCH] libnes: Fix "make dist" In-Reply-To: Message-ID: <5E701717F2B2ED4EA60F87C8AA57B7CC07950099@venom2> Applied! Thanks, Glenn > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org]On Behalf Of > Roland Dreier > Sent: Monday, March 24, 2008 12:33 AM > To: Glenn Streiff > Cc: general at lists.openfabrics.org > Subject: [ofa-general] [PATCH] libnes: Fix "make dist" > > > Need to include src/nes_umain.h in the generated tar file to create a > buildable distribution. Otherwise "make distcheck" fails with > > configure: error: cannot find sources (src/nes_umain.h) in .. > > Signed-off-by: Roland Dreier > --- > diff --git a/Makefile.am b/Makefile.am > index 73aea71..a5fdfc2 100644 > --- a/Makefile.am > +++ b/Makefile.am > @@ -19,7 +19,7 @@ nesconf_DATA = nes.driver > #DEBIAN = debian/changelog debian/compat debian/control > debian/copyright \ > # debian/libnes1.install debian/libnes-dev.install debian/rules > > -EXTRA_DIST = src/nes-abi.h \ > +EXTRA_DIST = src/nes-abi.h src/nes_umain.h \ > src/nes.map libnes.spec.in nes.driver $(DEBIAN) > > dist-hook: libnes.spec > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From tziporet at mellanox.co.il Mon Mar 24 08:42:54 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Mon, 24 Mar 2008 17:42:54 +0200 Subject: [ofa-general] Agenda for OFED meeting today (March 24) Message-ID: <6C2C79E72C305246B504CBA17B5500C9039C8D1A@mtlexch01.mtl.com> Agenda for OFED meeting today about OFED 1.4 plans: 1. Kernel base: since we target our release to Sep it seems the right kernel base will be 2.6.27 (and not 26 as I suggested before) 2. Suggestions for new features: * NFS-RDMA * Verbs: Reliable Multicast * SDP - Zero copy * IPoIB - continue with performance enhancements * Xsigo new virtual NIC (Hal correct me if I am wrong here) * New vendor HW support - ? * OpenSM: * Incremental routing * Temporary SA DB - to answer queries and a heavy sweep is done * APM - disjoint paths (?) * MKey manager (?) * MPI: * Open MPI 1.3 * APM support in MPI * mvapich ??? Please send more suggestions or raise them in the meeting today Tziporet -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Mon Mar 24 08:47:19 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 24 Mar 2008 17:47:19 +0200 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos In-Reply-To: <47E78657.60105@voltaire.com> References: <47E7565E.1040206@voltaire.com> <47E766CC.9070905@mellanox.co.il> <47E78657.60105@voltaire.com> Message-ID: <47E7CD07.9060509@mellanox.co.il> Or Gerlitz wrote: > Yevgeny Kliteynik wrote: >> OK. Can you rephrase what you said in two sentences? Something short >> to replace the existing (and wrong, as you pointed out) three lines: >> >> 151 IPoIB queries the SA for its broadcast group information. >> 152 It provides the broadcast group SL, MTU, and RATE in every >> following >> 153 PathRecord query performed when a new UDAV is needed by IPoIB. > Will try to send something later this week. Thanks > This document describes the QoS related architecture of Linux IB > stack, hence do you agree that the file name would be changed to > reflect that? ie that the word "ofed" will not appear there? Not sure I'm following. This doc describes OFED components (SDP, RDS, SRP, IPoIB, iSER), their support of QoS annex, and a way for a QoS manager (OpenSM, which is part of OFED) to enforce the QoS policy. I really don't have any special sentiments for file name, but is QoS_in_OFED.txt wrong? You probably meant something like QoS_architecture.txt? >>> This creates confusion and I am not sure its well defined, at least >>> in my case when I put my hands on QoS testing, I couldn't guess that >>> "ipoib" == "pkey 0x7fff". For example, is it correct that "ipoib, >>> pkey 0x8001 : " is not valid rule? >> >> This rule is also valid, providing that you actually have a partition >> with pkey 0x8001 configured in the partition cfg file. > OK, so the rule is valid, but would you agree that the word "ipoib" > adds nothing to the rule? Agree, it's just an alias. 'ipoib : ' is useful 'any, pkey 0xNNNN' is useful too. 'ipoib, pkey 0xNNNN' is just for better readability of the matching rules. -- Yevgeny > > Or. > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From hrosenstock at xsigo.com Mon Mar 24 08:54:31 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 24 Mar 2008 08:54:31 -0700 Subject: [ofa-general] [PATCH][TRIVIAL] infiniband-diags/ibping.c: Remove extraneous semicolon Message-ID: <1206374071.8099.453.camel@hrosenstock-ws.xsigo.com> Signed-off-by: Hal Rosenstock diff --git a/infiniband-diags/src/ibping.c b/infiniband-diags/src/ibping.c index 6c40e63..e847f42 100644 --- a/infiniband-diags/src/ibping.c +++ b/infiniband-diags/src/ibping.c @@ -132,7 +132,7 @@ ibping(ib_portid_t *portid, int quiet) if (!ib_vendor_call(data, portid, &call)) return ~0llu; - rtt = getcurrenttime() - start;; + rtt = getcurrenttime() - start; if (!last_host[0]) memcpy(last_host, data, sizeof last_host); From hrosenstock at xsigo.com Mon Mar 24 09:15:18 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 24 Mar 2008 09:15:18 -0700 Subject: [ofa-general] [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <47E67B1F.8050102@dev.mellanox.co.il> References: <47E67B1F.8050102@dev.mellanox.co.il> Message-ID: <1206375318.8099.461.camel@hrosenstock-ws.xsigo.com> On Sun, 2008-03-23 at 17:45 +0200, Yevgeny Kliteynik wrote: > But then I realized that this scenario will be a problem for any > communication (not only IPoIB), because as long as clients are not > issuing new PR request, Note that there are optional new 1.2.1 traps (68 and 69) for this. -- Hal From a-abruck at aeolian.com Mon Mar 24 09:17:16 2008 From: a-abruck at aeolian.com (Elwood Cote) Date: Tue, 25 Mar 2008 00:17:16 +0800 Subject: [ofa-general] I was looking for you Message-ID: <01c88e0d$94aace00$584ca9de@a-abruck> Hello! I am tired this evening. I am nice girl that would like to chat with you. Email me at Ellen at BestGolova.com only, because I am using my friend's email to write this. Mind me sending some of my pictures to you? From sashak at voltaire.com Mon Mar 24 09:29:30 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 16:29:30 +0000 Subject: [ofa-general] Re: [PATCH] opensm/QoS: setting SL in the IPoIB MCast groups In-Reply-To: <47E7BF68.7050104@dev.mellanox.co.il> References: <47E7BF68.7050104@dev.mellanox.co.il> Message-ID: <20080324162930.GJ595@sashak.voltaire.com> Hi Yevgeny, On 16:49 Mon 24 Mar , Yevgeny Kliteynik wrote: > > This is a reworked patch that sets SL in the IPoIB mcast groups > Note that I'm using functions here! :) > > Added mcast mlid to the partition structure. > This mlid is used later by the QoS manager for quick access > to the mcast group. Basically looks fine for me. One question however. > > The patch is for master only. > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/include/opensm/osm_partition.h | 5 ++ > opensm/opensm/osm_prtn.c | 9 +++- > opensm/opensm/osm_qos_policy.c | 84 ++++++++++++++++++++++++++------ > 3 files changed, 80 insertions(+), 18 deletions(-) > > diff --git a/opensm/include/opensm/osm_partition.h b/opensm/include/opensm/osm_partition.h > index 70da27d..326aeb6 100644 > --- a/opensm/include/opensm/osm_partition.h > +++ b/opensm/include/opensm/osm_partition.h > @@ -97,6 +97,7 @@ BEGIN_C_DECLS > typedef struct _osm_prtn { > cl_map_item_t map_item; > ib_net16_t pkey; > + ib_net16_t mlid; Why to not use 'osm_mgrp_t *mgr' instead of mlid and to not access mcast group directly w/out qmap search? Sasha > uint8_t sl; > cl_map_t full_guid_tbl; > cl_map_t part_guid_tbl; > @@ -110,6 +111,10 @@ typedef struct _osm_prtn { > * pkey > * The IBA defined P_KEY of this Partition. > * > +* mlid > +* The network ordered LID of the well known Multicast Group > +* that was created for this partition. > +* > * sl > * The Service Level (SL) associated with this Partiton. > * > diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c > index 2d0b313..187cff6 100644 > --- a/opensm/opensm/osm_prtn.c > +++ b/opensm/opensm/osm_prtn.c > @@ -230,8 +230,10 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, > OSM_LOG(p_log, OSM_LOG_ERROR, > "Failed to create MC group with pkey 0x%04x\n", > cl_ntoh16(pkey)); > - if (p_mgrp) > + if (p_mgrp) { > p_mgrp->well_known = TRUE; > + p->mlid = p_mgrp->mlid; > + } > > /* workaround for TS */ > /* FIXME: remove this upon TS fixes */ > @@ -243,8 +245,11 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, > > status = osm_mcmr_rcv_find_or_create_new_mgrp(p_sa, comp_mask, &mc_rec, > &p_mgrp); > - if (p_mgrp) > + if (p_mgrp) { > p_mgrp->well_known = TRUE; > + if (!p->mlid) > + p->mlid = p_mgrp->mlid; > + } > > return status; > } > diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c > index aef1856..1c428e5 100644 > --- a/opensm/opensm/osm_qos_policy.c > +++ b/opensm/opensm/osm_qos_policy.c > @@ -763,6 +763,69 @@ static osm_qos_port_group_t *__qos_policy_get_port_group_by_name( > /*************************************************** > ***************************************************/ > > +static void __qos_policy_validate_pkey( > + osm_qos_policy_t * p_qos_policy, > + osm_qos_match_rule_t * p_qos_match_rule, > + osm_prtn_t * p_prtn) > +{ > + uint8_t sl; > + uint32_t flow; > + uint8_t hop; > + osm_mgrp_t * p_mgrp; > + > + if (!p_qos_policy || !p_qos_match_rule || !p_prtn) > + return; > + > + if (!p_qos_match_rule->p_qos_level->sl_set || > + p_prtn->sl == p_qos_match_rule->p_qos_level->sl) > + return; > + > + /* overriding partition's SL */ > + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_ERROR, > + "ERR AC15: pkey 0x%04X in match rule - " > + "overriding partition SL (%u) with QoS Level SL (%u)\n", > + cl_ntoh16(p_prtn->pkey), p_prtn->sl, > + p_qos_match_rule->p_qos_level->sl); > + p_prtn->sl = p_qos_match_rule->p_qos_level->sl; > + > + > + /* If this partition is an IPoIB partition, there should > + be a matching MCast group. Fix this group's SL too */ > + > + if (!p_prtn->mlid) > + return; > + > + p_mgrp = (osm_mgrp_t *) cl_qmap_get( > + &p_qos_policy->p_subn->mgrp_mlid_tbl, > + p_prtn->mlid); > + if (p_mgrp == (osm_mgrp_t *) > + cl_qmap_end(&p_qos_policy->p_subn->mgrp_mlid_tbl)) { > + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_ERROR, > + "ERR AC16: MCast group for partition with " > + "pkey 0x%04X not found\n", > + cl_ntoh16(p_prtn->pkey)); > + return; > + } > + > + CL_ASSERT((cl_ntoh16(p_mgrp->mcmember_rec.pkey) & 0x7fff) == > + (cl_ntoh16(p_prtn->pkey) & 0x7fff)); > + > + ib_member_get_sl_flow_hop(p_mgrp->mcmember_rec.sl_flow_hop, > + &sl, &flow, &hop); > + if (sl != p_prtn->sl) { > + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_DEBUG, > + "Updating MCGroup (MLID 0x%04x) SL to " > + "match partition SL (%u)\n", > + cl_hton16(p_mgrp->mcmember_rec.mlid), > + p_prtn->sl); > + p_mgrp->mcmember_rec.sl_flow_hop = > + ib_member_set_sl_flow_hop(p_prtn->sl, flow, hop); > + } > +} > + > +/*************************************************** > + ***************************************************/ > + > int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, > osm_log_t *p_log) > { > @@ -901,27 +964,16 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, > &p_qos_policy->p_subn->prtn_pkey_tbl, pkey); > > if (p_prtn == (osm_prtn_t *)cl_qmap_end( > - &p_qos_policy->p_subn->prtn_pkey_tbl)) { > + &p_qos_policy->p_subn->prtn_pkey_tbl)) > /* partition for this pkey not found */ > OSM_LOG(p_log, OSM_LOG_ERROR, "ERR AC14: " > "pkey 0x%04X in match rule - " > "partition doesn't exist\n", > cl_ntoh16(pkey)); > - continue; > - } > - > - if (p_qos_match_rule->p_qos_level->sl_set && > - p_prtn->sl != p_qos_match_rule->p_qos_level->sl) { > - /* overriding partition's SL */ > - OSM_LOG(p_log, OSM_LOG_ERROR, "ERR AC15: " > - "pkey 0x%04X in match rule - " > - "overriding partition SL (%u) " > - "with QoS Level SL (%u)\n", > - cl_ntoh16(pkey), > - p_prtn->sl, > - p_qos_match_rule->p_qos_level->sl); > - p_prtn->sl = p_qos_match_rule->p_qos_level->sl; > - } > + else > + __qos_policy_validate_pkey(p_qos_policy, > + p_qos_match_rule, > + p_prtn); > } > } > > -- > 1.5.1.4 > From hrosenstock at xsigo.com Mon Mar 24 09:19:15 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 24 Mar 2008 09:19:15 -0700 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <47E78395.9020604@dev.mellanox.co.il> References: <47E67B1F.8050102@dev.mellanox.co.il> <20080324061421.GH2102@sashak.voltaire.com> <47E753BF.9070505@dev.mellanox.co.il> <20080324085407.GA595@sashak.voltaire.com> <47E78395.9020604@dev.mellanox.co.il> Message-ID: <1206375555.8099.464.camel@hrosenstock-ws.xsigo.com> On Mon, 2008-03-24 at 12:33 +0200, Yevgeny Kliteynik wrote: > OK, I'll repost the patch for master only. > For OFED 1.3 the partition SL configuration should > be done through partition config file only. Should that tidbit be noted in the OpenSM documentation/man page anywhere (for OFED 1.3) ? From hrosenstock at xsigo.com Mon Mar 24 09:19:52 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 24 Mar 2008 09:19:52 -0700 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos In-Reply-To: <47E7CD07.9060509@mellanox.co.il> References: <47E7565E.1040206@voltaire.com> <47E766CC.9070905@mellanox.co.il> <47E78657.60105@voltaire.com> <47E7CD07.9060509@mellanox.co.il> Message-ID: <1206375592.8099.466.camel@hrosenstock-ws.xsigo.com> On Mon, 2008-03-24 at 17:47 +0200, Yevgeny Kliteynik wrote: > Or Gerlitz wrote: > >> This rule is also valid, providing that you actually have a partition > >> with pkey 0x8001 configured in the partition cfg file. > > OK, so the rule is valid, but would you agree that the word "ipoib" > > adds nothing to the rule? > > Agree, it's just an alias. > 'ipoib : ' is useful > 'any, pkey 0xNNNN' is useful too. > 'ipoib, pkey 0xNNNN' is just for better readability of the matching rules. What rule(s) are being discussed here ? QoS annex rules only ? QoS annex rules in concert with partition rules ? partition rules only ? AFAIK, ipoib keyword has a specific useful meaning to OpenSM (apart from SL). -- Hal From sashak at voltaire.com Mon Mar 24 09:32:28 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 24 Mar 2008 16:32:28 +0000 Subject: [ofa-general] Re: [PATCH][TRIVIAL] infiniband-diags/ibping.c: Remove extraneous semicolon In-Reply-To: <1206374071.8099.453.camel@hrosenstock-ws.xsigo.com> References: <1206374071.8099.453.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080324163228.GK595@sashak.voltaire.com> On 08:54 Mon 24 Mar , Hal Rosenstock wrote: > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From eli at dev.mellanox.co.il Mon Mar 24 09:30:45 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 24 Mar 2008 18:30:45 +0200 Subject: [ofa-general] [PATCH] IB/ipoib: Split CQs for IPOIB UD Message-ID: <1206376245.25950.317.camel@mtls03> I do not send the unsignaled patch since I could not see the it improves anything on the upstream kernel. The bellow patch have a significant improvement on small UDP messages message rate. You can see the affect on netperf: without the patch: sw175:~ # netperf -H 14.4.3.178 -t UDP_STREAM -- -m 128 UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 14.4.3.178 (14.4.3.178) port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 114688 128 10.00 2976016 0 304.67 114688 10.00 2973579 304.42 with the patch: sw175:~ # netperf -H 14.4.3.178 -t UDP_STREAM -- -m 128 UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 14.4.3.178 (14.4.3.178) port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 114688 128 10.00 5102042 0 522.23 114688 10.00 4246433 434.65 >From c56ce2c51d3003eef20da01678ade5762714ccc2 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 20 Mar 2008 16:35:30 +0200 Subject: [PATCH] IB/ipoib: Split CQs for IPOIB UD Use a dedicated CQ for UD send. Also, do not arm the UD send CQ thus reducing the number of interrupts generated by the HCA. This patch farther reduces overhead by not calling poll CQ for every posted send WR - it does it only when there 16 or more outstanding work requests. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 9 ++++-- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 8 ++-- drivers/infiniband/ulp/ipoib/ipoib_etool.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 45 ++++++++++++++++------------ drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 38 +++++++++++++++-------- 5 files changed, 62 insertions(+), 40 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 43feffc..fb28f0b 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -95,6 +95,8 @@ enum { IPOIB_MCAST_FLAG_SENDONLY = 1, IPOIB_MCAST_FLAG_BUSY = 2, /* joining or already joined */ IPOIB_MCAST_FLAG_ATTACHED = 3, + + MAX_SEND_CQE = 16, }; #define IPOIB_OP_RECV (1ul << 31) @@ -285,7 +287,8 @@ struct ipoib_dev_priv { u16 pkey_index; struct ib_pd *pd; struct ib_mr *mr; - struct ib_cq *cq; + struct ib_cq *rcq; + struct ib_cq *scq; struct ib_qp *qp; u32 qkey; @@ -305,7 +308,8 @@ struct ipoib_dev_priv { struct ib_send_wr tx_wr; unsigned tx_outstanding; - struct ib_wc ibwc[IPOIB_NUM_WC]; + struct ib_wc ibwc[IPOIB_NUM_WC]; + struct ib_wc send_wc[MAX_SEND_CQE]; struct list_head dead_ahs; @@ -650,7 +654,6 @@ static inline int ipoib_register_debugfs(void) { return 0; } static inline void ipoib_unregister_debugfs(void) { } #endif - #define ipoib_printk(level, priv, format, arg...) \ printk(level "%s: " format, ((struct ipoib_dev_priv *) priv)->dev->name , ## arg) #define ipoib_warn(priv, format, arg...) \ diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 90ff2c9..dfabb38 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -249,8 +249,8 @@ static struct ib_qp *ipoib_cm_create_rx_qp(struct net_device *dev, struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { .event_handler = ipoib_cm_rx_event_handler, - .send_cq = priv->cq, /* For drain WR */ - .recv_cq = priv->cq, + .send_cq = priv->rcq, /* For drain WR */ + .recv_cq = priv->rcq, .srq = priv->cm.srq, .cap.max_send_wr = 1, /* For drain WR */ .cap.max_send_sge = 1, /* FIXME: 0 Seems not to work */ @@ -951,8 +951,8 @@ static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ipoib_ { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { - .send_cq = priv->cq, - .recv_cq = priv->cq, + .send_cq = priv->rcq, + .recv_cq = priv->rcq, .srq = priv->cm.srq, .cap.max_send_wr = ipoib_sendq_size, .cap.max_send_sge = 1, diff --git a/drivers/infiniband/ulp/ipoib/ipoib_etool.c b/drivers/infiniband/ulp/ipoib/ipoib_etool.c index a3ac4cf..b4f4f0f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_etool.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_etool.c @@ -73,7 +73,7 @@ static int ipoib_set_coalesce(struct net_device *dev, coal->rx_max_coalesced_frames > 0xffff) return -EINVAL; - ret = ib_modify_cq(priv->cq, coal->rx_max_coalesced_frames, + ret = ib_modify_cq(priv->rcq, coal->rx_max_coalesced_frames, coal->rx_coalesce_usecs); if (ret) { ipoib_dbg(priv, "failed modifying CQ\n"); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 6605109..7cb8bd3 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -311,7 +311,6 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) struct ipoib_dev_priv *priv = netdev_priv(dev); unsigned int wr_id = wc->wr_id; struct ipoib_tx_buf *tx_req; - unsigned long flags; ipoib_dbg_data(priv, "send completion: id %d, status: %d\n", wr_id, wc->status); @@ -331,13 +330,11 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) dev_kfree_skb_any(tx_req->skb); - spin_lock_irqsave(&priv->tx_lock, flags); ++priv->tx_tail; if (unlikely(--priv->tx_outstanding == ipoib_sendq_size >> 1) && netif_queue_stopped(dev) && test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) netif_wake_queue(dev); - spin_unlock_irqrestore(&priv->tx_lock, flags); if (wc->status != IB_WC_SUCCESS && wc->status != IB_WC_WR_FLUSH_ERR) @@ -346,6 +343,17 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) wc->status, wr_id, wc->vendor_err); } +static int poll_tx(struct ipoib_dev_priv *priv) +{ + int n, i; + + n = ib_poll_cq(priv->scq, MAX_SEND_CQE, priv->send_wc); + for (i = 0; i < n; ++i) + ipoib_ib_handle_tx_wc(priv->dev, priv->send_wc + i); + + return n == MAX_SEND_CQE; +} + int ipoib_poll(struct napi_struct *napi, int budget) { struct ipoib_dev_priv *priv = container_of(napi, struct ipoib_dev_priv, napi); @@ -361,7 +369,7 @@ poll_more: int max = (budget - done); t = min(IPOIB_NUM_WC, max); - n = ib_poll_cq(priv->cq, t, priv->ibwc); + n = ib_poll_cq(priv->rcq, t, priv->ibwc); for (i = 0; i < n; i++) { struct ib_wc *wc = priv->ibwc + i; @@ -372,12 +380,8 @@ poll_more: ipoib_cm_handle_rx_wc(dev, wc); else ipoib_ib_handle_rx_wc(dev, wc); - } else { - if (wc->wr_id & IPOIB_OP_CM) - ipoib_cm_handle_tx_wc(dev, wc); - else - ipoib_ib_handle_tx_wc(dev, wc); - } + } else + ipoib_cm_handle_tx_wc(priv->dev, wc); } if (n != t) @@ -386,7 +390,7 @@ poll_more: if (done < budget) { netif_rx_complete(dev, napi); - if (unlikely(ib_req_notify_cq(priv->cq, + if (unlikely(ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS)) && netif_rx_reschedule(dev, napi)) @@ -515,12 +519,17 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, address->last_send = priv->tx_head; ++priv->tx_head; + skb_orphan(skb); if (++priv->tx_outstanding == ipoib_sendq_size) { ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); netif_stop_queue(dev); } } + + if (unlikely(priv->tx_outstanding > MAX_SEND_CQE)) + poll_tx(priv); + return; drop: @@ -673,7 +682,7 @@ void ipoib_drain_cq(struct net_device *dev) struct ipoib_dev_priv *priv = netdev_priv(dev); int i, n; do { - n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); + n = ib_poll_cq(priv->rcq, IPOIB_NUM_WC, priv->ibwc); for (i = 0; i < n; ++i) { /* * Convert any successful completions to flush @@ -688,14 +697,12 @@ void ipoib_drain_cq(struct net_device *dev) ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); else ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); - } else { - if (priv->ibwc[i].wr_id & IPOIB_OP_CM) - ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); - else - ipoib_ib_handle_tx_wc(dev, priv->ibwc + i); - } + } else + ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); } } while (n == IPOIB_NUM_WC); + + while(poll_tx(priv)); } int ipoib_ib_dev_stop(struct net_device *dev, int flush) @@ -787,7 +794,7 @@ timeout: msleep(1); } - ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP); + ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP); return 0; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 1d59f27..ce12ecd 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -171,7 +171,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) goto out_free_pd; } - size = ipoib_sendq_size + ipoib_recvq_size + 1; + size = ipoib_recvq_size + 1; ret = ipoib_cm_dev_init(dev); if (!ret) { if (ipoib_cm_has_srq(dev)) @@ -180,17 +180,23 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) size += ipoib_recvq_size * ipoib_max_conn_qp; } - priv->cq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); - if (IS_ERR(priv->cq)) { - printk(KERN_WARNING "%s: failed to create CQ\n", ca->name); + priv->rcq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); + if (IS_ERR(priv->rcq)) { + printk(KERN_WARNING "%s: failed to create receive CQ\n", ca->name); goto out_free_mr; } - if (ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP)) - goto out_free_cq; + priv->scq = ib_create_cq(priv->ca, NULL, NULL, dev, ipoib_sendq_size, 0); + if (IS_ERR(priv->scq)) { + printk(KERN_WARNING "%s: failed to create send CQ\n", ca->name); + goto out_free_rcq; + } + + if (ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP)) + goto out_free_scq; - init_attr.send_cq = priv->cq; - init_attr.recv_cq = priv->cq; + init_attr.send_cq = priv->scq; + init_attr.recv_cq = priv->rcq; if (priv->hca_caps & IB_DEVICE_TCP_TSO) init_attr.create_flags = QP_CREATE_LSO; @@ -201,7 +207,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) priv->qp = ib_create_qp(priv->pd, &init_attr); if (IS_ERR(priv->qp)) { printk(KERN_WARNING "%s: failed to create QP\n", ca->name); - goto out_free_cq; + goto out_free_scq; } priv->dev->dev_addr[1] = (priv->qp->qp_num >> 16) & 0xff; @@ -217,8 +223,11 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) return 0; -out_free_cq: - ib_destroy_cq(priv->cq); +out_free_scq: + ib_destroy_cq(priv->scq); + +out_free_rcq: + ib_destroy_cq(priv->rcq); out_free_mr: ib_dereg_mr(priv->mr); @@ -241,8 +250,11 @@ void ipoib_transport_dev_cleanup(struct net_device *dev) clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); } - if (ib_destroy_cq(priv->cq)) - ipoib_warn(priv, "ib_cq_destroy failed\n"); + if (ib_destroy_cq(priv->scq)) + ipoib_warn(priv, "ib_cq_destroy (send) failed\n"); + + if (ib_destroy_cq(priv->rcq)) + ipoib_warn(priv, "ib_cq_destroy (recv) failed\n"); ipoib_cm_dev_cleanup(dev); -- 1.5.4.4 From eli at dev.mellanox.co.il Mon Mar 24 09:30:45 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 24 Mar 2008 18:30:45 +0200 Subject: [ofa-general] [PATCH] IB/ipoib: Split CQs for IPOIB UD Message-ID: <1206376245.25950.317.camel@mtls03> I do not send the unsignaled patch since I could not see the it improves anything on the upstream kernel. The bellow patch have a significant improvement on small UDP messages message rate. You can see the affect on netperf: without the patch: sw175:~ # netperf -H 14.4.3.178 -t UDP_STREAM -- -m 128 UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 14.4.3.178 (14.4.3.178) port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 114688 128 10.00 2976016 0 304.67 114688 10.00 2973579 304.42 with the patch: sw175:~ # netperf -H 14.4.3.178 -t UDP_STREAM -- -m 128 UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 14.4.3.178 (14.4.3.178) port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 114688 128 10.00 5102042 0 522.23 114688 10.00 4246433 434.65 >From c56ce2c51d3003eef20da01678ade5762714ccc2 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 20 Mar 2008 16:35:30 +0200 Subject: [PATCH] IB/ipoib: Split CQs for IPOIB UD Use a dedicated CQ for UD send. Also, do not arm the UD send CQ thus reducing the number of interrupts generated by the HCA. This patch farther reduces overhead by not calling poll CQ for every posted send WR - it does it only when there 16 or more outstanding work requests. Signed-off-by: Eli Cohen --- drivers/infiniband/ulp/ipoib/ipoib.h | 9 ++++-- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 8 ++-- drivers/infiniband/ulp/ipoib/ipoib_etool.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 45 ++++++++++++++++------------ drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 38 +++++++++++++++-------- 5 files changed, 62 insertions(+), 40 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 43feffc..fb28f0b 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -95,6 +95,8 @@ enum { IPOIB_MCAST_FLAG_SENDONLY = 1, IPOIB_MCAST_FLAG_BUSY = 2, /* joining or already joined */ IPOIB_MCAST_FLAG_ATTACHED = 3, + + MAX_SEND_CQE = 16, }; #define IPOIB_OP_RECV (1ul << 31) @@ -285,7 +287,8 @@ struct ipoib_dev_priv { u16 pkey_index; struct ib_pd *pd; struct ib_mr *mr; - struct ib_cq *cq; + struct ib_cq *rcq; + struct ib_cq *scq; struct ib_qp *qp; u32 qkey; @@ -305,7 +308,8 @@ struct ipoib_dev_priv { struct ib_send_wr tx_wr; unsigned tx_outstanding; - struct ib_wc ibwc[IPOIB_NUM_WC]; + struct ib_wc ibwc[IPOIB_NUM_WC]; + struct ib_wc send_wc[MAX_SEND_CQE]; struct list_head dead_ahs; @@ -650,7 +654,6 @@ static inline int ipoib_register_debugfs(void) { return 0; } static inline void ipoib_unregister_debugfs(void) { } #endif - #define ipoib_printk(level, priv, format, arg...) \ printk(level "%s: " format, ((struct ipoib_dev_priv *) priv)->dev->name , ## arg) #define ipoib_warn(priv, format, arg...) \ diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 90ff2c9..dfabb38 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -249,8 +249,8 @@ static struct ib_qp *ipoib_cm_create_rx_qp(struct net_device *dev, struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { .event_handler = ipoib_cm_rx_event_handler, - .send_cq = priv->cq, /* For drain WR */ - .recv_cq = priv->cq, + .send_cq = priv->rcq, /* For drain WR */ + .recv_cq = priv->rcq, .srq = priv->cm.srq, .cap.max_send_wr = 1, /* For drain WR */ .cap.max_send_sge = 1, /* FIXME: 0 Seems not to work */ @@ -951,8 +951,8 @@ static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ipoib_ { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { - .send_cq = priv->cq, - .recv_cq = priv->cq, + .send_cq = priv->rcq, + .recv_cq = priv->rcq, .srq = priv->cm.srq, .cap.max_send_wr = ipoib_sendq_size, .cap.max_send_sge = 1, diff --git a/drivers/infiniband/ulp/ipoib/ipoib_etool.c b/drivers/infiniband/ulp/ipoib/ipoib_etool.c index a3ac4cf..b4f4f0f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_etool.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_etool.c @@ -73,7 +73,7 @@ static int ipoib_set_coalesce(struct net_device *dev, coal->rx_max_coalesced_frames > 0xffff) return -EINVAL; - ret = ib_modify_cq(priv->cq, coal->rx_max_coalesced_frames, + ret = ib_modify_cq(priv->rcq, coal->rx_max_coalesced_frames, coal->rx_coalesce_usecs); if (ret) { ipoib_dbg(priv, "failed modifying CQ\n"); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 6605109..7cb8bd3 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -311,7 +311,6 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) struct ipoib_dev_priv *priv = netdev_priv(dev); unsigned int wr_id = wc->wr_id; struct ipoib_tx_buf *tx_req; - unsigned long flags; ipoib_dbg_data(priv, "send completion: id %d, status: %d\n", wr_id, wc->status); @@ -331,13 +330,11 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) dev_kfree_skb_any(tx_req->skb); - spin_lock_irqsave(&priv->tx_lock, flags); ++priv->tx_tail; if (unlikely(--priv->tx_outstanding == ipoib_sendq_size >> 1) && netif_queue_stopped(dev) && test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) netif_wake_queue(dev); - spin_unlock_irqrestore(&priv->tx_lock, flags); if (wc->status != IB_WC_SUCCESS && wc->status != IB_WC_WR_FLUSH_ERR) @@ -346,6 +343,17 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) wc->status, wr_id, wc->vendor_err); } +static int poll_tx(struct ipoib_dev_priv *priv) +{ + int n, i; + + n = ib_poll_cq(priv->scq, MAX_SEND_CQE, priv->send_wc); + for (i = 0; i < n; ++i) + ipoib_ib_handle_tx_wc(priv->dev, priv->send_wc + i); + + return n == MAX_SEND_CQE; +} + int ipoib_poll(struct napi_struct *napi, int budget) { struct ipoib_dev_priv *priv = container_of(napi, struct ipoib_dev_priv, napi); @@ -361,7 +369,7 @@ poll_more: int max = (budget - done); t = min(IPOIB_NUM_WC, max); - n = ib_poll_cq(priv->cq, t, priv->ibwc); + n = ib_poll_cq(priv->rcq, t, priv->ibwc); for (i = 0; i < n; i++) { struct ib_wc *wc = priv->ibwc + i; @@ -372,12 +380,8 @@ poll_more: ipoib_cm_handle_rx_wc(dev, wc); else ipoib_ib_handle_rx_wc(dev, wc); - } else { - if (wc->wr_id & IPOIB_OP_CM) - ipoib_cm_handle_tx_wc(dev, wc); - else - ipoib_ib_handle_tx_wc(dev, wc); - } + } else + ipoib_cm_handle_tx_wc(priv->dev, wc); } if (n != t) @@ -386,7 +390,7 @@ poll_more: if (done < budget) { netif_rx_complete(dev, napi); - if (unlikely(ib_req_notify_cq(priv->cq, + if (unlikely(ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS)) && netif_rx_reschedule(dev, napi)) @@ -515,12 +519,17 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, address->last_send = priv->tx_head; ++priv->tx_head; + skb_orphan(skb); if (++priv->tx_outstanding == ipoib_sendq_size) { ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); netif_stop_queue(dev); } } + + if (unlikely(priv->tx_outstanding > MAX_SEND_CQE)) + poll_tx(priv); + return; drop: @@ -673,7 +682,7 @@ void ipoib_drain_cq(struct net_device *dev) struct ipoib_dev_priv *priv = netdev_priv(dev); int i, n; do { - n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); + n = ib_poll_cq(priv->rcq, IPOIB_NUM_WC, priv->ibwc); for (i = 0; i < n; ++i) { /* * Convert any successful completions to flush @@ -688,14 +697,12 @@ void ipoib_drain_cq(struct net_device *dev) ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); else ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); - } else { - if (priv->ibwc[i].wr_id & IPOIB_OP_CM) - ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); - else - ipoib_ib_handle_tx_wc(dev, priv->ibwc + i); - } + } else + ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); } } while (n == IPOIB_NUM_WC); + + while(poll_tx(priv)); } int ipoib_ib_dev_stop(struct net_device *dev, int flush) @@ -787,7 +794,7 @@ timeout: msleep(1); } - ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP); + ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP); return 0; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 1d59f27..ce12ecd 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -171,7 +171,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) goto out_free_pd; } - size = ipoib_sendq_size + ipoib_recvq_size + 1; + size = ipoib_recvq_size + 1; ret = ipoib_cm_dev_init(dev); if (!ret) { if (ipoib_cm_has_srq(dev)) @@ -180,17 +180,23 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) size += ipoib_recvq_size * ipoib_max_conn_qp; } - priv->cq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); - if (IS_ERR(priv->cq)) { - printk(KERN_WARNING "%s: failed to create CQ\n", ca->name); + priv->rcq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); + if (IS_ERR(priv->rcq)) { + printk(KERN_WARNING "%s: failed to create receive CQ\n", ca->name); goto out_free_mr; } - if (ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP)) - goto out_free_cq; + priv->scq = ib_create_cq(priv->ca, NULL, NULL, dev, ipoib_sendq_size, 0); + if (IS_ERR(priv->scq)) { + printk(KERN_WARNING "%s: failed to create send CQ\n", ca->name); + goto out_free_rcq; + } + + if (ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP)) + goto out_free_scq; - init_attr.send_cq = priv->cq; - init_attr.recv_cq = priv->cq; + init_attr.send_cq = priv->scq; + init_attr.recv_cq = priv->rcq; if (priv->hca_caps & IB_DEVICE_TCP_TSO) init_attr.create_flags = QP_CREATE_LSO; @@ -201,7 +207,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) priv->qp = ib_create_qp(priv->pd, &init_attr); if (IS_ERR(priv->qp)) { printk(KERN_WARNING "%s: failed to create QP\n", ca->name); - goto out_free_cq; + goto out_free_scq; } priv->dev->dev_addr[1] = (priv->qp->qp_num >> 16) & 0xff; @@ -217,8 +223,11 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) return 0; -out_free_cq: - ib_destroy_cq(priv->cq); +out_free_scq: + ib_destroy_cq(priv->scq); + +out_free_rcq: + ib_destroy_cq(priv->rcq); out_free_mr: ib_dereg_mr(priv->mr); @@ -241,8 +250,11 @@ void ipoib_transport_dev_cleanup(struct net_device *dev) clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); } - if (ib_destroy_cq(priv->cq)) - ipoib_warn(priv, "ib_cq_destroy failed\n"); + if (ib_destroy_cq(priv->scq)) + ipoib_warn(priv, "ib_cq_destroy (send) failed\n"); + + if (ib_destroy_cq(priv->rcq)) + ipoib_warn(priv, "ib_cq_destroy (recv) failed\n"); ipoib_cm_dev_cleanup(dev); -- 1.5.4.4 From an.pocharski at usmagazine.com Mon Mar 24 09:53:01 2008 From: an.pocharski at usmagazine.com (EuroSoftware) Date: Mon, 24 Mar 2008 11:53:01 -0500 Subject: [ofa-general] Die groesste Standardsoftware fuer Minipreise Message-ID: <933134502.51674187447784@usmagazine.com> An HTML attachment was scrubbed... URL: From pawpawge at netdoor.com Mon Mar 24 08:23:23 2008 From: pawpawge at netdoor.com (hakim ignatius) Date: Mon, 24 Mar 2008 15:23:23 +0000 Subject: [ofa-general] Stunning presentation Nicole Kidman Message-ID: <000901c88dd1$01e1b162$24254f88@ngxfys> #DldejZMadonna New video with a naked celebrity. #ldejZqThe presentation is Stunning! #ldejZqOnly 1 day trial - get this Full mp3 now! #ldejZq Download it now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From julia at diku.dk Mon Mar 24 11:09:41 2008 From: julia at diku.dk (Julia Lawall) Date: Mon, 24 Mar 2008 19:09:41 +0100 (CET) Subject: [ofa-general] [PATCH 3/5] drivers/infiniband: test for IS_ERR rather than 0 Message-ID: From: Julia Lawall The function rdma_create_id always returns either a valid pointer or a value made with ERR_PTR, so its result should be tested with IS_ERR, not with a test for 0. The problem was found using the following semantic match. (http://www.emn.fr/x-info/coccinelle/) // @a@ expression E, E1; statement S,S1; position p; @@ E = rdma_create_id(...) ... when != E = E1 if at p (E) S else S1 @n@ position a.p; expression E,E1; statement S,S1; @@ E = NULL ... when != E = E1 if at p (E) S else S1 @depends on !n@ expression E; statement S,S1; position a.p; @@ * if at p (E) S else S1 // Signed-off-by: Julia Lawall --- drivers/infiniband/core/cma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff -u -p a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c --- a/drivers/infiniband/core/cma.c 2008-03-12 14:13:13.000000000 +0100 +++ b/drivers/infiniband/core/cma.c 2008-03-24 16:10:01.000000000 +0100 @@ -1289,7 +1289,7 @@ static int iw_conn_req_handler(struct iw new_cm_id = rdma_create_id(listen_id->id.event_handler, listen_id->id.context, RDMA_PS_TCP); - if (!new_cm_id) { + if (IS_ERR(new_cm_id)) { ret = -ENOMEM; goto out; } From rdreier at cisco.com Mon Mar 24 12:06:20 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 24 Mar 2008 12:06:20 -0700 Subject: [ofa-general] Re: [PATCH 3/5] drivers/infiniband: test for IS_ERR rather than 0 In-Reply-To: (Julia Lawall's message of "Mon, 24 Mar 2008 19:09:41 +0100 (CET)") References: Message-ID: Neat. Thanks, applied. From rdreier at cisco.com Mon Mar 24 12:11:19 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 24 Mar 2008 12:11:19 -0700 Subject: [ofa-general] max_mr limit In-Reply-To: <3307cdf90803212102x1d2500dfv22edd0e26175e7f6@mail.gmail.com> (Rajouri Jammu's message of "Fri, 21 Mar 2008 21:02:09 -0700") References: <3307cdf90803102252r2db7fe75ud13f1c6104a70f7a@mail.gmail.com> <47D68888.3090709@mellanox.co.il> <3307cdf90803192250pb85059enf23663b203f8e42c@mail.gmail.com> <3307cdf90803211157l1c5b3febra0d4c89028b4523b@mail.gmail.com> <3307cdf90803212102x1d2500dfv22edd0e26175e7f6@mail.gmail.com> Message-ID: > > > kernel: ib_mthca 0000:01:00.0: Failed to initialize memory region > > table, aborting. > > > > You may not have enough contiguous memory for the buddy allocator > > bitmaps for the MTT. What does /proc/buddyinfo show when this > > happens? > > > > I have 32GB of physical memory in the system. > > cat /proc/buddyinfo > Node 0, zone DMA 3 5 3 2 4 0 2 > 0 2 0 2 > Node 0, zone DMA32 1 0 0 2 5 4 2 > 2 3 3 486 > Node 0, zone Normal 80 38 8 189 335 46 40 > 3 3 1 7366 > > No idea how to interpret that. Actually looks like you have lots of contiguous memory. I guess you could add printk()s before each return/goto err statement in mthca_init_mr_table() and see exactly what is failing. > I thought 2M MTTs will allow to map up to 8GB of memory? > 2^21*4K. Is that not correct? Not positive I'm remembering correctly, but I'm pretty sure that mthca accounts for MTTs in segments of 8 entries. So you get another factor of 8 in your calculation, and 2M MTTs will cover 64GB. From rFjishu at shxinjie.com Mon Mar 24 11:42:24 2008 From: rFjishu at shxinjie.com (archer rom) Date: Mon, 24 Mar 2008 18:42:24 +0000 Subject: [ofa-general] over 2000 watch models Message-ID: <000701c88ded$06c9e811$66a21a83@ymohlnip> Detailed quality replicas of the most wanted designer watches are here! Genuine solid stainless steel, and 99.9% accurate markings, finish and weight. - The worlds largest online retailer of luxury products, including: Rolex Sports Models Rolex Datejusts Breitling Cartier Porsche Design Dolce & Gabbana Dior Gucci Hermes Watches Patek Philippe Visit - www.bebetsye.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at mellanox.co.il Mon Mar 24 13:45:01 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Mon, 24 Mar 2008 22:45:01 +0200 Subject: [ofa-general] OFED March 24 meeting summary on OFED 1.4 plans Message-ID: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> > OFED March 24 meeting summary about OFED 1.4 and 1.3.1 plans: > 1.3.1 Release: As we decided we should do a release in 2-3 month after 1.3. In addition if there are any special fixes as outcome from the interop we can do a release earlier. All - please send me your requests for fixed issues and needed time frame and I will publish 1.3.1 schedule based on this. OFED 1.4: > 1. Kernel base: since we target 1.4 release to Sep we target the > kernel base to be 2.6.27 This is a good target, but we may need to stay with 2.6.26 if the kernel progress will not be aligned. > 2. Suggestions for new features: > * NFS-RDMA > * Verbs: Reliable Multicast (to be presented at Sonoma) > * SDP - Zero copy (There was a question on IPv6 support - seems no > one interested for now) > * IPoIB - continue with performance enhancements > * Xsigo new virtual NIC > * New vendor HW support - non was reported so far (IBM and Chelsio > - do you have something?) > * OpenSM: > * Incremental routing > * Temporary SA DB - to answer queries and a heavy sweep is done > * APM - disjoint paths (?) > * MKey manager (?) * Sasha to send more management features > * MPI: > * Open MPI 1.3 > * APM support in MPI > * mvapich ??? > * uDAPl * Extensions for new APIs (like XRC) - ? * uDAPL provider for interop between Windows & Linux * 1.2 and 2.0 will stay > 3. Supported OSes & kernels * RHEL 4 up 5,6,7 * RHEL 5 up 1,2 - all vendors please see if we can drop RHEL5 (base) * SLES10 SP1 SP2 * Kernel.org: start from 2.6.18 - need to decide here based on customers requests > * Fedora C8 * OpenSuSE 10.3 Open (for discussion at Sonoma): - Do we want to move that user level part will be based on tarballs and not git - Kernel.org and OFED kernel modules > Tziporet -------------- next part -------------- An HTML attachment was scrubbed... URL: From panda at cse.ohio-state.edu Mon Mar 24 14:06:01 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Mon, 24 Mar 2008 17:06:01 -0400 (EDT) Subject: [ofa-general] Re: [ewg] OFED March 24 meeting summary on OFED 1.4 plans In-Reply-To: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> Message-ID: Tziporet, We had some constraints here today and none of us could attend the conference call. > > * MPI: > > * Open MPI 1.3 > > * APM support in MPI > > * mvapich ??? Regarding your question, it will be MVAPICH 1.1 and MVAPICH2 1.1 for OFED 1.4. APM support is already available in MVAPICH and MVAPICH2 for quite some time. Thanks, DK From rdreier at cisco.com Mon Mar 24 15:21:50 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 24 Mar 2008 15:21:50 -0700 Subject: [ofa-general] Re: [PATCH 3/10] IB/core: Add LSO support In-Reply-To: <1205767431.25950.138.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:51 +0200") References: <1205767431.25950.138.camel@mtls03> Message-ID: > @@ -622,7 +624,8 @@ enum ib_wr_opcode { > + IB_WR_LSO > @@ -630,7 +633,8 @@ enum ib_send_flags { > + IB_SEND_UDP_LSO = (1<<5) What is this send flag used for? It doesn't seem to appear anywhere else in the patch series and it seems redundant given that you've defined a new opcode for LSO send requests. I did a quick grep of the OFED 1.3 kernel tree and IB_SEND_UDP_LSO only seems to appear once, in the line where it is defined. So I guess it's OK to drop this chunk of the patch? - R. From rdreier at cisco.com Mon Mar 24 15:21:50 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 24 Mar 2008 15:21:50 -0700 Subject: [ofa-general] Re: [PATCH 3/10] IB/core: Add LSO support In-Reply-To: <1205767431.25950.138.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:51 +0200") References: <1205767431.25950.138.camel@mtls03> Message-ID: > @@ -622,7 +624,8 @@ enum ib_wr_opcode { > + IB_WR_LSO > @@ -630,7 +633,8 @@ enum ib_send_flags { > + IB_SEND_UDP_LSO = (1<<5) What is this send flag used for? It doesn't seem to appear anywhere else in the patch series and it seems redundant given that you've defined a new opcode for LSO send requests. I did a quick grep of the OFED 1.3 kernel tree and IB_SEND_UDP_LSO only seems to appear once, in the line where it is defined. So I guess it's OK to drop this chunk of the patch? - R. From ralph.campbell at qlogic.com Mon Mar 24 15:33:07 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Mon, 24 Mar 2008 15:33:07 -0700 Subject: [ofa-general] [PATCH 0/10] IB/ipath -- patches in for-roland for 2.6.26 In-Reply-To: References: <20080321001735.13431.12223.stgit@eng-46.mv.qlogic.com> Message-ID: <1206397987.5109.1115.camel@brick.pathscale.com> Thanks! Acked-by: Ralph Campbell On Fri, 2008-03-21 at 19:47 -0700, Roland Dreier wrote: > By the way... I got tired of seeing the warning, so I added the patch > below to my tree. Please let me know if this is not the right fix and > I will drop it... > > IB/ipath: Fix PCI config write size used to clear linkctrl error bits > > In slave_or_pri_blk(), pci_write_config_byte() is used to write a > 16-bit quantity to clear linkctrl CRC error bits. This is clearly a > bug and also causes the warning > > drivers/infiniband/hw/ipath/ipath_iba6110.c: In function 'slave_or_pri_blk': > drivers/infiniband/hw/ipath/ipath_iba6110.c:849: warning: overflow in implicit constant conversion > > Fix this by using pci_write_config_word() instead. > > Signed-off-by: Roland Dreier > --- > drivers/infiniband/hw/ipath/ipath_iba6110.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/drivers/infiniband/hw/ipath/ipath_iba6110.c b/drivers/infiniband/hw/ipath/ipath_iba6110.c > index 19e3955..d241f1c 100644 > --- a/drivers/infiniband/hw/ipath/ipath_iba6110.c > +++ b/drivers/infiniband/hw/ipath/ipath_iba6110.c > @@ -845,7 +845,7 @@ static void slave_or_pri_blk(struct ipath_devdata *dd, struct pci_dev *pdev, > /* > * now write them back to clear the error. > */ > - pci_write_config_byte(pdev, link_off, > + pci_write_config_word(pdev, link_off, > linkctrl & (0xf << 8)); > } > } From a-alanb at acenz.com Mon Mar 24 17:44:53 2008 From: a-alanb at acenz.com (Patti Saldana) Date: Tue, 25 Mar 2008 08:44:53 +0800 Subject: [ofa-general] message from dating website Message-ID: <01c88e54$7e743080$40206e7d@a-alanb> Hello! I am bored this evening. I am nice girl that would like to chat with you. Email me at Gunilla at BestGolova.com only, because I am using my friend's email to write this. Would you mind me showing some nice pictures of me? From LakishaofferMyles at hitmill.com Mon Mar 24 13:45:05 2008 From: LakishaofferMyles at hitmill.com (Earnestine Myles) Date: Mon, 24 Mar 2008 23:45:05 +0300 Subject: [ofa-general] Superstar stock report Message-ID: <6f5e01c88e22$6d206150$0100a8c0@FRANCO> Medical suuplies is one of the hottest sectors currently And the timing is very opportune. With cutting edge technology and state of the ART sortwares to do both improve complex patient care and improve profits for medical practitioners, we believe we have a winner in ZYTO CORP Symbol : ZYTC. Looks like the selling is over, Read the PR's and research ZYTO Corp and see why this is a value stock at bargain price. It's due and ripe for a breakout. Ride the easy 10 bagger Sym: ZYTC -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-antblu at acerracing.com Mon Mar 24 21:48:52 2008 From: a-antblu at acerracing.com (Jacklyn Diamond) Date: Tue, 25 Mar 2008 12:48:52 +0800 Subject: [ofa-general] the girl is looking for you Message-ID: <01c88e76$93fa5200$0ab9133c@a-antblu> Hello! I am tired this afternoon. I am nice girl that would like to chat with you. Email me at Camilla at BestGolova.com only, because I am using my friend's email to write this. Wanna see some pictures of me? From eli at dev.mellanox.co.il Mon Mar 24 21:59:26 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 06:59:26 +0200 Subject: [ofa-general] Re: [PATCH 3/10] IB/core: Add LSO support In-Reply-To: References: <1205767431.25950.138.camel@mtls03> Message-ID: <1206421166.6116.2.camel@eli-laptop> On Mon, 2008-03-24 at 15:21 -0700, Roland Dreier wrote: > > @@ -622,7 +624,8 @@ enum ib_wr_opcode { > > + IB_WR_LSO > > > @@ -630,7 +633,8 @@ enum ib_send_flags { > > + IB_SEND_UDP_LSO = (1<<5) > > What is this send flag used for? It doesn't seem to appear anywhere > else in the patch series and it seems redundant given that you've > defined a new opcode for LSO send requests. > > I did a quick grep of the OFED 1.3 kernel tree and IB_SEND_UDP_LSO > only seems to appear once, in the line where it is defined. So I > guess it's OK to drop this chunk of the patch? Yes, it can be removed. Thanks. From eli at dev.mellanox.co.il Mon Mar 24 21:59:26 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 06:59:26 +0200 Subject: [ofa-general] Re: [PATCH 3/10] IB/core: Add LSO support In-Reply-To: References: <1205767431.25950.138.camel@mtls03> Message-ID: <1206421166.6116.2.camel@eli-laptop> On Mon, 2008-03-24 at 15:21 -0700, Roland Dreier wrote: > > @@ -622,7 +624,8 @@ enum ib_wr_opcode { > > + IB_WR_LSO > > > @@ -630,7 +633,8 @@ enum ib_send_flags { > > + IB_SEND_UDP_LSO = (1<<5) > > What is this send flag used for? It doesn't seem to appear anywhere > else in the patch series and it seems redundant given that you've > defined a new opcode for LSO send requests. > > I did a quick grep of the OFED 1.3 kernel tree and IB_SEND_UDP_LSO > only seems to appear once, in the line where it is defined. So I > guess it's OK to drop this chunk of the patch? Yes, it can be removed. Thanks. From kliteyn at dev.mellanox.co.il Mon Mar 24 23:02:12 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 25 Mar 2008 08:02:12 +0200 Subject: [ofa-general] Re: [PATCH] opensm/QoS: setting SL in the IPoIB MCast groups In-Reply-To: <20080324162930.GJ595@sashak.voltaire.com> References: <47E7BF68.7050104@dev.mellanox.co.il> <20080324162930.GJ595@sashak.voltaire.com> Message-ID: <47E89564.1040608@dev.mellanox.co.il> Sasha Khapyorsky wrote: > Hi Yevgeny, > > On 16:49 Mon 24 Mar , Yevgeny Kliteynik wrote: >> This is a reworked patch that sets SL in the IPoIB mcast groups >> Note that I'm using functions here! :) >> >> Added mcast mlid to the partition structure. >> This mlid is used later by the QoS manager for quick access >> to the mcast group. > > Basically looks fine for me. One question however. > >> The patch is for master only. >> >> Signed-off-by: Yevgeny Kliteynik >> --- >> opensm/include/opensm/osm_partition.h | 5 ++ >> opensm/opensm/osm_prtn.c | 9 +++- >> opensm/opensm/osm_qos_policy.c | 84 ++++++++++++++++++++++++++------ >> 3 files changed, 80 insertions(+), 18 deletions(-) >> >> diff --git a/opensm/include/opensm/osm_partition.h b/opensm/include/opensm/osm_partition.h >> index 70da27d..326aeb6 100644 >> --- a/opensm/include/opensm/osm_partition.h >> +++ b/opensm/include/opensm/osm_partition.h >> @@ -97,6 +97,7 @@ BEGIN_C_DECLS >> typedef struct _osm_prtn { >> cl_map_item_t map_item; >> ib_net16_t pkey; >> + ib_net16_t mlid; > > Why to not use 'osm_mgrp_t *mgr' instead of mlid and to not access mcast > group directly w/out qmap search? No special reason. Didn't want to think what would happen if the mcast group was removed for some reason and the partition will still have the pointer. If you really think that having a pointer instead of an mlid will save some time (which is just a look up in the tree of mcast groups), I can repost the patch. -- Yevgeny > Sasha > >> uint8_t sl; >> cl_map_t full_guid_tbl; >> cl_map_t part_guid_tbl; >> @@ -110,6 +111,10 @@ typedef struct _osm_prtn { >> * pkey >> * The IBA defined P_KEY of this Partition. >> * >> +* mlid >> +* The network ordered LID of the well known Multicast Group >> +* that was created for this partition. >> +* >> * sl >> * The Service Level (SL) associated with this Partiton. >> * >> diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c >> index 2d0b313..187cff6 100644 >> --- a/opensm/opensm/osm_prtn.c >> +++ b/opensm/opensm/osm_prtn.c >> @@ -230,8 +230,10 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, >> OSM_LOG(p_log, OSM_LOG_ERROR, >> "Failed to create MC group with pkey 0x%04x\n", >> cl_ntoh16(pkey)); >> - if (p_mgrp) >> + if (p_mgrp) { >> p_mgrp->well_known = TRUE; >> + p->mlid = p_mgrp->mlid; >> + } >> >> /* workaround for TS */ >> /* FIXME: remove this upon TS fixes */ >> @@ -243,8 +245,11 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, >> >> status = osm_mcmr_rcv_find_or_create_new_mgrp(p_sa, comp_mask, &mc_rec, >> &p_mgrp); >> - if (p_mgrp) >> + if (p_mgrp) { >> p_mgrp->well_known = TRUE; >> + if (!p->mlid) >> + p->mlid = p_mgrp->mlid; >> + } >> >> return status; >> } >> diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c >> index aef1856..1c428e5 100644 >> --- a/opensm/opensm/osm_qos_policy.c >> +++ b/opensm/opensm/osm_qos_policy.c >> @@ -763,6 +763,69 @@ static osm_qos_port_group_t *__qos_policy_get_port_group_by_name( >> /*************************************************** >> ***************************************************/ >> >> +static void __qos_policy_validate_pkey( >> + osm_qos_policy_t * p_qos_policy, >> + osm_qos_match_rule_t * p_qos_match_rule, >> + osm_prtn_t * p_prtn) >> +{ >> + uint8_t sl; >> + uint32_t flow; >> + uint8_t hop; >> + osm_mgrp_t * p_mgrp; >> + >> + if (!p_qos_policy || !p_qos_match_rule || !p_prtn) >> + return; >> + >> + if (!p_qos_match_rule->p_qos_level->sl_set || >> + p_prtn->sl == p_qos_match_rule->p_qos_level->sl) >> + return; >> + >> + /* overriding partition's SL */ >> + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_ERROR, >> + "ERR AC15: pkey 0x%04X in match rule - " >> + "overriding partition SL (%u) with QoS Level SL (%u)\n", >> + cl_ntoh16(p_prtn->pkey), p_prtn->sl, >> + p_qos_match_rule->p_qos_level->sl); >> + p_prtn->sl = p_qos_match_rule->p_qos_level->sl; >> + >> + >> + /* If this partition is an IPoIB partition, there should >> + be a matching MCast group. Fix this group's SL too */ >> + >> + if (!p_prtn->mlid) >> + return; >> + >> + p_mgrp = (osm_mgrp_t *) cl_qmap_get( >> + &p_qos_policy->p_subn->mgrp_mlid_tbl, >> + p_prtn->mlid); >> + if (p_mgrp == (osm_mgrp_t *) >> + cl_qmap_end(&p_qos_policy->p_subn->mgrp_mlid_tbl)) { >> + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_ERROR, >> + "ERR AC16: MCast group for partition with " >> + "pkey 0x%04X not found\n", >> + cl_ntoh16(p_prtn->pkey)); >> + return; >> + } >> + >> + CL_ASSERT((cl_ntoh16(p_mgrp->mcmember_rec.pkey) & 0x7fff) == >> + (cl_ntoh16(p_prtn->pkey) & 0x7fff)); >> + >> + ib_member_get_sl_flow_hop(p_mgrp->mcmember_rec.sl_flow_hop, >> + &sl, &flow, &hop); >> + if (sl != p_prtn->sl) { >> + OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_DEBUG, >> + "Updating MCGroup (MLID 0x%04x) SL to " >> + "match partition SL (%u)\n", >> + cl_hton16(p_mgrp->mcmember_rec.mlid), >> + p_prtn->sl); >> + p_mgrp->mcmember_rec.sl_flow_hop = >> + ib_member_set_sl_flow_hop(p_prtn->sl, flow, hop); >> + } >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, >> osm_log_t *p_log) >> { >> @@ -901,27 +964,16 @@ int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, >> &p_qos_policy->p_subn->prtn_pkey_tbl, pkey); >> >> if (p_prtn == (osm_prtn_t *)cl_qmap_end( >> - &p_qos_policy->p_subn->prtn_pkey_tbl)) { >> + &p_qos_policy->p_subn->prtn_pkey_tbl)) >> /* partition for this pkey not found */ >> OSM_LOG(p_log, OSM_LOG_ERROR, "ERR AC14: " >> "pkey 0x%04X in match rule - " >> "partition doesn't exist\n", >> cl_ntoh16(pkey)); >> - continue; >> - } >> - >> - if (p_qos_match_rule->p_qos_level->sl_set && >> - p_prtn->sl != p_qos_match_rule->p_qos_level->sl) { >> - /* overriding partition's SL */ >> - OSM_LOG(p_log, OSM_LOG_ERROR, "ERR AC15: " >> - "pkey 0x%04X in match rule - " >> - "overriding partition SL (%u) " >> - "with QoS Level SL (%u)\n", >> - cl_ntoh16(pkey), >> - p_prtn->sl, >> - p_qos_match_rule->p_qos_level->sl); >> - p_prtn->sl = p_qos_match_rule->p_qos_level->sl; >> - } >> + else >> + __qos_policy_validate_pkey(p_qos_policy, >> + p_qos_match_rule, >> + p_prtn); >> } >> } >> >> -- >> 1.5.1.4 >> > From kliteyn at dev.mellanox.co.il Mon Mar 24 23:10:03 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 25 Mar 2008 08:10:03 +0200 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <1206375555.8099.464.camel@hrosenstock-ws.xsigo.com> References: <47E67B1F.8050102@dev.mellanox.co.il> <20080324061421.GH2102@sashak.voltaire.com> <47E753BF.9070505@dev.mellanox.co.il> <20080324085407.GA595@sashak.voltaire.com> <47E78395.9020604@dev.mellanox.co.il> <1206375555.8099.464.camel@hrosenstock-ws.xsigo.com> Message-ID: <47E8973B.7080901@dev.mellanox.co.il> Hal Rosenstock wrote: > On Mon, 2008-03-24 at 12:33 +0200, Yevgeny Kliteynik wrote: >> OK, I'll repost the patch for master only. >> For OFED 1.3 the partition SL configuration should >> be done through partition config file only. > > Should that tidbit be noted in the OpenSM documentation/man page > anywhere (for OFED 1.3) ? It will be noted > From kliteyn at mellanox.co.il Mon Mar 24 23:20:35 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 25 Mar 2008 08:20:35 +0200 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos In-Reply-To: <1206375592.8099.466.camel@hrosenstock-ws.xsigo.com> References: <47E7565E.1040206@voltaire.com> <47E766CC.9070905@mellanox.co.il> <47E78657.60105@voltaire.com> <47E7CD07.9060509@mellanox.co.il> <1206375592.8099.466.camel@hrosenstock-ws.xsigo.com> Message-ID: <47E899B3.5020001@mellanox.co.il> Hal Rosenstock wrote: > On Mon, 2008-03-24 at 17:47 +0200, Yevgeny Kliteynik wrote: > >> Or Gerlitz wrote: >> > > >>>> This rule is also valid, providing that you actually have a partition >>>> with pkey 0x8001 configured in the partition cfg file. >>>> >>> OK, so the rule is valid, but would you agree that the word "ipoib" >>> adds nothing to the rule? >>> >> Agree, it's just an alias. >> 'ipoib : ' is useful >> 'any, pkey 0xNNNN' is useful too. >> 'ipoib, pkey 0xNNNN' is just for better readability of the matching rules. >> > > What rule(s) are being discussed here ? QoS annex rules only ? QoS annex > rules in concert with partition rules ? partition rules only ? > It's QoS annex rules only. > AFAIK, ipoib keyword has a specific useful meaning to OpenSM (apart from > SL). > > Sure, in partition cfg file ipoib has a special meaning. But for QoS annex I treat both ipoib and just a pkey the same - I always check whether this pkey belongs to an ipoib-capable partition and update SL in the mcast group. -- Yevgeny > -- Hal > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From mprk03 at yahoo.com.hk Mon Mar 24 21:17:46 2008 From: mprk03 at yahoo.com.hk (Mrs Margareth Troder) Date: Tue, 25 Mar 2008 01:17:46 -0300 (BRT) Subject: [ofa-general] I NEED YOUR HELP/URGENT Message-ID: <62641.195.166.237.254.1206418666.squirrel@eros.unifei.edu.br> Dear Beloved in Christ. It is by the grace of God that I received Christ, having known the truth; I had no choice than to do what is lawful and just in the sight of God for eternal life and in the sight of man for witness of God & His Mercies and glory upon my life. I am Mrs. Margareth Troder, the wife of Mr. Robert Troder, my husband worked with the Chevron/Texaco in Kenya for twenty years before he died in the year 2003.We were married for ten years without a child. My Husband died after a brief illness that lasted for only four days. Before his death we both got born-again and dedicated Christians. Since his death I decided not to re-marry or get a child outside my matrimonial home which the Bible is strongly against. When my late husband was alive he deposited a huge amount with a Bank in Europe and I would let you know on your contacting me. Presently, this money is still with the Bank and the management just wrote me as the beneficiary that our account has been DORMANT and if I, as the beneficiary of the funds, do not re-activate the account; the funds will be CONFISCATED or I rather issue a letter of authorization to somebody to receive it on my behalf (note that you Need to activate this account) as I can not come over. Presently, I’m in a hospital in Kenya where I have been undergoing treatment for esophageal cancer. I have since lost my ability to talk and my doctors have told me that I have only a few weeks to live. It is my last wish to see this money distributed to charity organizations anywhere in the World. Because relatives and friends have plundered so much of my wealth since my illness, I cannot live with the agony of entrusting this huge responsibility to any of them. Please, I beg you in the name of God to help me Stand-in as the beneficiary and collect the Funds from the Bank. I want a person that is God-fearing who will use this money to fund churches, orphanages and widows propagating the word of God and to ensure that the house of God is maintained. The Bible made us to understand that blessed is the hand that giveth. I took this decision because I don't have any child that will inherit this money and my husband's relatives are not Christians and I don't want my husband's hard earned money to be misused by unbelievers. I don't want a situation where this money will be used in an ungodly manner. Hence the reason for taking this bold decision. I am not afraid of death since I know where I am going to. I know that I am going to be in the bosom of the Lord. {Exodus14 VS14} says that the lord will fight my case and I shall hold my peace. I don't need any telephone communication in this regard because of my soundless voice and presence of my husband's relatives around me always. I don't want them to know about this development. Contact via: mparkermp at webdunia.com With God all things are possible. Your Sister in Christ, Mrs. Margareth Troder. From ogerlitz at voltaire.com Tue Mar 25 01:32:09 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 25 Mar 2008 10:32:09 +0200 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <47E75F08.3040007@dev.mellanox.co.il> References: <47E67B1F.8050102@dev.mellanox.co.il> <47E758C8.3060307@voltaire.com> <47E75F08.3040007@dev.mellanox.co.il> Message-ID: <47E8B889.70807@voltaire.com> Yevgeny Kliteynik wrote: > OpenSM re-reads the policy file every heavy sweep and creates new rules. OK, got it. > > The explanation of what the patch is in the beginning of the mail: > > "QoS manager should set the right SL in the IPoIB broadcast > groups to enforce the right SL for IPoIB traffic." yes, but when the patch lands in git, what remains is only the change-log comment and not anything else, so one someone looks on it without this info, what can he/she do? please add it when you resubmit the patch. > > QoS manager finds the IPoIB broadcast group that was created for > pkey that is specified in the qos policy file, and updates the > broadcast (multicast) group's SL to match the SL that was specified > for this partition in the qos policy file. > OK, got it, so just to make sure I got it, without this patch, the QoS (SL) for the per partition ipoib broadcast has to be specified through the partition manager config file, correct? Or. From kliteyn at dev.mellanox.co.il Tue Mar 25 01:35:32 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 25 Mar 2008 10:35:32 +0200 Subject: [ofa-general] Re: [PATCH] opensm/osm_qos_policy.c : set SL for IPoIB broadcast groups In-Reply-To: <47E8B889.70807@voltaire.com> References: <47E67B1F.8050102@dev.mellanox.co.il> <47E758C8.3060307@voltaire.com> <47E75F08.3040007@dev.mellanox.co.il> <47E8B889.70807@voltaire.com> Message-ID: <47E8B954.4040901@dev.mellanox.co.il> Or Gerlitz wrote: > Yevgeny Kliteynik wrote: >> OpenSM re-reads the policy file every heavy sweep and creates new rules. > OK, got it. >> >> The explanation of what the patch is in the beginning of the mail: >> >> "QoS manager should set the right SL in the IPoIB broadcast >> groups to enforce the right SL for IPoIB traffic." > yes, but when the patch lands in git, what remains is only the > change-log comment and not anything else, so one someone looks on it > without this info, what can he/she do? please add it when you resubmit > the patch. >> >> QoS manager finds the IPoIB broadcast group that was created for >> pkey that is specified in the qos policy file, and updates the >> broadcast (multicast) group's SL to match the SL that was specified >> for this partition in the qos policy file. >> > OK, got it, so just to make sure I got it, without this patch, the QoS > (SL) for the per partition ipoib broadcast has to be specified through > the partition manager config file, correct? Correct. -- Yevgeny > Or. > > > > From ogerlitz at voltaire.com Tue Mar 25 01:38:38 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 25 Mar 2008 10:38:38 +0200 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos In-Reply-To: <47E7CD07.9060509@mellanox.co.il> References: <47E7565E.1040206@voltaire.com> <47E766CC.9070905@mellanox.co.il> <47E78657.60105@voltaire.com> <47E7CD07.9060509@mellanox.co.il> Message-ID: <47E8BA0E.8080607@voltaire.com> Yevgeny Kliteynik wrote: > This doc describes OFED components (SDP, RDS, SRP, IPoIB, iSER), > their support of QoS annex, and a way for a QoS manager (OpenSM, > which is part of OFED) all these components are being worked out by bunch of developers and maintainers, ofed is just the framework for temporal distribution of them. > I really don't have any special sentiments for file name, but > is QoS_in_OFED.txt wrong? > You probably meant something like QoS_architecture.txt? yes. > >> OK, so the rule is valid, but would you agree that the word "ipoib" >> adds nothing to the rule? > > Agree, it's just an alias. > 'ipoib : ' is useful > 'any, pkey 0xNNNN' is useful too. > 'ipoib, pkey 0xNNNN' is just for better readability of the matching > rules. When the user wants to posses his QoS management on partitions (i.e associate SL with PKEY provided in a path query) the word ipoib is confusing, this those rules apply to all other ulps that issue path queries (sdp,iser,lustre,rnfs,srp,ipoib,you named it). Or. From rbara.nagi at hlkf.de Mon Mar 24 23:51:24 2008 From: rbara.nagi at hlkf.de (hubey louie) Date: Tue, 25 Mar 2008 06:51:24 +0000 Subject: [ofa-general] New video without cowards Jennifer Aniston Message-ID: <000701c88e53$04003301$5d795098@ghlyos> #WoVPQySalma Hayek Shocking porno dvd. #GdrakaThe mp3 is Full! #oVPQyuOnly 1 day trial - get this New mpeg4 now! #Gdraka Download it now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Tue Mar 25 01:54:57 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 25 Mar 2008 10:54:57 +0200 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos In-Reply-To: <47E8BA0E.8080607@voltaire.com> References: <47E7565E.1040206@voltaire.com> <47E766CC.9070905@mellanox.co.il> <47E78657.60105@voltaire.com> <47E7CD07.9060509@mellanox.co.il> <47E8BA0E.8080607@voltaire.com> Message-ID: <47E8BDE1.1040805@mellanox.co.il> Or Gerlitz wrote: > Yevgeny Kliteynik wrote: >> This doc describes OFED components (SDP, RDS, SRP, IPoIB, iSER), >> their support of QoS annex, and a way for a QoS manager (OpenSM, >> which is part of OFED) > all these components are being worked out by bunch of developers and > maintainers, ofed is just the framework for temporal distribution of > them. >> I really don't have any special sentiments for file name, but >> is QoS_in_OFED.txt wrong? >> You probably meant something like QoS_architecture.txt? > yes. >> >>> OK, so the rule is valid, but would you agree that the word "ipoib" >>> adds nothing to the rule? >> >> Agree, it's just an alias. >> 'ipoib : ' is useful >> 'any, pkey 0xNNNN' is useful too. >> 'ipoib, pkey 0xNNNN' is just for better readability of the matching >> rules. > When the user wants to posses his QoS management on partitions (i.e > associate SL with PKEY provided in a path query) the word ipoib is > confusing, this those rules apply to all other ulps that issue path > queries (sdp,iser,lustre,rnfs,srp,ipoib,you named it). OK, then perhaps "any, pkey 0xNNNN" and "ipoib, pkey 0xNNNN" should have different meaning, and should be treated differently by the QoS manager. Let's look at the IPoIB (the real IPoIB, not the keyword). If I understood your previous main correctly, then IPoIB gets its SL from the multicast group only right? So what kind of PR queries does IPoIB do? Only to MGID? Suppose A wants to send something over IPoIB to B. How will it use the right SL? Does it take it from the mcast group, or does it do a PR query to SM for the dgid of node B (using the right pkey), and takes the SL that os returned in the PR responce? -- Yevgeny > Or. > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From kliteyn at dev.mellanox.co.il Tue Mar 25 02:04:50 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 25 Mar 2008 11:04:50 +0200 Subject: [ofa-general] [PATCH] ofed/docs: fixing wrong syntax in the example in QoS_management_in_OpenSM.txt Message-ID: <47E8C032.2050907@dev.mellanox.co.il> Hi Tziporet, Fixing a wrong syntax in the example in QoS_management_in_OpenSM.txt. Please apply this patch to the OFED 1.3 docs. Signed-off-by: Yevgeny Kliteynik --- QoS_management_in_OpenSM.txt | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/QoS_management_in_OpenSM.txt b/QoS_management_in_OpenSM.txt index 8426a0f..307c80f 100644 --- a/QoS_management_in_OpenSM.txt +++ b/QoS_management_in_OpenSM.txt @@ -354,9 +354,9 @@ of the ULPs is matched: IPoIB query is matched by PKey. Default PKey for IPoIB partition is 0x7fff, so the following three match rules are equivalent: - ipoib : - ipoib, 0x7fff : - any, pkey 0x7fff : + ipoib : + ipoib, pkey 0x7fff : + any, pkey 0x7fff : 6.2 SDP SDP PR query is matched by Service ID. The Service-ID for SDP is -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Tue Mar 25 02:07:43 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 25 Mar 2008 11:07:43 +0200 Subject: [ofa-general] [PATCH] ofed/docs: some fixes in OpenSM release notes Message-ID: <47E8C0DF.6010203@dev.mellanox.co.il> Hi Tziporet, Added ConnectX to list of supported devices, fixed FW versions to match OFED release notes, and removed device IDs (no reason for them to be there, OFED release notes doesn't have them). Please apply OFED 1.3 docs. Signed-off-by: Yevgeny Kliteynik --- opensm_release_notes.txt | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) diff --git a/opensm_release_notes.txt b/opensm_release_notes.txt index 825d758..bdefc11 100644 --- a/opensm_release_notes.txt +++ b/opensm_release_notes.txt @@ -459,14 +459,15 @@ Table 3 - Qualified Devices and Corresponding Firmware ====================================================== Mellanox -Device | FW versions ---------|----------------------------------------------------------- -MT43132 | InfiniScale - fw-43132 5.2.0 (and later) -MT47396 | InfiniScale III - fw-47396 0.5.0 (and later) -MT23108 | InfiniHost - fw-23108 3.3.2 (and later) -MT25204 | InfiniHost III Lx - fw-25204 1.0.1i (and later) -MT25208 | InfiniHost III Ex (InfiniHost Mode) - fw-25208 4.6.2 (and later) -MT25208 | InfiniHost III Ex (MemFree Mode) - fw-25218 5.0.1 (and later) +Device | FW versions +------------------------------------|------------------------------- +InfiniScale | fw-43132 5.2.000 (and later) +InfiniScale III | fw-47396 0.5.000 (and later) +InfiniHost | fw-23108 3.5.000 (and later) +InfiniHost III Lx | fw-25204 1.2.000 (and later) +InfiniHost III Ex (InfiniHost Mode) | fw-25208 4.8.200 (and later) +InfiniHost III Ex (MemFree Mode) | fw-25218 5.3.000 (and later) +ConnectX IB | fw-25408 2.3.000 (and later) QLogic/PathScale Device | Note -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Tue Mar 25 02:12:15 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 25 Mar 2008 11:12:15 +0200 Subject: [ofa-general] Rename the QoS_in_OFED.txt file QoS_architecture.txt in OFED docs. Message-ID: <47E8C1EF.60700@dev.mellanox.co.il> Tziporet, Please rename the QoS_in_OFED.txt file to QoS_architecture.txt in OFED docs. Thanks -- Yevgeny > Or Gerlitz wrote: >> Yevgeny Kliteynik wrote: >>> This doc describes OFED components (SDP, RDS, SRP, IPoIB, iSER), >>> their support of QoS annex, and a way for a QoS manager (OpenSM, >>> which is part of OFED) >> >> all these components are being worked out by bunch of developers and >> maintainers, ofed is just the framework for temporal distribution of >> them. >> >>> I really don't have any special sentiments for file name, but >>> is QoS_in_OFED.txt wrong? >>> You probably meant something like QoS_architecture.txt? >>> >> yes. >> From dotanb at dev.mellanox.co.il Tue Mar 25 02:40:25 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 25 Mar 2008 11:40:25 +0200 Subject: [ofa-general] [PATCH][TRIVIAL] infiniband-diags/saquery.c: Remove extraneous semicolon Message-ID: <200803251140.25413.dotanb@dev.mellanox.co.il> Remove extraneous semicolon. Signed-off-by: Dotan Barak --- diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c index 523a4c6..ed61721 100644 --- a/infiniband-diags/src/saquery.c +++ b/infiniband-diags/src/saquery.c @@ -1596,7 +1596,7 @@ main(int argc, char **argv) requested_lid = (ib_net16_t)strtoul(argv[0], NULL, 0); requested_lid_flag++; } else if (node_print_desc == NAME_OF_GUID) { - requested_guid = (ib_net64_t)strtoul(argv[0], NULL, 0);; + requested_guid = (ib_net64_t)strtoul(argv[0], NULL, 0); requested_guid_flag++; } else { requested_name = argv[0]; From dotanb at dev.mellanox.co.il Tue Mar 25 02:41:56 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 25 Mar 2008 11:41:56 +0200 Subject: [ofa-general] [PATCH][TRIVIAL] libibmad: Remove extraneous semicolon Message-ID: <200803251141.56531.dotanb@dev.mellanox.co.il> Remove extraneous semicolon. Signed-off-by: Dotan Barak --- diff --git a/libibmad/src/fields.c b/libibmad/src/fields.c index 9118729..21f98cf 100644 --- a/libibmad/src/fields.c +++ b/libibmad/src/fields.c @@ -439,7 +439,7 @@ _get_field(void *buf, int base_offs, ib_field_t *f) void _set_array(void *buf, int base_offs, ib_field_t *f, void *val) { - int bitoffs = f->bitoffs;; + int bitoffs = f->bitoffs; if (f->bitlen < 32) bitoffs = BE_TO_BITSOFFS(bitoffs, f->bitlen); @@ -450,7 +450,7 @@ _set_array(void *buf, int base_offs, ib_field_t *f, void *val) void _get_array(void *buf, int base_offs, ib_field_t *f, void *val) { - int bitoffs = f->bitoffs;; + int bitoffs = f->bitoffs; if (f->bitlen < 32) bitoffs = BE_TO_BITSOFFS(bitoffs, f->bitlen); From dotanb at dev.mellanox.co.il Tue Mar 25 02:43:25 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 25 Mar 2008 11:43:25 +0200 Subject: [ofa-general] [PATCH][TRIVIAL] opensm: Remove extraneous semicolon Message-ID: <200803251143.25301.dotanb@dev.mellanox.co.il> Remove extraneous semicolon. Signed-off-by: Dotan Barak --- diff --git a/opensm/opensm/osm_ucast_lash.c b/opensm/opensm/osm_ucast_lash.c index 0735aed..74cdd7f 100644 --- a/opensm/opensm/osm_ucast_lash.c +++ b/opensm/opensm/osm_ucast_lash.c @@ -911,7 +911,7 @@ static int init_lash_structures(lash_t * p_lash) } // initialise num_mst_in_lane[num_switches], default 0 - p_lash->num_mst_in_lane = (int *)malloc(num_switches * sizeof(int));; + p_lash->num_mst_in_lane = (int *)malloc(num_switches * sizeof(int)); if (p_lash->num_mst_in_lane == NULL) goto Exit_Mem_Error; memset(p_lash->num_mst_in_lane, 0, From ogerlitz at voltaire.com Tue Mar 25 02:50:07 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 25 Mar 2008 11:50:07 +0200 Subject: [ofa-general] [PATCH 3/10] IB/core: Add LSO support In-Reply-To: <1206369486.25950.289.camel@mtls03> References: <1205767431.25950.138.camel@mtls03> <47E5FA6D.7070307@voltaire.com> <1206369486.25950.289.camel@mtls03> Message-ID: <47E8CACF.4030007@voltaire.com> Eli Cohen wrote: > On Sun, 2008-03-23 at 08:36 +0200, Or Gerlitz wrote: >> with IB_WC_LSO never being used over this patchset, can we just remove it? > It is used in cq.c. I don't think we can remove it even though IPoIB > does not use this completion type - other clients might want to. How about adding this only when someone really wants it? I don't see more clients using it other than IPoIB serving TCP traffic in the near future. >>> >>> @@ -660,6 +664,9 @@ struct ib_send_wr { >>> + void *header; >>> + int hlen; >>> + int mss; >>> >> Can you add shorting documentation for the new fields? > Of course I can add - I just followed the "spirit" of other parts of the > code where there is not description. Is this an exception in this > regard? yes, this is an exception since all the other fields has strict relation to the IB spec, and not that I think that stateless offloading has to be specc-ed before implemented or specc-ed at all, but its not sure not obvious for someone looking on IB send WQE that is an LSO material. If its easier for you to document it under Documentation/infiniband that's fine as well. Or. From a-andrei at addmorepersonnel.com Tue Mar 25 02:54:33 2008 From: a-andrei at addmorepersonnel.com (Theron Landers) Date: Tue, 25 Mar 2008 17:54:33 +0800 Subject: [ofa-general] What's up? Message-ID: <787588607.60870184171357@addmorepersonnel.com> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Alexis at BestGolova.com only, because I am using my friend's email to write this. If you would like to see my pictures. From ogerlitz at voltaire.com Tue Mar 25 02:56:54 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 25 Mar 2008 11:56:54 +0200 Subject: [ofa-general] [PATCH 4/10] IB/ipoib: Add LSO support to ipoib In-Reply-To: <1206370718.25950.302.camel@mtls03> References: <1205767435.25950.139.camel@mtls03> <47E607D2.5000207@voltaire.com> <1206370718.25950.302.camel@mtls03> Message-ID: <47E8CC66.8080307@voltaire.com> Eli Cohen wrote: > On Sun, 2008-03-23 at 09:33 +0200, Or Gerlitz wrote: >> is the 60 bytes being a limitation of the connectX HW, the Linux kernel >> stack or some "lso spec"? > It's an implementation decision - I assume that I will never get TSO > SKBs where the headers exceed 60 bytes. What does this assumption buys you? do you have to allocate/copy the header and you want to be limited to 60 bytes? can you point me to where this limitation comes into play over this patch set? is there an equivalent design/check in one of the Ethernet drivers supporting LSO? > It looks like I would get the same effect if I'd used e1000 style though > I'm not sure which approach is faster. I think that the e1000 approach is --much-- more readable, I would go on clarity here. >> shouldn't the tx_dropped counter be incremented here? > I am not sure. Does every erroneous tx packets imply incrementing the > drop counter too? I think so, but you can look around in ipoib or other driver or get more opinions. Or. From ogerlitz at voltaire.com Tue Mar 25 02:59:26 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 25 Mar 2008 11:59:26 +0200 Subject: [ofa-general] Re: [PATCH 3/10] IB/core: Add LSO support In-Reply-To: References: <1205767431.25950.138.camel@mtls03> Message-ID: <47E8CCFE.4080807@voltaire.com> Roland Dreier wrote: > > @@ -622,7 +624,8 @@ enum ib_wr_opcode { > > + IB_WR_LSO > > > @@ -630,7 +633,8 @@ enum ib_send_flags { > > + IB_SEND_UDP_LSO = (1<<5) > > What is this send flag used for? It doesn't seem to appear anywhere > else in the patch series and it seems redundant given that you've > defined a new opcode for LSO send requests. > > I did a quick grep of the OFED 1.3 kernel tree and IB_SEND_UDP_LSO > only seems to appear once, in the line where it is defined. > I think it would be more clear to have LSO as a send flag as does the checksum offload. So some work request will be just sends, other sends with checksum offload and others sends with checksum offload and lso, etc, etc. Or. From ogerlitz at voltaire.com Tue Mar 25 03:01:05 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 25 Mar 2008 12:01:05 +0200 Subject: [ofa-general] Re: Remove extraneous semicolon In-Reply-To: <200803251143.25301.dotanb@dev.mellanox.co.il> References: <200803251143.25301.dotanb@dev.mellanox.co.il> Message-ID: <47E8CD61.7010701@voltaire.com> Dotan Barak wrote: > Remove extraneous semicolon. > Hi Dotan, Sasha How about finding all the extraneous semicolons in the management git and sending --one-- patch that cleans them... Or. From ogerlitz at voltaire.com Tue Mar 25 03:43:04 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 25 Mar 2008 12:43:04 +0200 Subject: [ofa-general] some comments/clarifications on ipoib/opensm re qos In-Reply-To: <47E8BDE1.1040805@mellanox.co.il> References: <47E7565E.1040206@voltaire.com> <47E766CC.9070905@mellanox.co.il> <47E78657.60105@voltaire.com> <47E7CD07.9060509@mellanox.co.il> <47E8BA0E.8080607@voltaire.com> <47E8BDE1.1040805@mellanox.co.il> Message-ID: <47E8D738.8090800@voltaire.com> Yevgeny Kliteynik wrote: > Let's look at the IPoIB (the real IPoIB, not the keyword). > If I understood your previous main correctly, then IPoIB gets its SL from > the multicast group only right? So what kind of PR queries does IPoIB do? > Only to MGID? Suppose A wants to send something over IPoIB to B. How will > it use the right SL? Does it take it from the mcast group, or does it do > a PR query to SM for the dgid of node B (using the right pkey), and takes > the SL that os returned in the PR responce? Again, IPoIB does PQ only for unicast traffic, and it provides the following data/component mask bits to the SA , where the relevant info for the QoS scheme deployed today at the opensm is the pkey. So to deploy QoS for IPoIB one needs to set an SL for the partition. Or. From aviram at mellanox.co.il Tue Mar 25 03:44:19 2008 From: aviram at mellanox.co.il (Aviram Gutman) Date: Tue, 25 Mar 2008 12:44:19 +0200 Subject: [ofa-general] RE: [ewg] OFED March 24 meeting summary on OFED 1.4 plans In-Reply-To: References: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9039C9215@mtlexch01.mtl.com> Thanks. What about XRC support? Do you plan it for MVAPICH 1.1? -----Original Message----- From: ewg-bounces at lists.openfabrics.org [mailto:ewg-bounces at lists.openfabrics.org] On Behalf Of Dhabaleswar Panda Sent: Monday, March 24, 2008 11:06 PM To: Tziporet Koren Cc: ewg at lists.openfabrics.org; general at lists.openfabrics.org Subject: Re: [ewg] OFED March 24 meeting summary on OFED 1.4 plans Tziporet, We had some constraints here today and none of us could attend the conference call. > > * MPI: > > * Open MPI 1.3 > > * APM support in MPI > > * mvapich ??? Regarding your question, it will be MVAPICH 1.1 and MVAPICH2 1.1 for OFED 1.4. APM support is already available in MVAPICH and MVAPICH2 for quite some time. Thanks, DK _______________________________________________ ewg mailing list ewg at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg From hrosenstock at xsigo.com Tue Mar 25 03:49:52 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 25 Mar 2008 03:49:52 -0700 Subject: [ofa-general] Re: [PATCH] ofed/docs: some fixes in OpenSM release notes In-Reply-To: <47E8C0DF.6010203@dev.mellanox.co.il> References: <47E8C0DF.6010203@dev.mellanox.co.il> Message-ID: <1206442192.8099.593.camel@hrosenstock-ws.xsigo.com> Yevgeny/Tziporet, On Tue, 2008-03-25 at 11:07 +0200, Yevgeny Kliteynik wrote: > Hi Tziporet, > > Added ConnectX to list of supported devices, fixed FW versions > to match OFED release notes, and removed device IDs (no reason > for them to be there, OFED release notes doesn't have them). > > Please apply OFED 1.3 docs. Shouldn't this come from Sasha's OFED 1.3 OpenSM release notes ? (There were some other minor changes). I think Sasha's latest OFED 1.3 branch needs pulling into OFED by Vlad. -- Hal > Signed-off-by: Yevgeny Kliteynik > --- > opensm_release_notes.txt | 17 +++++++++-------- > 1 files changed, 9 insertions(+), 8 deletions(-) > > diff --git a/opensm_release_notes.txt b/opensm_release_notes.txt > index 825d758..bdefc11 100644 > --- a/opensm_release_notes.txt > +++ b/opensm_release_notes.txt > @@ -459,14 +459,15 @@ Table 3 - Qualified Devices and Corresponding Firmware > ====================================================== > > Mellanox > -Device | FW versions > ---------|----------------------------------------------------------- > -MT43132 | InfiniScale - fw-43132 5.2.0 (and later) > -MT47396 | InfiniScale III - fw-47396 0.5.0 (and later) > -MT23108 | InfiniHost - fw-23108 3.3.2 (and later) > -MT25204 | InfiniHost III Lx - fw-25204 1.0.1i (and later) > -MT25208 | InfiniHost III Ex (InfiniHost Mode) - fw-25208 4.6.2 (and later) > -MT25208 | InfiniHost III Ex (MemFree Mode) - fw-25218 5.0.1 (and later) > +Device | FW versions > +------------------------------------|------------------------------- > +InfiniScale | fw-43132 5.2.000 (and later) > +InfiniScale III | fw-47396 0.5.000 (and later) > +InfiniHost | fw-23108 3.5.000 (and later) > +InfiniHost III Lx | fw-25204 1.2.000 (and later) > +InfiniHost III Ex (InfiniHost Mode) | fw-25208 4.8.200 (and later) > +InfiniHost III Ex (MemFree Mode) | fw-25218 5.3.000 (and later) > +ConnectX IB | fw-25408 2.3.000 (and later) > > QLogic/PathScale > Device | Note From Marcos at ymcanyc.org Tue Mar 25 03:54:55 2008 From: Marcos at ymcanyc.org (Marcos Ingram) Date: Tue, 25 Mar 2008 12:54:55 +0200 Subject: [ofa-general] Feel yourself fine and dandy! Message-ID: <025101c88e66$a956b720$c0a80102@Marcos> also didn't react "hours early when some people startedgovernor and on February 2007 a court from Oregon For certain sure, it's the best selection of discounted posh items you could ever find! Avail yourself of our heavily discounted prices! http://ttebulcs.com/ 0101406020933226910condition, this is to be expected when a baby is born so -------------- next part -------------- An HTML attachment was scrubbed... URL: From Marcos at ymcanyc.org Tue Mar 25 03:55:42 2008 From: Marcos at ymcanyc.org (Marcos Ingram) Date: Tue, 25 Mar 2008 12:55:42 +0200 Subject: [ofa-general] For those who value time and money Message-ID: <02db01c88e66$c525db20$c0a80102@Marcos> force.3. To falsely hide one's opinions or feelings. The quality of these rep1!c at ted items will exceed all your expectations! Get a benefit from our massive discounts! http://ttebulcs.com/ 9862111470298647947Jim Balsillie has resigned as chairman of RIM, the maker -------------- next part -------------- An HTML attachment was scrubbed... URL: From ZacheryagglutinateHebert at cameras101.com Tue Mar 25 22:02:52 2008 From: ZacheryagglutinateHebert at cameras101.com (Errol Horne) Date: Tue, 25 Mar 2008 20:02:52 -0900 Subject: [ofa-general] Hot Shot Stock Report Message-ID: Medical suuplies is one of the hottest sectors currently And the timing is very opportune. With cutting edge technology and state of the ART sortwares to do both improve complex patient care and improve profits for medical practitioners, we believe we have a winner in ZYTO CORP Symbol : ZYTC. Looks like the selling is over, Read the PR's and research ZYTO Corp and see why this is a value stock at bargain price. It's due and ripe for a breakout. Ride the easy 10 bagger Sym: ZYTC -------------- next part -------------- An HTML attachment was scrubbed... URL: From Doris487 at mayridge.com Tue Mar 25 04:09:00 2008 From: Doris487 at mayridge.com (Wilford Shafer) Date: Tue, 25 Mar 2008 12:09:00 +0100 Subject: [ofa-general] =?koi8-r?b?7MXHwczJ2sHDydEg0NLB1yDTz8LT1NfFzs7P09TJ?= Message-ID: <691483551.30565662946249@mayridge.com> Привет, -- Банкротство: преднамеренное банкротство и лжебанкротство; реальная практика действий в интересах любой из сторон; подготовка, вывод финансовых потоков и использование собственности в период банкротства; практика разрешения споров, связанных с применением законодательства о несостоятельности (банкротстве) Весь перечень интересных вопросов и решений, которые Вас интересовали, пожалуйста, смотрите на: http://amatyesha.tripod.com/ С уважением, Антон. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dotanb at dev.mellanox.co.il Tue Mar 25 04:54:48 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 25 Mar 2008 13:54:48 +0200 Subject: [ofa-general] [PATCH V2][TRIVIAL] management: Remove extraneous semicolon from several files Message-ID: <200803251354.48321.dotanb@dev.mellanox.co.il> Remove extraneous semicolon from several components in the management. Signed-off-by: Dotan Barak --- diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c index 523a4c6..ed61721 100644 --- a/infiniband-diags/src/saquery.c +++ b/infiniband-diags/src/saquery.c @@ -1596,7 +1596,7 @@ main(int argc, char **argv) requested_lid = (ib_net16_t)strtoul(argv[0], NULL, 0); requested_lid_flag++; } else if (node_print_desc == NAME_OF_GUID) { - requested_guid = (ib_net64_t)strtoul(argv[0], NULL, 0);; + requested_guid = (ib_net64_t)strtoul(argv[0], NULL, 0); requested_guid_flag++; } else { requested_name = argv[0]; diff --git a/libibmad/src/fields.c b/libibmad/src/fields.c index 9118729..21f98cf 100644 --- a/libibmad/src/fields.c +++ b/libibmad/src/fields.c @@ -439,7 +439,7 @@ _get_field(void *buf, int base_offs, ib_field_t *f) void _set_array(void *buf, int base_offs, ib_field_t *f, void *val) { - int bitoffs = f->bitoffs;; + int bitoffs = f->bitoffs; if (f->bitlen < 32) bitoffs = BE_TO_BITSOFFS(bitoffs, f->bitlen); @@ -450,7 +450,7 @@ _set_array(void *buf, int base_offs, ib_field_t *f, void *val) void _get_array(void *buf, int base_offs, ib_field_t *f, void *val) { - int bitoffs = f->bitoffs;; + int bitoffs = f->bitoffs; if (f->bitlen < 32) bitoffs = BE_TO_BITSOFFS(bitoffs, f->bitlen); diff --git a/opensm/opensm/osm_ucast_lash.c b/opensm/opensm/osm_ucast_lash.c index 0735aed..74cdd7f 100644 --- a/opensm/opensm/osm_ucast_lash.c +++ b/opensm/opensm/osm_ucast_lash.c @@ -911,7 +911,7 @@ static int init_lash_structures(lash_t * p_lash) } // initialise num_mst_in_lane[num_switches], default 0 - p_lash->num_mst_in_lane = (int *)malloc(num_switches * sizeof(int));; + p_lash->num_mst_in_lane = (int *)malloc(num_switches * sizeof(int)); if (p_lash->num_mst_in_lane == NULL) goto Exit_Mem_Error; memset(p_lash->num_mst_in_lane, 0, From d1bolden at tescom-intl.com Tue Mar 25 03:06:41 2008 From: d1bolden at tescom-intl.com (kiel eustace) Date: Tue, 25 Mar 2008 10:06:41 +0000 Subject: [ofa-general] To: openib-general Message-ID: <000501c88e6e$013d6f3e$22fa8092@gsjsyram> Hey have you heard? Finally, the 2008 Collections are in, enjoy 70% OFF Brand Name Shoes & Boots for Men & Women from TOP Fashion Designers. Choose from a variety of the season's hottest models from Gucci, Prada, Chanel, Dior, Ugg Boots, Burberry, D&G, Dsquared & much more. Enter and Save TODAY! Free International Shipping on ALL ORDERS! Click here! Make your way here & Save Today! NoW That's an AMAZING Offer! -------------- next part -------------- An HTML attachment was scrubbed... URL: From dotanb at dev.mellanox.co.il Tue Mar 25 04:57:48 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 25 Mar 2008 13:57:48 +0200 Subject: [ofa-general] Re: Remove extraneous semicolon In-Reply-To: <47E8CD61.7010701@voltaire.com> References: <200803251143.25301.dotanb@dev.mellanox.co.il> <47E8CD61.7010701@voltaire.com> Message-ID: <47E8E8BC.8040908@dev.mellanox.co.il> Or Gerlitz wrote: > Dotan Barak wrote: >> Remove extraneous semicolon. >> > Hi Dotan, Sasha > > How about finding all the extraneous semicolons in the management git > and sending --one-- patch that cleans them... Not a problem, I sent all of the patches that i sent combined in a single one. Dotan From skydiver at c2i.net Tue Mar 25 04:58:43 2008 From: skydiver at c2i.net (EuroSoftware) Date: Tue, 25 Mar 2008 13:58:43 +0200 Subject: [ofa-general] Wir haben alle Arten der Softwareprodukte Message-ID: <01c88e80$56022b80$8fb4955b@skydiver> Brand new software at old priceUnsere Software koennen Sie sofort downloaden! Einfach bezahlen und schon kann das runterladen beginnen. Unsere Programme funktionieren unter Windows und mit Macintosh und sind in allen europaeischen Sprachen verfuegbar. Wir haben die guenstigsten Preise, aber es handelt sich natuerlich nur um Originalversionen, die voellig legal und im vollen Umfang sind.http://geocities.com/colincrawford96/* Windows XP Professional With SP2 Full Version: $59.95 * Office Enterprise 2007: $79.95 * AutoCAD 2008: $129.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://geocities.com/colincrawford96/Unsere Kundenberater sind immer bereit Ihnen bei der Installation zu helfen. Wir antworten sehr schnell und geben Ihnen auch Geld-Zurueck-Garantie! -------------- next part -------------- An HTML attachment was scrubbed... URL: From eli at mellanox.co.il Tue Mar 25 05:39:40 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 14:39:40 +0200 Subject: [ofa-general] [PATCH 4/10] IB/ipoib: Add LSO support to ipoib In-Reply-To: <47E8CC66.8080307@voltaire.com> References: <1205767435.25950.139.camel@mtls03> <47E607D2.5000207@voltaire.com> <1206370718.25950.302.camel@mtls03> <47E8CC66.8080307@voltaire.com> Message-ID: <1206448780.25950.349.camel@mtls03> On Tue, 2008-03-25 at 11:56 +0200, Or Gerlitz wrote: > Eli Cohen wrote: > > On Sun, 2008-03-23 at 09:33 +0200, Or Gerlitz wrote: > >> is the 60 bytes being a limitation of the connectX HW, the Linux kernel > >> stack or some "lso spec"? > > It's an implementation decision - I assume that I will never get TSO > > SKBs where the headers exceed 60 bytes. > What does this assumption buys you? do you have to allocate/copy the > header and you want to be limited to 60 bytes? can you point me to where > this limitation comes into play over this patch set? is there an > equivalent design/check in one of the Ethernet drivers supporting LSO? This is a temporary ConnectX limitation that will be removed in one of the next FW releases. I should have put this check in mlx4 so I will modify the patches accordingly and re-send. Note that I have never seen this assertion on any of the kernels we've been using so I don't know if the network stack will ever pass a TSO SKB with header length exceeding 60 bytes. > > > It looks like I would get the same effect if I'd used e1000 style though > > I'm not sure which approach is faster. > I think that the e1000 approach is --much-- more readable, I would go on > clarity here. OK, I will change this. > >> shouldn't the tx_dropped counter be incremented here? > > I am not sure. Does every erroneous tx packets imply incrementing the > > drop counter too? > I think so, but you can look around in ipoib or other driver or get more > opinions. > ipoib is does not follow this rule - is it a bug? This is from ipoib_ib.c: if (unlikely(ipoib_dma_map_tx(priv->ca, tx_req))) { ++dev->stats.tx_errors; dev_kfree_skb_any(skb); return; } Other drivers do not increment any of these counters. It looks like they read them from directly from hardware registers or ignore them (for example both e1000 and bnx2 assume dma mapping operations never fail). From eli at dev.mellanox.co.il Tue Mar 25 06:30:39 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 15:30:39 +0200 Subject: [ofa-general] [PATCH 4/10 v1] IB/ipoib: Add LSO support to ipoib Message-ID: <1206451839.25950.356.camel@mtls03> >From b9b5c4dacf49e0d77601b27a963f87bf5967d107 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 28 Feb 2008 14:13:45 +0200 Subject: [PATCH] IB/ipoib: Add LSO support to ipoib Signed-off-by: Eli Cohen --- Changes since last post: 1. Remove check that ipoib header does not exceed 60 bytes. 2. Use skb_transport_offset(skb) and tcp_hdrlen(skb) for calculating header length. drivers/infiniband/ulp/ipoib/ipoib.h | 1 + drivers/infiniband/ulp/ipoib/ipoib_cm.c | 7 ++- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 109 ++++++++++++++++++++-------- drivers/infiniband/ulp/ipoib/ipoib_main.c | 10 ++- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 3 + 5 files changed, 95 insertions(+), 35 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 08930ca..19a41ff 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -319,6 +319,7 @@ struct ipoib_dev_priv { struct dentry *mcg_dentry; struct dentry *path_dentry; #endif + int hca_caps; }; struct ipoib_ah { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index edf63dc..90ff2c9 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1384,7 +1384,7 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); - dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG); + dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO); priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; ipoib_flush_paths(dev); @@ -1396,8 +1396,11 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, dev->mtu = min(priv->mcast_mtu, dev->mtu); ipoib_flush_paths(dev); - if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) + if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) { dev->features |= NETIF_F_IP_CSUM | NETIF_F_SG; + if (priv->hca_caps & IB_DEVICE_TCP_TSO) + dev->features |= NETIF_F_TSO; + } return count; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index d13f4fb..5c61a81 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -39,6 +39,8 @@ #include #include +#include +#include #include "ipoib.h" @@ -249,29 +251,37 @@ static int ipoib_dma_map_tx(struct ib_device *ca, struct sk_buff *skb = tx_req->skb; u64 *mapping = tx_req->mapping; int i; + int off; - mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), - DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) - return -EIO; + if (skb_headlen(skb)) { + mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), + DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) + return -EIO; + + off = 1; + } else + off = 0; for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - mapping[i + 1] = ib_dma_map_page(ca, frag->page, + mapping[i + off] = ib_dma_map_page(ca, frag->page, frag->page_offset, frag->size, DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(ca, mapping[i + 1]))) + if (unlikely(ib_dma_mapping_error(ca, mapping[i + off]))) goto partial_error; } return 0; partial_error: - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); - for (; i > 0; --i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i - 1]; - ib_dma_unmap_page(ca, mapping[i], frag->size, DMA_TO_DEVICE); + ib_dma_unmap_page(ca, mapping[i - !off], frag->size, DMA_TO_DEVICE); } + + if (off) + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + return -EIO; } @@ -281,12 +291,17 @@ static void ipoib_dma_unmap_tx(struct ib_device *ca, struct sk_buff *skb = tx_req->skb; u64 *mapping = tx_req->mapping; int i; + int off; - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + if (skb_headlen(skb)) { + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + off = 1; + } else + off = 0; for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - ib_dma_unmap_page(ca, mapping[i + 1], frag->size, + ib_dma_unmap_page(ca, mapping[i + off], frag->size, DMA_TO_DEVICE); } } @@ -392,24 +407,40 @@ void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr) static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - u64 *mapping, int headlen, - skb_frag_t *frags, - int nr_frags) + struct ipoib_tx_buf *tx_req, + void *head, int hlen) { struct ib_send_wr *bad_wr; - int i; + int i, off; + struct sk_buff *skb = tx_req->skb; + skb_frag_t *frags = skb_shinfo(skb)->frags; + int nr_frags = skb_shinfo(skb)->nr_frags; + u64 *mapping = tx_req->mapping; + + if (skb_headlen(skb)) { + priv->tx_sge[0].addr = mapping[0]; + priv->tx_sge[0].length = skb_headlen(skb); + off = 1; + } else + off = 0; - priv->tx_sge[0].addr = mapping[0]; - priv->tx_sge[0].length = headlen; for (i = 0; i < nr_frags; ++i) { - priv->tx_sge[i + 1].addr = mapping[i + 1]; - priv->tx_sge[i + 1].length = frags[i].size; + priv->tx_sge[i + off].addr = mapping[i + off]; + priv->tx_sge[i + off].length = frags[i].size; } - priv->tx_wr.num_sge = nr_frags + 1; + priv->tx_wr.num_sge = nr_frags + off; priv->tx_wr.wr_id = wr_id; priv->tx_wr.wr.ud.remote_qpn = qpn; priv->tx_wr.wr.ud.ah = address; + if (head) { + priv->tx_wr.wr.ud.mss = skb_shinfo(skb)->gso_size; + priv->tx_wr.wr.ud.header = head; + priv->tx_wr.wr.ud.hlen = hlen; + priv->tx_wr.opcode = IB_WR_LSO; + } else + priv->tx_wr.opcode = IB_WR_SEND; + return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); } @@ -418,14 +449,27 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_tx_buf *tx_req; - - if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { - ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", - skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); - ++dev->stats.tx_dropped; - ++dev->stats.tx_errors; - ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); - return; + int hlen; + void *phead; + + if (!skb_is_gso(skb)) { + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); + ++dev->stats.tx_dropped; + ++dev->stats.tx_errors; + ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); + return; + } + phead = 0; + hlen = 0; + } else { + hlen = skb_transport_offset(skb) + tcp_hdrlen(skb); + phead = skb->data; + if (unlikely(!skb_pull(skb, hlen))) { + ipoib_warn(priv, "linear data too small\n"); + goto drop; + } } ipoib_dbg_data(priv, "sending packet, length=%d address=%p qpn=0x%06x\n", @@ -453,8 +497,7 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), address->ah, qpn, - tx_req->mapping, skb_headlen(skb), - skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags))) { + tx_req, phead, hlen))) { ipoib_warn(priv, "post_send failed\n"); ++dev->stats.tx_errors; ipoib_dma_unmap_tx(priv->ca, tx_req); @@ -470,6 +513,12 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, netif_stop_queue(dev); } } + return; + +drop: + ++dev->stats.tx_errors; + dev_kfree_skb_any(skb); + return; } static void __ipoib_reap_ah(struct net_device *dev) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 8d358d2..ca2c6f9 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -1136,14 +1136,15 @@ static struct net_device *ipoib_add_port(const char *format, kfree(device_attr); goto device_init_failed; } + priv->hca_caps = device_attr->device_cap_flags; - if (device_attr->device_cap_flags & IB_DEVICE_UD_IP_CSUM) { + kfree(device_attr); + + if (priv->hca_caps & IB_DEVICE_UD_IP_CSUM) { set_bit(IPOIB_FLAG_CSUM, &priv->flags); priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; } - kfree(device_attr); - /* * Set the full membership bit, so that we join the right * broadcast group, etc. @@ -1178,6 +1179,9 @@ static struct net_device *ipoib_add_port(const char *format, goto event_failed; } + if (priv->dev->features & NETIF_F_SG && priv->hca_caps & IB_DEVICE_TCP_TSO) + priv->dev->features |= NETIF_F_TSO; + result = register_netdev(priv->dev); if (result) { printk(KERN_WARNING "%s: couldn't register ipoib port %d; error %d\n", diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index a3aeb91..1d59f27 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -192,6 +192,9 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) init_attr.send_cq = priv->cq; init_attr.recv_cq = priv->cq; + if (priv->hca_caps & IB_DEVICE_TCP_TSO) + init_attr.create_flags = QP_CREATE_LSO; + if (dev->features & NETIF_F_SG) init_attr.cap.max_send_sge = MAX_SKB_FRAGS + 1; -- 1.5.4.4 From eli at dev.mellanox.co.il Tue Mar 25 06:35:12 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 15:35:12 +0200 Subject: [ofa-general] [PATCH 6/10 v1] IB/mlx4: Add LSO support Message-ID: <1206452112.25950.360.camel@mtls03> >From 496515bb85afddd4918f19e288fa96de24a11957 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Tue, 11 Mar 2008 15:26:38 +0200 Subject: [PATCH] IB/mlx4: Add LSO support Add LSO support to mlx4 driver such that it will be able to send SKBs passed from the driver which publish NETIF_TSO. Signed-off-by: Eli Cohen --- Changes since last post: 1. Verify that header length does not exceed 60 bytes. 2. Remove unnecessary printk calls drivers/infiniband/hw/mlx4/cq.c | 3 ++ drivers/infiniband/hw/mlx4/main.c | 3 ++ drivers/infiniband/hw/mlx4/qp.c | 64 ++++++++++++++++++++++++++++++++++-- drivers/net/mlx4/fw.c | 9 +++++ drivers/net/mlx4/fw.h | 1 + drivers/net/mlx4/main.c | 1 + include/linux/mlx4/device.h | 1 + include/linux/mlx4/qp.h | 5 +++ 8 files changed, 83 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index d2e32b0..7d70af7 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -420,6 +420,9 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, case MLX4_OPCODE_BIND_MW: wc->opcode = IB_WC_BIND_MW; break; + case MLX4_OPCODE_LSO: + wc->opcode = IB_WC_LSO; + break; } } else { wc->byte_len = be32_to_cpu(cqe->byte_cnt); diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 6ea4746..7a24b85 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -101,6 +101,9 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) props->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; + if (dev->dev->caps.max_gso_sz) + props->device_cap_flags |= IB_DEVICE_TCP_TSO; + props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 31b2b5b..fd19f80 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -71,6 +71,7 @@ enum { static const __be32 mlx4_ib_opcode[] = { [IB_WR_SEND] = __constant_cpu_to_be32(MLX4_OPCODE_SEND), + [IB_WR_LSO] = __constant_cpu_to_be32(MLX4_OPCODE_LSO), [IB_WR_SEND_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_SEND_IMM), [IB_WR_RDMA_WRITE] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE), [IB_WR_RDMA_WRITE_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE_IMM), @@ -311,6 +312,7 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, enum ib_qp_type type, struct mlx4_ib_qp *qp) { int s; + int reserve; /* Sanity check SQ size before proceeding */ if (cap->max_send_wr > dev->dev->caps.max_wqes || @@ -327,9 +329,11 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, cap->max_send_sge + 2 > dev->dev->caps.max_sq_sg) return -EINVAL; + reserve = qp->flags & MLX4_QP_LSO ? 64 : 0; + s = max(cap->max_send_sge * sizeof (struct mlx4_wqe_data_seg), cap->max_inline_data + sizeof (struct mlx4_wqe_inline_seg)) + - send_wqe_overhead(type); + send_wqe_overhead(type) + reserve; /* * Hermon supports shrinking WQEs, such that a single work @@ -393,7 +397,7 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, ++qp->sq.wqe_shift; } - qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - + qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - reserve - send_wqe_overhead(type)) / sizeof (struct mlx4_wqe_data_seg); qp->buf_size = (qp->rq.wqe_cnt << qp->rq.wqe_shift) + @@ -503,6 +507,9 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } else { qp->sq_no_prefetch = 0; + if (init_attr->create_flags & QP_CREATE_LSO) + qp->flags |= MLX4_QP_LSO; + err = set_kernel_sq_size(dev, &init_attr->cap, init_attr->qp_type, qp); if (err) goto err; @@ -876,9 +883,15 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, } } - if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI || - ibqp->qp_type == IB_QPT_UD) + if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI) context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + else if (ibqp->qp_type == IB_QPT_UD) { + if (qp->flags & MLX4_QP_LSO) + context->mtu_msgmax = (IB_MTU_4096 << 5) | + ilog2(dev->dev->caps.max_gso_sz); + else + context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + } else if (attr_mask & IB_QP_PATH_MTU) { if (attr->path_mtu < IB_MTU_256 || attr->path_mtu > IB_MTU_4096) { printk(KERN_ERR "path MTU (%u) is invalid\n", @@ -1396,6 +1409,36 @@ static void __set_data_seg(struct mlx4_wqe_data_seg *dseg, struct ib_sge *sg) dseg->addr = cpu_to_be64(sg->addr); } +static int build_lso_seg(struct mlx4_lso_seg *wqe, struct ib_send_wr *wr, + struct mlx4_ib_qp *qp, int *lso_seg_len) +{ + int halign; + + halign = ALIGN(wr->wr.ud.hlen, 16); + + /* + * this is a temporary limitation and will be removed in + * a forthcoming FW release + */ + if (unlikely(wr->wr.ud.hlen) > 60) + return -EINVAL; + + if (unlikely(!(qp->flags & MLX4_QP_LSO) && wr->num_sge > qp->sq.max_gs - (halign >> 4))) + return -EINVAL; + + memcpy(wqe->header, wr->wr.ud.header, wr->wr.ud.hlen); + + /* make sure LSO header is written before + overwriting stamping */ + wmb(); + + wqe->mss_hdr_size = cpu_to_be32(((wr->wr.ud.mss - wr->wr.ud.hlen) + << 16) | wr->wr.ud.hlen); + + *lso_seg_len = halign; + return 0; +} + int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, struct ib_send_wr **bad_wr) { @@ -1487,6 +1530,19 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, set_datagram_seg(wqe, wr); wqe += sizeof (struct mlx4_wqe_datagram_seg); size += sizeof (struct mlx4_wqe_datagram_seg) / 16; + + if (wr->opcode == IB_WR_LSO) { + int hlen; + + err = build_lso_seg(wqe, wr, qp, &hlen); + if (err) { + *bad_wr = wr; + goto out; + } + wqe += hlen; + size += hlen >> 4; + } + break; case IB_QPT_SMI: diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index f494c3e..d82f275 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -133,6 +133,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) #define QUERY_DEV_CAP_MAX_AV_OFFSET 0x27 #define QUERY_DEV_CAP_MAX_REQ_QP_OFFSET 0x29 #define QUERY_DEV_CAP_MAX_RES_QP_OFFSET 0x2b +#define QUERY_DEV_CAP_MAX_GSO_OFFSET 0x2d #define QUERY_DEV_CAP_MAX_RDMA_OFFSET 0x2f #define QUERY_DEV_CAP_RSZ_SRQ_OFFSET 0x33 #define QUERY_DEV_CAP_ACK_DELAY_OFFSET 0x35 @@ -215,6 +216,13 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_requester_per_qp = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RES_QP_OFFSET); dev_cap->max_responder_per_qp = 1 << (field & 0x3f); + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_GSO_OFFSET); + field &= 0x1f; + if (!field) + dev_cap->max_gso_sz = 0; + else + dev_cap->max_gso_sz = 1 << field; + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RDMA_OFFSET); dev_cap->max_rdma_global = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_ACK_DELAY_OFFSET); @@ -377,6 +385,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_sq_desc_sz, dev_cap->max_sq_sg); mlx4_dbg(dev, "Max RQ desc size: %d, max RQ S/G: %d\n", dev_cap->max_rq_desc_sz, dev_cap->max_rq_sg); + mlx4_dbg(dev, "Max GSO size: %d\n", dev_cap->max_gso_sz); dump_dev_cap_flags(dev, dev_cap->flags); diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index e16dec8..306cb9b 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -96,6 +96,7 @@ struct mlx4_dev_cap { u8 bmme_flags; u32 reserved_lkey; u64 max_icm_sz; + int max_gso_sz; }; struct mlx4_adapter { diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index 08bfc13..7cfbe75 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -159,6 +159,7 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.page_size_cap = ~(u32) (dev_cap->min_page_sz - 1); dev->caps.flags = dev_cap->flags; dev->caps.stat_rate_support = dev_cap->stat_rate_support; + dev->caps.max_gso_sz = dev_cap->max_gso_sz; return 0; } diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 6cdf813..ff7df1a 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -186,6 +186,7 @@ struct mlx4_caps { u32 flags; u16 stat_rate_support; u8 port_width_cap[MLX4_MAX_PORTS + 1]; + int max_gso_sz; }; struct mlx4_buf_list { diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 31f9eb3..cf0bf4e 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -219,6 +219,11 @@ struct mlx4_wqe_datagram_seg { __be32 reservd[2]; }; +struct mlx4_lso_seg { + __be32 mss_hdr_size; + __be32 header[0]; +}; + struct mlx4_wqe_bind_seg { __be32 flags1; __be32 flags2; -- 1.5.4.4 From eli at dev.mellanox.co.il Tue Mar 25 06:35:12 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 15:35:12 +0200 Subject: [ofa-general] [PATCH 6/10 v1] IB/mlx4: Add LSO support Message-ID: <1206452112.25950.360.camel@mtls03> >From 496515bb85afddd4918f19e288fa96de24a11957 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Tue, 11 Mar 2008 15:26:38 +0200 Subject: [PATCH] IB/mlx4: Add LSO support Add LSO support to mlx4 driver such that it will be able to send SKBs passed from the driver which publish NETIF_TSO. Signed-off-by: Eli Cohen --- Changes since last post: 1. Verify that header length does not exceed 60 bytes. 2. Remove unnecessary printk calls drivers/infiniband/hw/mlx4/cq.c | 3 ++ drivers/infiniband/hw/mlx4/main.c | 3 ++ drivers/infiniband/hw/mlx4/qp.c | 64 ++++++++++++++++++++++++++++++++++-- drivers/net/mlx4/fw.c | 9 +++++ drivers/net/mlx4/fw.h | 1 + drivers/net/mlx4/main.c | 1 + include/linux/mlx4/device.h | 1 + include/linux/mlx4/qp.h | 5 +++ 8 files changed, 83 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index d2e32b0..7d70af7 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -420,6 +420,9 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, case MLX4_OPCODE_BIND_MW: wc->opcode = IB_WC_BIND_MW; break; + case MLX4_OPCODE_LSO: + wc->opcode = IB_WC_LSO; + break; } } else { wc->byte_len = be32_to_cpu(cqe->byte_cnt); diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index 6ea4746..7a24b85 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -101,6 +101,9 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) props->device_cap_flags |= IB_DEVICE_UD_IP_CSUM; + if (dev->dev->caps.max_gso_sz) + props->device_cap_flags |= IB_DEVICE_TCP_TSO; + props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 31b2b5b..fd19f80 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -71,6 +71,7 @@ enum { static const __be32 mlx4_ib_opcode[] = { [IB_WR_SEND] = __constant_cpu_to_be32(MLX4_OPCODE_SEND), + [IB_WR_LSO] = __constant_cpu_to_be32(MLX4_OPCODE_LSO), [IB_WR_SEND_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_SEND_IMM), [IB_WR_RDMA_WRITE] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE), [IB_WR_RDMA_WRITE_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE_IMM), @@ -311,6 +312,7 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, enum ib_qp_type type, struct mlx4_ib_qp *qp) { int s; + int reserve; /* Sanity check SQ size before proceeding */ if (cap->max_send_wr > dev->dev->caps.max_wqes || @@ -327,9 +329,11 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, cap->max_send_sge + 2 > dev->dev->caps.max_sq_sg) return -EINVAL; + reserve = qp->flags & MLX4_QP_LSO ? 64 : 0; + s = max(cap->max_send_sge * sizeof (struct mlx4_wqe_data_seg), cap->max_inline_data + sizeof (struct mlx4_wqe_inline_seg)) + - send_wqe_overhead(type); + send_wqe_overhead(type) + reserve; /* * Hermon supports shrinking WQEs, such that a single work @@ -393,7 +397,7 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, ++qp->sq.wqe_shift; } - qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - + qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - reserve - send_wqe_overhead(type)) / sizeof (struct mlx4_wqe_data_seg); qp->buf_size = (qp->rq.wqe_cnt << qp->rq.wqe_shift) + @@ -503,6 +507,9 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, } else { qp->sq_no_prefetch = 0; + if (init_attr->create_flags & QP_CREATE_LSO) + qp->flags |= MLX4_QP_LSO; + err = set_kernel_sq_size(dev, &init_attr->cap, init_attr->qp_type, qp); if (err) goto err; @@ -876,9 +883,15 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, } } - if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI || - ibqp->qp_type == IB_QPT_UD) + if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI) context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + else if (ibqp->qp_type == IB_QPT_UD) { + if (qp->flags & MLX4_QP_LSO) + context->mtu_msgmax = (IB_MTU_4096 << 5) | + ilog2(dev->dev->caps.max_gso_sz); + else + context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + } else if (attr_mask & IB_QP_PATH_MTU) { if (attr->path_mtu < IB_MTU_256 || attr->path_mtu > IB_MTU_4096) { printk(KERN_ERR "path MTU (%u) is invalid\n", @@ -1396,6 +1409,36 @@ static void __set_data_seg(struct mlx4_wqe_data_seg *dseg, struct ib_sge *sg) dseg->addr = cpu_to_be64(sg->addr); } +static int build_lso_seg(struct mlx4_lso_seg *wqe, struct ib_send_wr *wr, + struct mlx4_ib_qp *qp, int *lso_seg_len) +{ + int halign; + + halign = ALIGN(wr->wr.ud.hlen, 16); + + /* + * this is a temporary limitation and will be removed in + * a forthcoming FW release + */ + if (unlikely(wr->wr.ud.hlen) > 60) + return -EINVAL; + + if (unlikely(!(qp->flags & MLX4_QP_LSO) && wr->num_sge > qp->sq.max_gs - (halign >> 4))) + return -EINVAL; + + memcpy(wqe->header, wr->wr.ud.header, wr->wr.ud.hlen); + + /* make sure LSO header is written before + overwriting stamping */ + wmb(); + + wqe->mss_hdr_size = cpu_to_be32(((wr->wr.ud.mss - wr->wr.ud.hlen) + << 16) | wr->wr.ud.hlen); + + *lso_seg_len = halign; + return 0; +} + int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, struct ib_send_wr **bad_wr) { @@ -1487,6 +1530,19 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, set_datagram_seg(wqe, wr); wqe += sizeof (struct mlx4_wqe_datagram_seg); size += sizeof (struct mlx4_wqe_datagram_seg) / 16; + + if (wr->opcode == IB_WR_LSO) { + int hlen; + + err = build_lso_seg(wqe, wr, qp, &hlen); + if (err) { + *bad_wr = wr; + goto out; + } + wqe += hlen; + size += hlen >> 4; + } + break; case IB_QPT_SMI: diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c index f494c3e..d82f275 100644 --- a/drivers/net/mlx4/fw.c +++ b/drivers/net/mlx4/fw.c @@ -133,6 +133,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) #define QUERY_DEV_CAP_MAX_AV_OFFSET 0x27 #define QUERY_DEV_CAP_MAX_REQ_QP_OFFSET 0x29 #define QUERY_DEV_CAP_MAX_RES_QP_OFFSET 0x2b +#define QUERY_DEV_CAP_MAX_GSO_OFFSET 0x2d #define QUERY_DEV_CAP_MAX_RDMA_OFFSET 0x2f #define QUERY_DEV_CAP_RSZ_SRQ_OFFSET 0x33 #define QUERY_DEV_CAP_ACK_DELAY_OFFSET 0x35 @@ -215,6 +216,13 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_requester_per_qp = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RES_QP_OFFSET); dev_cap->max_responder_per_qp = 1 << (field & 0x3f); + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_GSO_OFFSET); + field &= 0x1f; + if (!field) + dev_cap->max_gso_sz = 0; + else + dev_cap->max_gso_sz = 1 << field; + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RDMA_OFFSET); dev_cap->max_rdma_global = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_ACK_DELAY_OFFSET); @@ -377,6 +385,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev_cap->max_sq_desc_sz, dev_cap->max_sq_sg); mlx4_dbg(dev, "Max RQ desc size: %d, max RQ S/G: %d\n", dev_cap->max_rq_desc_sz, dev_cap->max_rq_sg); + mlx4_dbg(dev, "Max GSO size: %d\n", dev_cap->max_gso_sz); dump_dev_cap_flags(dev, dev_cap->flags); diff --git a/drivers/net/mlx4/fw.h b/drivers/net/mlx4/fw.h index e16dec8..306cb9b 100644 --- a/drivers/net/mlx4/fw.h +++ b/drivers/net/mlx4/fw.h @@ -96,6 +96,7 @@ struct mlx4_dev_cap { u8 bmme_flags; u32 reserved_lkey; u64 max_icm_sz; + int max_gso_sz; }; struct mlx4_adapter { diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index 08bfc13..7cfbe75 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c @@ -159,6 +159,7 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap) dev->caps.page_size_cap = ~(u32) (dev_cap->min_page_sz - 1); dev->caps.flags = dev_cap->flags; dev->caps.stat_rate_support = dev_cap->stat_rate_support; + dev->caps.max_gso_sz = dev_cap->max_gso_sz; return 0; } diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 6cdf813..ff7df1a 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -186,6 +186,7 @@ struct mlx4_caps { u32 flags; u16 stat_rate_support; u8 port_width_cap[MLX4_MAX_PORTS + 1]; + int max_gso_sz; }; struct mlx4_buf_list { diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h index 31f9eb3..cf0bf4e 100644 --- a/include/linux/mlx4/qp.h +++ b/include/linux/mlx4/qp.h @@ -219,6 +219,11 @@ struct mlx4_wqe_datagram_seg { __be32 reservd[2]; }; +struct mlx4_lso_seg { + __be32 mss_hdr_size; + __be32 header[0]; +}; + struct mlx4_wqe_bind_seg { __be32 flags1; __be32 flags2; -- 1.5.4.4 From eli at dev.mellanox.co.il Tue Mar 25 06:30:39 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 15:30:39 +0200 Subject: [ofa-general] [PATCH 4/10 v1] IB/ipoib: Add LSO support to ipoib Message-ID: <1206451839.25950.356.camel@mtls03> >From b9b5c4dacf49e0d77601b27a963f87bf5967d107 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 28 Feb 2008 14:13:45 +0200 Subject: [PATCH] IB/ipoib: Add LSO support to ipoib Signed-off-by: Eli Cohen --- Changes since last post: 1. Remove check that ipoib header does not exceed 60 bytes. 2. Use skb_transport_offset(skb) and tcp_hdrlen(skb) for calculating header length. drivers/infiniband/ulp/ipoib/ipoib.h | 1 + drivers/infiniband/ulp/ipoib/ipoib_cm.c | 7 ++- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 109 ++++++++++++++++++++-------- drivers/infiniband/ulp/ipoib/ipoib_main.c | 10 ++- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 3 + 5 files changed, 95 insertions(+), 35 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 08930ca..19a41ff 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -319,6 +319,7 @@ struct ipoib_dev_priv { struct dentry *mcg_dentry; struct dentry *path_dentry; #endif + int hca_caps; }; struct ipoib_ah { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index edf63dc..90ff2c9 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1384,7 +1384,7 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); - dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG); + dev->features &= ~(NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_TSO); priv->tx_wr.send_flags &= ~IB_SEND_IP_CSUM; ipoib_flush_paths(dev); @@ -1396,8 +1396,11 @@ static ssize_t set_mode(struct device *d, struct device_attribute *attr, dev->mtu = min(priv->mcast_mtu, dev->mtu); ipoib_flush_paths(dev); - if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) + if (test_bit(IPOIB_FLAG_CSUM, &priv->flags)) { dev->features |= NETIF_F_IP_CSUM | NETIF_F_SG; + if (priv->hca_caps & IB_DEVICE_TCP_TSO) + dev->features |= NETIF_F_TSO; + } return count; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index d13f4fb..5c61a81 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -39,6 +39,8 @@ #include #include +#include +#include #include "ipoib.h" @@ -249,29 +251,37 @@ static int ipoib_dma_map_tx(struct ib_device *ca, struct sk_buff *skb = tx_req->skb; u64 *mapping = tx_req->mapping; int i; + int off; - mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), - DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) - return -EIO; + if (skb_headlen(skb)) { + mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), + DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) + return -EIO; + + off = 1; + } else + off = 0; for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - mapping[i + 1] = ib_dma_map_page(ca, frag->page, + mapping[i + off] = ib_dma_map_page(ca, frag->page, frag->page_offset, frag->size, DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(ca, mapping[i + 1]))) + if (unlikely(ib_dma_mapping_error(ca, mapping[i + off]))) goto partial_error; } return 0; partial_error: - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); - for (; i > 0; --i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i - 1]; - ib_dma_unmap_page(ca, mapping[i], frag->size, DMA_TO_DEVICE); + ib_dma_unmap_page(ca, mapping[i - !off], frag->size, DMA_TO_DEVICE); } + + if (off) + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + return -EIO; } @@ -281,12 +291,17 @@ static void ipoib_dma_unmap_tx(struct ib_device *ca, struct sk_buff *skb = tx_req->skb; u64 *mapping = tx_req->mapping; int i; + int off; - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + if (skb_headlen(skb)) { + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + off = 1; + } else + off = 0; for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - ib_dma_unmap_page(ca, mapping[i + 1], frag->size, + ib_dma_unmap_page(ca, mapping[i + off], frag->size, DMA_TO_DEVICE); } } @@ -392,24 +407,40 @@ void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr) static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - u64 *mapping, int headlen, - skb_frag_t *frags, - int nr_frags) + struct ipoib_tx_buf *tx_req, + void *head, int hlen) { struct ib_send_wr *bad_wr; - int i; + int i, off; + struct sk_buff *skb = tx_req->skb; + skb_frag_t *frags = skb_shinfo(skb)->frags; + int nr_frags = skb_shinfo(skb)->nr_frags; + u64 *mapping = tx_req->mapping; + + if (skb_headlen(skb)) { + priv->tx_sge[0].addr = mapping[0]; + priv->tx_sge[0].length = skb_headlen(skb); + off = 1; + } else + off = 0; - priv->tx_sge[0].addr = mapping[0]; - priv->tx_sge[0].length = headlen; for (i = 0; i < nr_frags; ++i) { - priv->tx_sge[i + 1].addr = mapping[i + 1]; - priv->tx_sge[i + 1].length = frags[i].size; + priv->tx_sge[i + off].addr = mapping[i + off]; + priv->tx_sge[i + off].length = frags[i].size; } - priv->tx_wr.num_sge = nr_frags + 1; + priv->tx_wr.num_sge = nr_frags + off; priv->tx_wr.wr_id = wr_id; priv->tx_wr.wr.ud.remote_qpn = qpn; priv->tx_wr.wr.ud.ah = address; + if (head) { + priv->tx_wr.wr.ud.mss = skb_shinfo(skb)->gso_size; + priv->tx_wr.wr.ud.header = head; + priv->tx_wr.wr.ud.hlen = hlen; + priv->tx_wr.opcode = IB_WR_LSO; + } else + priv->tx_wr.opcode = IB_WR_SEND; + return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); } @@ -418,14 +449,27 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_tx_buf *tx_req; - - if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { - ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", - skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); - ++dev->stats.tx_dropped; - ++dev->stats.tx_errors; - ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); - return; + int hlen; + void *phead; + + if (!skb_is_gso(skb)) { + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); + ++dev->stats.tx_dropped; + ++dev->stats.tx_errors; + ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); + return; + } + phead = 0; + hlen = 0; + } else { + hlen = skb_transport_offset(skb) + tcp_hdrlen(skb); + phead = skb->data; + if (unlikely(!skb_pull(skb, hlen))) { + ipoib_warn(priv, "linear data too small\n"); + goto drop; + } } ipoib_dbg_data(priv, "sending packet, length=%d address=%p qpn=0x%06x\n", @@ -453,8 +497,7 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), address->ah, qpn, - tx_req->mapping, skb_headlen(skb), - skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags))) { + tx_req, phead, hlen))) { ipoib_warn(priv, "post_send failed\n"); ++dev->stats.tx_errors; ipoib_dma_unmap_tx(priv->ca, tx_req); @@ -470,6 +513,12 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, netif_stop_queue(dev); } } + return; + +drop: + ++dev->stats.tx_errors; + dev_kfree_skb_any(skb); + return; } static void __ipoib_reap_ah(struct net_device *dev) diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 8d358d2..ca2c6f9 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -1136,14 +1136,15 @@ static struct net_device *ipoib_add_port(const char *format, kfree(device_attr); goto device_init_failed; } + priv->hca_caps = device_attr->device_cap_flags; - if (device_attr->device_cap_flags & IB_DEVICE_UD_IP_CSUM) { + kfree(device_attr); + + if (priv->hca_caps & IB_DEVICE_UD_IP_CSUM) { set_bit(IPOIB_FLAG_CSUM, &priv->flags); priv->dev->features |= NETIF_F_SG | NETIF_F_IP_CSUM; } - kfree(device_attr); - /* * Set the full membership bit, so that we join the right * broadcast group, etc. @@ -1178,6 +1179,9 @@ static struct net_device *ipoib_add_port(const char *format, goto event_failed; } + if (priv->dev->features & NETIF_F_SG && priv->hca_caps & IB_DEVICE_TCP_TSO) + priv->dev->features |= NETIF_F_TSO; + result = register_netdev(priv->dev); if (result) { printk(KERN_WARNING "%s: couldn't register ipoib port %d; error %d\n", diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index a3aeb91..1d59f27 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -192,6 +192,9 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) init_attr.send_cq = priv->cq; init_attr.recv_cq = priv->cq; + if (priv->hca_caps & IB_DEVICE_TCP_TSO) + init_attr.create_flags = QP_CREATE_LSO; + if (dev->features & NETIF_F_SG) init_attr.cap.max_send_sge = MAX_SKB_FRAGS + 1; -- 1.5.4.4 From hvauhfval at boggemann.com Tue Mar 25 07:23:54 2008 From: hvauhfval at boggemann.com (Taylor Sylvester) Date: Tue, 25 Mar 2008 21:23:54 +0700 Subject: [ofa-general] Master in bed games Message-ID: <01c88ebe$87719f00$aea80775@hvauhfval> Take her to seven heaven Look attached details and find MORE! -------------- next part -------------- A non-text attachment was scrubbed... Name: vo.zip Type: application/zip Size: 636 bytes Desc: not available URL: From weiny2 at llnl.gov Tue Mar 25 09:05:49 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 25 Mar 2008 09:05:49 -0700 Subject: [ofa-general] [PATCH 0/5] trap 144, node descriptor change handling. In-Reply-To: <1206192814.8099.375.camel@hrosenstock-ws.xsigo.com> References: <20080321155112.289cf10f.weiny2@llnl.gov> <1206192814.8099.375.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080325090549.36598500.weiny2@llnl.gov> On Sat, 22 Mar 2008 06:33:34 -0700 Hal Rosenstock wrote: > On Fri, 2008-03-21 at 15:51 -0700, Ira Weiny wrote: > > I have altered these patches based on input from Hal. There are now 5 of them. > > > > 1) Enhance set_nodedesc.sh > > 2) Add optional ibsendtrap test tool and create --enable-test-utils > > configure option > > 3) Add mcm_rereg_test to optional test tool config option > > (This also updates the tool to use the new ibmad_gid_t since it would not > > compile otherwise.) > > 4) Update OpenSM to respond to node description change flag > > 5) Create osm_node_desc.h and move extern declarations into this common > > header. > > Oh, I was looking at an older OpenSM when I suggested adding these to > osm_node_desc_rcv.h. OFED 1.3 has this but master doesn't. NP; In this case I think it works since we now have multiple functions being exported from that c file. Thanks, Ira > > -- Hal > > > This includes Hal's minor stylistic fixes as well as makes ibsendtrap an > > optional testing util, rather than a default diag. > > > > Thanks, > > Ira > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From vlad at dev.mellanox.co.il Tue Mar 25 09:09:09 2008 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 25 Mar 2008 18:09:09 +0200 Subject: [ofa-general] [PATCH 0/1] MLX4: Added resize_cq capability. Message-ID: <47E923A5.3020104@dev.mellanox.co.il> Hi Roland, Following this mail I am sending the resize cq patches for MLX4. 0001-MLX4-Added-resize_cq-capability.patch - I would like to get it into 2.6.26. - This patch depends on 0008-IB-core-Add-support-for-modify-CQ.patch from Eli Cohen. http://lists.openfabrics.org/pipermail/general/2008-March/047972.html 0001-Added-resize-CQ-capability.patch - for libmlx4 Regards, Vladimir From vlad at dev.mellanox.co.il Tue Mar 25 09:09:46 2008 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 25 Mar 2008 18:09:46 +0200 Subject: [ofa-general] [PATCH 1/1] MLX4: Added resize_cq capability. Message-ID: <47E923CA.90804@dev.mellanox.co.il> From 644e491927e900337785ed664cc141681f7c730f Mon Sep 17 00:00:00 2001 From: Vladimir Sokolovsky Date: Tue, 25 Mar 2008 10:23:09 +0200 Subject: [PATCH] MLX4: Added resize_cq capability. Signed-off-by: Vladimir Sokolovsky --- drivers/infiniband/hw/mlx4/cq.c | 298 ++++++++++++++++++++++++++++++++++ drivers/infiniband/hw/mlx4/main.c | 2 + drivers/infiniband/hw/mlx4/mlx4_ib.h | 13 ++ 3 files changed, 313 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 401178f..9302502 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -36,6 +36,14 @@ #include "mlx4_ib.h" #include "user.h" +enum { + MLX4_MAX_DIRECT_CQ_SIZE = 2 * PAGE_SIZE +}; + +enum { + MLX4_CQ_ENTRY_SIZE = 0x20 +}; + static void mlx4_ib_cq_comp(struct mlx4_cq *cq) { struct ib_cq *ibcq = &to_mibcq(cq)->ibcq; @@ -124,7 +132,11 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector entries = roundup_pow_of_two(entries + 1); cq->ibcq.cqe = entries - 1; buf_size = entries * sizeof (struct mlx4_cqe); + mutex_init(&cq->resize_mutex); spin_lock_init(&cq->lock); + cq->is_kernel = !context; + cq->resize_buf = NULL; + cq->resize_umem = NULL; if (context) { struct mlx4_ib_create_cq ucmd; @@ -223,6 +235,238 @@ err_cq: return ERR_PTR(err); } +static int mlx4_alloc_resize_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq *cq, + int entries) +{ + int err; + + if (cq->resize_buf) { + err = -EBUSY; + goto out; + } + + cq->resize_buf = kmalloc(sizeof *cq->resize_buf, GFP_ATOMIC); + if (!cq->resize_buf) { + err = -ENOMEM; + goto out; + } + + err = mlx4_alloc_cq_buf(dev, &cq->resize_buf->buf, entries); + if (err) { + spin_lock_irq(&cq->lock); + kfree(cq->resize_buf); + cq->resize_buf = NULL; + spin_unlock_irq(&cq->lock); + goto out; + } + + cq->resize_buf->cqe = entries - 1; + + return 0; + +out: + return err; +} + +static int mlx4_alloc_resize_umem(struct mlx4_ib_dev *dev, struct mlx4_ib_cq *cq, + int entries, struct ib_udata *udata) +{ + struct mlx4_ib_resize_cq ucmd; + int buf_size; + int err; + + if (cq->resize_umem) { + err = -EBUSY; + goto out; + } + + if (ib_copy_from_udata(&ucmd, udata, sizeof ucmd)) { + err = -EFAULT; + goto out; + } + + cq->resize_buf = kmalloc(sizeof *cq->resize_buf, GFP_ATOMIC); + if (!cq->resize_buf) { + err = -ENOMEM; + goto out; + } + + buf_size = entries * sizeof(struct mlx4_cqe); + + cq->resize_umem = ib_umem_get(cq->umem->context, ucmd.buf_addr, buf_size, + IB_ACCESS_LOCAL_WRITE); + if (IS_ERR(cq->resize_umem)) { + err = PTR_ERR(cq->resize_umem); + goto err_umem; + } + + err = mlx4_mtt_init(dev->dev, ib_umem_page_count(cq->resize_umem), + ilog2(cq->resize_umem->page_size), &cq->resize_buf->buf.mtt); + if (err) + goto err_buf; + + err = mlx4_ib_umem_write_mtt(dev, &cq->resize_buf->buf.mtt, cq->resize_umem); + if (err) + goto err_mtt; + + cq->resize_buf->cqe = entries - 1; + + return 0; + +err_mtt: + mlx4_mtt_cleanup(dev->dev, &cq->resize_buf->buf.mtt); + +err_buf: + ib_umem_release(cq->resize_umem); + cq->resize_umem = NULL; + +err_umem: + kfree(cq->resize_buf); + cq->resize_buf = NULL; + +out: + return err; +} + +int mlx4_get_outstanding_cqes(struct mlx4_ib_cq *cq) +{ + int i; + + i = cq->mcq.cons_index; + while (get_sw_cqe(cq, i & cq->ibcq.cqe)) + ++i; + + return i - cq->mcq.cons_index; +} + +void mlx4_cq_resize_copy_cqes(struct mlx4_ib_cq *cq) +{ + struct mlx4_cqe *cqe; + int i; + + i = cq->mcq.cons_index; + cqe = get_cqe(cq, i & cq->ibcq.cqe); + while ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) != MLX4_CQE_OPCODE_RESIZE) { + memcpy(get_cqe_from_buf(&cq->resize_buf->buf, + (i + 1) & cq->resize_buf->cqe), + get_cqe(cq, i & cq->ibcq.cqe), MLX4_CQ_ENTRY_SIZE); + cqe = get_cqe(cq, ++i & cq->ibcq.cqe); + } + ++cq->mcq.cons_index; +} + +int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) +{ + struct mlx4_ib_dev *dev = to_mdev(ibcq->device); + struct mlx4_ib_cq *cq = to_mcq(ibcq); + struct mlx4_cq_context *context; + struct ib_umem *tumem = NULL; + u64 mtt_addr; + int outst_cqe; + int err; + + mutex_lock(&cq->resize_mutex); + + if (entries < 1 || entries > dev->dev->caps.max_cqes) { + err = -EINVAL; + goto out; + } + + entries = roundup_pow_of_two(entries + 1); + if (entries == ibcq->cqe + 1) { + err = 0; + goto out; + } + + if (cq->is_kernel) { + /* Can't be smaller then the number of outstanding CQEs */ + outst_cqe = mlx4_get_outstanding_cqes(cq); + if (entries < outst_cqe + 1) { + err = 0; + goto out; + } + + err = mlx4_alloc_resize_buf(dev, cq, entries); + if (err) + goto out; + } else { + err = mlx4_alloc_resize_umem(dev, cq, entries, udata); + if (err) + goto out; + } + + context = kzalloc(sizeof *context, GFP_KERNEL); + + if (!context) { + err = -ENOMEM; + goto err_buf; + } + + memset(context, 0, sizeof *context); + + context->logsize_usrpage = cpu_to_be32(ilog2(entries) << 24); + context->log_page_size = cq->resize_buf->buf.mtt.page_shift - 12; + mtt_addr = mlx4_mtt_addr(dev->dev, &cq->resize_buf->buf.mtt); + context->mtt_base_addr_h = mtt_addr >> 32; + context->mtt_base_addr_l = cpu_to_be32(mtt_addr & 0xffffffff); + + err = mlx4_cq_modify(dev->dev, &cq->mcq, context, 0); + if (err) + goto err_buf; + + spin_lock_irq(&cq->lock); + if (cq->is_kernel) { + if (cq->resize_buf) { + mlx4_cq_resize_copy_cqes(cq); + mlx4_free_cq_buf(dev, &cq->buf, cq->ibcq.cqe); + cq->buf = cq->resize_buf->buf; + cq->ibcq.cqe = cq->resize_buf->cqe; + + kfree(cq->resize_buf); + cq->resize_buf = NULL; + } + } else { + cq->buf = cq->resize_buf->buf; + cq->ibcq.cqe = cq->resize_buf->cqe; + tumem = cq->umem; + cq->umem = cq->resize_umem; + + kfree(cq->resize_buf); + cq->resize_buf = NULL; + cq->resize_umem = NULL; + } + spin_unlock_irq(&cq->lock); + + if (!cq->is_kernel) + if (tumem) + ib_umem_release(tumem); + + goto free_context; + +err_buf: + if (cq->resize_buf) { + if (cq->is_kernel) + mlx4_free_cq_buf(dev, &cq->resize_buf->buf, + cq->resize_buf->cqe); + + spin_lock_irq(&cq->lock); + kfree(cq->resize_buf); + cq->resize_buf = NULL; + spin_unlock_irq(&cq->lock); + } + if (cq->resize_umem) { + ib_umem_release(cq->resize_umem); + cq->resize_umem = NULL; + } + +free_context: + kfree(context); + +out: + mutex_unlock(&cq->resize_mutex); + return err; +} + int mlx4_ib_destroy_cq(struct ib_cq *cq) { struct mlx4_ib_dev *dev = to_mdev(cq->device); @@ -255,6 +499,44 @@ static void dump_cqe(void *cqe) be32_to_cpu(buf[6]), be32_to_cpu(buf[7])); } +int mlx4_alloc_cq_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq_buf *buf, int nent) +{ + int err; + + err = mlx4_buf_alloc(dev->dev, nent * MLX4_CQ_ENTRY_SIZE, + MLX4_MAX_DIRECT_CQ_SIZE, + &buf->buf); + + if (err) + goto out; + + err = mlx4_mtt_init(dev->dev, buf->buf.npages, buf->buf.page_shift, + &buf->mtt); + if (err) + goto err_buf; + + err = mlx4_buf_write_mtt(dev->dev, &buf->mtt, &buf->buf); + if (err) + goto err_mtt; + + return 0; + +err_mtt: + mlx4_mtt_cleanup(dev->dev, &buf->mtt); + +err_buf: + mlx4_buf_free(dev->dev, nent * MLX4_CQ_ENTRY_SIZE, + &buf->buf); + +out: + return err; +} + +void mlx4_free_cq_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq_buf *buf, int cqe) +{ + mlx4_buf_free(dev->dev, (cqe + 1) * MLX4_CQ_ENTRY_SIZE, &buf->buf); +} + static void mlx4_ib_handle_error_cqe(struct mlx4_err_cqe *cqe, struct ib_wc *wc) { @@ -343,6 +625,7 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, u32 g_mlpath_rqpn; u16 wqe_ctr; +repoll: cqe = next_cqe_sw(cq); if (!cqe) return -EAGAIN; @@ -364,6 +647,21 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, printk(KERN_WARNING "Completion for NOP opcode detected!\n"); return -EINVAL; } + /* Resize CQ */ + if (unlikely((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_CQE_OPCODE_RESIZE)) { + if (cq->resize_buf) { + struct mlx4_ib_dev *dev = to_mdev(cq->ibcq.device); + + mlx4_free_cq_buf(dev, &cq->buf, cq->ibcq.cqe); + cq->buf = cq->resize_buf->buf; + cq->ibcq.cqe = cq->resize_buf->cqe; + + kfree(cq->resize_buf); + cq->resize_buf = NULL; + } + + goto repoll; + } if (!*cur_qp || (be32_to_cpu(cqe->my_qpn) & 0xffffff) != (*cur_qp)->mqp.qpn) { diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index f770610..6ee7f46 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -569,6 +569,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) (1ull << IB_USER_VERBS_CMD_DEREG_MR) | (1ull << IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL) | (1ull << IB_USER_VERBS_CMD_CREATE_CQ) | + (1ull << IB_USER_VERBS_CMD_RESIZE_CQ) | (1ull << IB_USER_VERBS_CMD_DESTROY_CQ) | (1ull << IB_USER_VERBS_CMD_CREATE_QP) | (1ull << IB_USER_VERBS_CMD_MODIFY_QP) | @@ -608,6 +609,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.post_recv = mlx4_ib_post_recv; ibdev->ib_dev.create_cq = mlx4_ib_create_cq; ibdev->ib_dev.modify_cq = mlx4_ib_modify_cq; + ibdev->ib_dev.resize_cq = mlx4_ib_resize_cq; ibdev->ib_dev.destroy_cq = mlx4_ib_destroy_cq; ibdev->ib_dev.poll_cq = mlx4_ib_poll_cq; ibdev->ib_dev.req_notify_cq = mlx4_ib_arm_cq; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 83eb071..425970f 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -78,13 +78,22 @@ struct mlx4_ib_cq_buf { struct mlx4_mtt mtt; }; +struct mlx4_ib_cq_resize { + struct mlx4_ib_cq_buf buf; + int cqe; +}; + struct mlx4_ib_cq { struct ib_cq ibcq; struct mlx4_cq mcq; struct mlx4_ib_cq_buf buf; + struct mlx4_ib_cq_resize *resize_buf; + int is_kernel; struct mlx4_ib_db db; spinlock_t lock; + struct mutex resize_mutex; struct ib_umem *umem; + struct ib_umem *resize_umem; }; struct mlx4_ib_mr { @@ -250,9 +259,13 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, int mlx4_ib_dereg_mr(struct ib_mr *mr); int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); +void mlx4_cq_resize_copy_cqes(struct mlx4_ib_cq *cq); +int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata); struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector, struct ib_ucontext *context, struct ib_udata *udata); +int mlx4_alloc_cq_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq_buf *buf, int nent); +void mlx4_free_cq_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq_buf *buf, int cqe); int mlx4_ib_destroy_cq(struct ib_cq *cq); int mlx4_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); int mlx4_ib_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); -- 1.5.4.2 From vlad at dev.mellanox.co.il Tue Mar 25 09:15:53 2008 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 25 Mar 2008 18:15:53 +0200 Subject: [ofa-general] [PATCH] libmlx4: Added resize CQ capability. Message-ID: <47E92539.7030908@dev.mellanox.co.il> From bfb3fb43bab5f03e124c4eae13012e27432fe405 Mon Sep 17 00:00:00 2001 From: Vladimir Sokolovsky Date: Tue, 25 Mar 2008 17:09:08 +0200 Subject: [PATCH] Added resize CQ capability. Signed-off-by: Vladimir Sokolovsky --- src/cq.c | 48 +++++++++++++++++++++++++++++++++++++++++++----- src/mlx4.h | 4 ++++ src/verbs.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 101 insertions(+), 7 deletions(-) diff --git a/src/cq.c b/src/cq.c index 91297e4..ee7dd7b 100644 --- a/src/cq.c +++ b/src/cq.c @@ -114,10 +114,10 @@ static struct mlx4_cqe *get_cqe(struct mlx4_cq *cq, int entry) static void *get_sw_cqe(struct mlx4_cq *cq, int n) { - struct mlx4_cqe *cqe = get_cqe(cq, n & cq->ibv_cq.cqe); + struct mlx4_cqe *cqe = get_cqe(cq, n & cq->cqe); return (!!(cqe->owner_sr_opcode & MLX4_CQE_OWNER_MASK) ^ - !!(n & (cq->ibv_cq.cqe + 1))) ? NULL : cqe; + !!(n & (cq->cqe + 1))) ? NULL : cqe; } static struct mlx4_cqe *next_cqe_sw(struct mlx4_cq *cq) @@ -201,6 +201,7 @@ static int mlx4_poll_one(struct mlx4_cq *cq, int is_error; int is_send; +repoll: cqe = next_cqe_sw(cq); if (!cqe) return CQ_EMPTY; @@ -215,6 +216,9 @@ static int mlx4_poll_one(struct mlx4_cq *cq, */ rmb(); + if ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_CQE_OPCODE_RESIZE) + goto repoll; + qpn = ntohl(cqe->my_qpn); is_send = cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK; @@ -398,7 +402,7 @@ void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) * from our QP and therefore don't need to be checked. */ for (prod_index = cq->cons_index; get_sw_cqe(cq, prod_index); ++prod_index) - if (prod_index == cq->cons_index + cq->ibv_cq.cqe) + if (prod_index == cq->cons_index + cq->cqe) break; /* @@ -406,13 +410,13 @@ void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) * that match our QP by copying older entries on top of them. */ while ((int) --prod_index - (int) cq->cons_index >= 0) { - cqe = get_cqe(cq, prod_index & cq->ibv_cq.cqe); + cqe = get_cqe(cq, prod_index & cq->cqe); if ((ntohl(cqe->my_qpn) & 0xffffff) == qpn) { if (srq && !(cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK)) mlx4_free_srq_wqe(srq, ntohs(cqe->wqe_index)); ++nfreed; } else if (nfreed) { - dest = get_cqe(cq, (prod_index + nfreed) & cq->ibv_cq.cqe); + dest = get_cqe(cq, (prod_index + nfreed) & cq->cqe); owner_bit = dest->owner_sr_opcode & MLX4_CQE_OWNER_MASK; memcpy(dest, cqe, sizeof *cqe); dest->owner_sr_opcode = owner_bit | @@ -433,6 +437,40 @@ void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) pthread_spin_unlock(&cq->lock); } +int mlx4_get_outstanding_cqes(struct mlx4_cq *cq) +{ + int i; + + for (i = cq->cons_index; get_sw_cqe(cq, (i & cq->cqe)); ++i) + ; + + return i - cq->cons_index; +} + void mlx4_cq_resize_copy_cqes(struct mlx4_cq *cq, void *buf, int old_cqe) { + struct mlx4_cqe *cqe; + int i; + + i = cq->cons_index; + cqe = get_cqe(cq, (i & old_cqe)); + + while ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) != MLX4_CQE_OPCODE_RESIZE) { + memcpy(buf + ((i + 1) & cq->ibv_cq.cqe) * MLX4_CQ_ENTRY_SIZE, + cqe, MLX4_CQ_ENTRY_SIZE); + ++i; + cqe = get_cqe(cq, (i & old_cqe)); + } + + ++cq->cons_index; +} + +int mlx4_alloc_cq_buf(struct mlx4_device *dev, struct mlx4_buf *buf, int nent) +{ + if (mlx4_alloc_buf(buf, align(nent * MLX4_CQ_ENTRY_SIZE, dev->page_size), + dev->page_size)) + return -1; + memset(buf->buf, 0, nent * MLX4_CQ_ENTRY_SIZE); + + return 0; } diff --git a/src/mlx4.h b/src/mlx4.h index 3710a17..61076ac 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -174,12 +174,14 @@ struct mlx4_pd { struct mlx4_cq { struct ibv_cq ibv_cq; struct mlx4_buf buf; + struct mlx4_buf resize_buf; pthread_spinlock_t lock; uint32_t cqn; uint32_t cons_index; uint32_t *set_ci_db; uint32_t *arm_db; int arm_sn; + int cqe; }; struct mlx4_srq { @@ -307,6 +309,7 @@ int mlx4_dereg_mr(struct ibv_mr *mr); struct ibv_cq *mlx4_create_cq(struct ibv_context *context, int cqe, struct ibv_comp_channel *channel, int comp_vector); +int mlx4_alloc_cq_buf(struct mlx4_device *dev, struct mlx4_buf *buf, int nent); int mlx4_resize_cq(struct ibv_cq *cq, int cqe); int mlx4_destroy_cq(struct ibv_cq *cq); int mlx4_poll_cq(struct ibv_cq *cq, int ne, struct ibv_wc *wc); @@ -314,6 +317,7 @@ int mlx4_arm_cq(struct ibv_cq *cq, int solicited); void mlx4_cq_event(struct ibv_cq *cq); void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq); +int mlx4_get_outstanding_cqes(struct mlx4_cq *cq); void mlx4_cq_resize_copy_cqes(struct mlx4_cq *cq, void *buf, int new_cqe); struct ibv_srq *mlx4_create_srq(struct ibv_pd *pd, diff --git a/src/verbs.c b/src/verbs.c index 50e0947..ba04af7 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -209,6 +209,7 @@ struct ibv_cq *mlx4_create_cq(struct ibv_context *context, int cqe, goto err_db; cq->cqn = resp.cqn; + cq->cqe = cq->ibv_cq.cqe; return &cq->ibv_cq; @@ -226,8 +227,59 @@ err: int mlx4_resize_cq(struct ibv_cq *ibcq, int cqe) { - /* XXX resize CQ not implemented */ - return ENOSYS; + struct mlx4_cq *cq = to_mcq(ibcq); + struct mlx4_resize_cq cmd; + struct mlx4_buf buf; + int old_cqe, outst_cqe, ret; + + /* Sanity check CQ size before proceeding */ + if (cqe > 0x3fffff) + return EINVAL; + + pthread_spin_lock(&cq->lock); + + cqe = align_queue_size(cqe); + if (cqe == ibcq->cqe + 1) { + ret = 0; + goto out; + } + + /* Can't be smaller then the number of outstanding CQEs */ + outst_cqe = mlx4_get_outstanding_cqes(cq); + if (cqe < outst_cqe + 1) { + ret = 0; + goto out; + } + + ret = mlx4_alloc_cq_buf(to_mdev(ibcq->context->device), &buf, cqe); + if (ret) + goto out; + + cmd.buf_addr = (uintptr_t) buf.buf; + +#ifdef IBV_CMD_RESIZE_CQ_HAS_RESP_PARAMS + { + struct ibv_resize_cq_resp resp; + ret = ibv_cmd_resize_cq(ibcq, cqe - 1, &cmd.ibv_cmd, sizeof cmd, + &resp, sizeof resp); + } +#else + ret = ibv_cmd_resize_cq(ibcq, cqe - 1, &cmd.ibv_cmd, sizeof cmd); +#endif + if (ret) { + mlx4_free_buf(&buf); + goto out; + } + + mlx4_cq_resize_copy_cqes(cq, buf.buf, cq->cqe); + mlx4_free_buf(&cq->buf); + + cq->buf = buf; + cq->cqe = cq->ibv_cq.cqe; + +out: + pthread_spin_unlock(&cq->lock); + return ret; } int mlx4_destroy_cq(struct ibv_cq *cq) -- 1.5.4.2 From eli at dev.mellanox.co.il Tue Mar 25 09:43:42 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 18:43:42 +0200 Subject: [ofa-general] [PATCH v1] IB/ipoib: Split CQs for IPOIB UD Message-ID: <1206463422.25950.363.camel@mtls03> >From afb4d7406868c58d98e47d08bfbb45996412be42 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 20 Mar 2008 16:35:30 +0200 Subject: [PATCH] IB/ipoib: Split CQs for IPOIB UD Use a dedicated CQ for UD send. Also, do not arm the UD send CQ thus reducing the number of interrupts generated by the HCA. This patch farther reduces overhead by not calling poll CQ for every posted send WR - it does it only when there 16 or more outstanding work requests. Signed-off-by: Eli Cohen --- Changes since last post: Fix CQ calculation drivers/infiniband/ulp/ipoib/ipoib.h | 9 ++++-- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 8 ++-- drivers/infiniband/ulp/ipoib/ipoib_etool.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 45 ++++++++++++++++------------ drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 39 ++++++++++++++++-------- 5 files changed, 63 insertions(+), 40 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 43feffc..fb28f0b 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -95,6 +95,8 @@ enum { IPOIB_MCAST_FLAG_SENDONLY = 1, IPOIB_MCAST_FLAG_BUSY = 2, /* joining or already joined */ IPOIB_MCAST_FLAG_ATTACHED = 3, + + MAX_SEND_CQE = 16, }; #define IPOIB_OP_RECV (1ul << 31) @@ -285,7 +287,8 @@ struct ipoib_dev_priv { u16 pkey_index; struct ib_pd *pd; struct ib_mr *mr; - struct ib_cq *cq; + struct ib_cq *rcq; + struct ib_cq *scq; struct ib_qp *qp; u32 qkey; @@ -305,7 +308,8 @@ struct ipoib_dev_priv { struct ib_send_wr tx_wr; unsigned tx_outstanding; - struct ib_wc ibwc[IPOIB_NUM_WC]; + struct ib_wc ibwc[IPOIB_NUM_WC]; + struct ib_wc send_wc[MAX_SEND_CQE]; struct list_head dead_ahs; @@ -650,7 +654,6 @@ static inline int ipoib_register_debugfs(void) { return 0; } static inline void ipoib_unregister_debugfs(void) { } #endif - #define ipoib_printk(level, priv, format, arg...) \ printk(level "%s: " format, ((struct ipoib_dev_priv *) priv)->dev->name , ## arg) #define ipoib_warn(priv, format, arg...) \ diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 90ff2c9..dfabb38 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -249,8 +249,8 @@ static struct ib_qp *ipoib_cm_create_rx_qp(struct net_device *dev, struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { .event_handler = ipoib_cm_rx_event_handler, - .send_cq = priv->cq, /* For drain WR */ - .recv_cq = priv->cq, + .send_cq = priv->rcq, /* For drain WR */ + .recv_cq = priv->rcq, .srq = priv->cm.srq, .cap.max_send_wr = 1, /* For drain WR */ .cap.max_send_sge = 1, /* FIXME: 0 Seems not to work */ @@ -951,8 +951,8 @@ static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ipoib_ { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { - .send_cq = priv->cq, - .recv_cq = priv->cq, + .send_cq = priv->rcq, + .recv_cq = priv->rcq, .srq = priv->cm.srq, .cap.max_send_wr = ipoib_sendq_size, .cap.max_send_sge = 1, diff --git a/drivers/infiniband/ulp/ipoib/ipoib_etool.c b/drivers/infiniband/ulp/ipoib/ipoib_etool.c index a3ac4cf..b4f4f0f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_etool.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_etool.c @@ -73,7 +73,7 @@ static int ipoib_set_coalesce(struct net_device *dev, coal->rx_max_coalesced_frames > 0xffff) return -EINVAL; - ret = ib_modify_cq(priv->cq, coal->rx_max_coalesced_frames, + ret = ib_modify_cq(priv->rcq, coal->rx_max_coalesced_frames, coal->rx_coalesce_usecs); if (ret) { ipoib_dbg(priv, "failed modifying CQ\n"); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 5c61a81..8222b50 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -311,7 +311,6 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) struct ipoib_dev_priv *priv = netdev_priv(dev); unsigned int wr_id = wc->wr_id; struct ipoib_tx_buf *tx_req; - unsigned long flags; ipoib_dbg_data(priv, "send completion: id %d, status: %d\n", wr_id, wc->status); @@ -331,13 +330,11 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) dev_kfree_skb_any(tx_req->skb); - spin_lock_irqsave(&priv->tx_lock, flags); ++priv->tx_tail; if (unlikely(--priv->tx_outstanding == ipoib_sendq_size >> 1) && netif_queue_stopped(dev) && test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) netif_wake_queue(dev); - spin_unlock_irqrestore(&priv->tx_lock, flags); if (wc->status != IB_WC_SUCCESS && wc->status != IB_WC_WR_FLUSH_ERR) @@ -346,6 +343,17 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) wc->status, wr_id, wc->vendor_err); } +static int poll_tx(struct ipoib_dev_priv *priv) +{ + int n, i; + + n = ib_poll_cq(priv->scq, MAX_SEND_CQE, priv->send_wc); + for (i = 0; i < n; ++i) + ipoib_ib_handle_tx_wc(priv->dev, priv->send_wc + i); + + return n == MAX_SEND_CQE; +} + int ipoib_poll(struct napi_struct *napi, int budget) { struct ipoib_dev_priv *priv = container_of(napi, struct ipoib_dev_priv, napi); @@ -361,7 +369,7 @@ poll_more: int max = (budget - done); t = min(IPOIB_NUM_WC, max); - n = ib_poll_cq(priv->cq, t, priv->ibwc); + n = ib_poll_cq(priv->rcq, t, priv->ibwc); for (i = 0; i < n; i++) { struct ib_wc *wc = priv->ibwc + i; @@ -372,12 +380,8 @@ poll_more: ipoib_cm_handle_rx_wc(dev, wc); else ipoib_ib_handle_rx_wc(dev, wc); - } else { - if (wc->wr_id & IPOIB_OP_CM) - ipoib_cm_handle_tx_wc(dev, wc); - else - ipoib_ib_handle_tx_wc(dev, wc); - } + } else + ipoib_cm_handle_tx_wc(priv->dev, wc); } if (n != t) @@ -386,7 +390,7 @@ poll_more: if (done < budget) { netif_rx_complete(dev, napi); - if (unlikely(ib_req_notify_cq(priv->cq, + if (unlikely(ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS)) && netif_rx_reschedule(dev, napi)) @@ -507,12 +511,17 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, address->last_send = priv->tx_head; ++priv->tx_head; + skb_orphan(skb); if (++priv->tx_outstanding == ipoib_sendq_size) { ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); netif_stop_queue(dev); } } + + if (unlikely(priv->tx_outstanding > MAX_SEND_CQE)) + poll_tx(priv); + return; drop: @@ -665,7 +674,7 @@ void ipoib_drain_cq(struct net_device *dev) struct ipoib_dev_priv *priv = netdev_priv(dev); int i, n; do { - n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); + n = ib_poll_cq(priv->rcq, IPOIB_NUM_WC, priv->ibwc); for (i = 0; i < n; ++i) { /* * Convert any successful completions to flush @@ -680,14 +689,12 @@ void ipoib_drain_cq(struct net_device *dev) ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); else ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); - } else { - if (priv->ibwc[i].wr_id & IPOIB_OP_CM) - ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); - else - ipoib_ib_handle_tx_wc(dev, priv->ibwc + i); - } + } else + ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); } } while (n == IPOIB_NUM_WC); + + while(poll_tx(priv)); } int ipoib_ib_dev_stop(struct net_device *dev, int flush) @@ -779,7 +786,7 @@ timeout: msleep(1); } - ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP); + ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP); return 0; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 1d59f27..b9e7eab 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -171,26 +171,33 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) goto out_free_pd; } - size = ipoib_sendq_size + ipoib_recvq_size + 1; + size = ipoib_recvq_size + 1; ret = ipoib_cm_dev_init(dev); if (!ret) { + size += ipoib_sendq_size; if (ipoib_cm_has_srq(dev)) size += ipoib_recvq_size + 1; /* 1 extra for rx_drain_qp */ else size += ipoib_recvq_size * ipoib_max_conn_qp; } - priv->cq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); - if (IS_ERR(priv->cq)) { - printk(KERN_WARNING "%s: failed to create CQ\n", ca->name); + priv->rcq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); + if (IS_ERR(priv->rcq)) { + printk(KERN_WARNING "%s: failed to create receive CQ\n", ca->name); goto out_free_mr; } - if (ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP)) - goto out_free_cq; + priv->scq = ib_create_cq(priv->ca, NULL, NULL, dev, ipoib_sendq_size, 0); + if (IS_ERR(priv->scq)) { + printk(KERN_WARNING "%s: failed to create send CQ\n", ca->name); + goto out_free_rcq; + } + + if (ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP)) + goto out_free_scq; - init_attr.send_cq = priv->cq; - init_attr.recv_cq = priv->cq; + init_attr.send_cq = priv->scq; + init_attr.recv_cq = priv->rcq; if (priv->hca_caps & IB_DEVICE_TCP_TSO) init_attr.create_flags = QP_CREATE_LSO; @@ -201,7 +208,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) priv->qp = ib_create_qp(priv->pd, &init_attr); if (IS_ERR(priv->qp)) { printk(KERN_WARNING "%s: failed to create QP\n", ca->name); - goto out_free_cq; + goto out_free_scq; } priv->dev->dev_addr[1] = (priv->qp->qp_num >> 16) & 0xff; @@ -217,8 +224,11 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) return 0; -out_free_cq: - ib_destroy_cq(priv->cq); +out_free_scq: + ib_destroy_cq(priv->scq); + +out_free_rcq: + ib_destroy_cq(priv->rcq); out_free_mr: ib_dereg_mr(priv->mr); @@ -241,8 +251,11 @@ void ipoib_transport_dev_cleanup(struct net_device *dev) clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); } - if (ib_destroy_cq(priv->cq)) - ipoib_warn(priv, "ib_cq_destroy failed\n"); + if (ib_destroy_cq(priv->scq)) + ipoib_warn(priv, "ib_cq_destroy (send) failed\n"); + + if (ib_destroy_cq(priv->rcq)) + ipoib_warn(priv, "ib_cq_destroy (recv) failed\n"); ipoib_cm_dev_cleanup(dev); -- 1.5.4.4 From eli at dev.mellanox.co.il Tue Mar 25 09:43:42 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Tue, 25 Mar 2008 18:43:42 +0200 Subject: [ofa-general] [PATCH v1] IB/ipoib: Split CQs for IPOIB UD Message-ID: <1206463422.25950.363.camel@mtls03> >From afb4d7406868c58d98e47d08bfbb45996412be42 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Thu, 20 Mar 2008 16:35:30 +0200 Subject: [PATCH] IB/ipoib: Split CQs for IPOIB UD Use a dedicated CQ for UD send. Also, do not arm the UD send CQ thus reducing the number of interrupts generated by the HCA. This patch farther reduces overhead by not calling poll CQ for every posted send WR - it does it only when there 16 or more outstanding work requests. Signed-off-by: Eli Cohen --- Changes since last post: Fix CQ calculation drivers/infiniband/ulp/ipoib/ipoib.h | 9 ++++-- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 8 ++-- drivers/infiniband/ulp/ipoib/ipoib_etool.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 45 ++++++++++++++++------------ drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 39 ++++++++++++++++-------- 5 files changed, 63 insertions(+), 40 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 43feffc..fb28f0b 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -95,6 +95,8 @@ enum { IPOIB_MCAST_FLAG_SENDONLY = 1, IPOIB_MCAST_FLAG_BUSY = 2, /* joining or already joined */ IPOIB_MCAST_FLAG_ATTACHED = 3, + + MAX_SEND_CQE = 16, }; #define IPOIB_OP_RECV (1ul << 31) @@ -285,7 +287,8 @@ struct ipoib_dev_priv { u16 pkey_index; struct ib_pd *pd; struct ib_mr *mr; - struct ib_cq *cq; + struct ib_cq *rcq; + struct ib_cq *scq; struct ib_qp *qp; u32 qkey; @@ -305,7 +308,8 @@ struct ipoib_dev_priv { struct ib_send_wr tx_wr; unsigned tx_outstanding; - struct ib_wc ibwc[IPOIB_NUM_WC]; + struct ib_wc ibwc[IPOIB_NUM_WC]; + struct ib_wc send_wc[MAX_SEND_CQE]; struct list_head dead_ahs; @@ -650,7 +654,6 @@ static inline int ipoib_register_debugfs(void) { return 0; } static inline void ipoib_unregister_debugfs(void) { } #endif - #define ipoib_printk(level, priv, format, arg...) \ printk(level "%s: " format, ((struct ipoib_dev_priv *) priv)->dev->name , ## arg) #define ipoib_warn(priv, format, arg...) \ diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 90ff2c9..dfabb38 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -249,8 +249,8 @@ static struct ib_qp *ipoib_cm_create_rx_qp(struct net_device *dev, struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { .event_handler = ipoib_cm_rx_event_handler, - .send_cq = priv->cq, /* For drain WR */ - .recv_cq = priv->cq, + .send_cq = priv->rcq, /* For drain WR */ + .recv_cq = priv->rcq, .srq = priv->cm.srq, .cap.max_send_wr = 1, /* For drain WR */ .cap.max_send_sge = 1, /* FIXME: 0 Seems not to work */ @@ -951,8 +951,8 @@ static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ipoib_ { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { - .send_cq = priv->cq, - .recv_cq = priv->cq, + .send_cq = priv->rcq, + .recv_cq = priv->rcq, .srq = priv->cm.srq, .cap.max_send_wr = ipoib_sendq_size, .cap.max_send_sge = 1, diff --git a/drivers/infiniband/ulp/ipoib/ipoib_etool.c b/drivers/infiniband/ulp/ipoib/ipoib_etool.c index a3ac4cf..b4f4f0f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_etool.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_etool.c @@ -73,7 +73,7 @@ static int ipoib_set_coalesce(struct net_device *dev, coal->rx_max_coalesced_frames > 0xffff) return -EINVAL; - ret = ib_modify_cq(priv->cq, coal->rx_max_coalesced_frames, + ret = ib_modify_cq(priv->rcq, coal->rx_max_coalesced_frames, coal->rx_coalesce_usecs); if (ret) { ipoib_dbg(priv, "failed modifying CQ\n"); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 5c61a81..8222b50 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -311,7 +311,6 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) struct ipoib_dev_priv *priv = netdev_priv(dev); unsigned int wr_id = wc->wr_id; struct ipoib_tx_buf *tx_req; - unsigned long flags; ipoib_dbg_data(priv, "send completion: id %d, status: %d\n", wr_id, wc->status); @@ -331,13 +330,11 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) dev_kfree_skb_any(tx_req->skb); - spin_lock_irqsave(&priv->tx_lock, flags); ++priv->tx_tail; if (unlikely(--priv->tx_outstanding == ipoib_sendq_size >> 1) && netif_queue_stopped(dev) && test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) netif_wake_queue(dev); - spin_unlock_irqrestore(&priv->tx_lock, flags); if (wc->status != IB_WC_SUCCESS && wc->status != IB_WC_WR_FLUSH_ERR) @@ -346,6 +343,17 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) wc->status, wr_id, wc->vendor_err); } +static int poll_tx(struct ipoib_dev_priv *priv) +{ + int n, i; + + n = ib_poll_cq(priv->scq, MAX_SEND_CQE, priv->send_wc); + for (i = 0; i < n; ++i) + ipoib_ib_handle_tx_wc(priv->dev, priv->send_wc + i); + + return n == MAX_SEND_CQE; +} + int ipoib_poll(struct napi_struct *napi, int budget) { struct ipoib_dev_priv *priv = container_of(napi, struct ipoib_dev_priv, napi); @@ -361,7 +369,7 @@ poll_more: int max = (budget - done); t = min(IPOIB_NUM_WC, max); - n = ib_poll_cq(priv->cq, t, priv->ibwc); + n = ib_poll_cq(priv->rcq, t, priv->ibwc); for (i = 0; i < n; i++) { struct ib_wc *wc = priv->ibwc + i; @@ -372,12 +380,8 @@ poll_more: ipoib_cm_handle_rx_wc(dev, wc); else ipoib_ib_handle_rx_wc(dev, wc); - } else { - if (wc->wr_id & IPOIB_OP_CM) - ipoib_cm_handle_tx_wc(dev, wc); - else - ipoib_ib_handle_tx_wc(dev, wc); - } + } else + ipoib_cm_handle_tx_wc(priv->dev, wc); } if (n != t) @@ -386,7 +390,7 @@ poll_more: if (done < budget) { netif_rx_complete(dev, napi); - if (unlikely(ib_req_notify_cq(priv->cq, + if (unlikely(ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS)) && netif_rx_reschedule(dev, napi)) @@ -507,12 +511,17 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, address->last_send = priv->tx_head; ++priv->tx_head; + skb_orphan(skb); if (++priv->tx_outstanding == ipoib_sendq_size) { ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); netif_stop_queue(dev); } } + + if (unlikely(priv->tx_outstanding > MAX_SEND_CQE)) + poll_tx(priv); + return; drop: @@ -665,7 +674,7 @@ void ipoib_drain_cq(struct net_device *dev) struct ipoib_dev_priv *priv = netdev_priv(dev); int i, n; do { - n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); + n = ib_poll_cq(priv->rcq, IPOIB_NUM_WC, priv->ibwc); for (i = 0; i < n; ++i) { /* * Convert any successful completions to flush @@ -680,14 +689,12 @@ void ipoib_drain_cq(struct net_device *dev) ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); else ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); - } else { - if (priv->ibwc[i].wr_id & IPOIB_OP_CM) - ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); - else - ipoib_ib_handle_tx_wc(dev, priv->ibwc + i); - } + } else + ipoib_cm_handle_tx_wc(dev, priv->ibwc + i); } } while (n == IPOIB_NUM_WC); + + while(poll_tx(priv)); } int ipoib_ib_dev_stop(struct net_device *dev, int flush) @@ -779,7 +786,7 @@ timeout: msleep(1); } - ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP); + ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP); return 0; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 1d59f27..b9e7eab 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -171,26 +171,33 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) goto out_free_pd; } - size = ipoib_sendq_size + ipoib_recvq_size + 1; + size = ipoib_recvq_size + 1; ret = ipoib_cm_dev_init(dev); if (!ret) { + size += ipoib_sendq_size; if (ipoib_cm_has_srq(dev)) size += ipoib_recvq_size + 1; /* 1 extra for rx_drain_qp */ else size += ipoib_recvq_size * ipoib_max_conn_qp; } - priv->cq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); - if (IS_ERR(priv->cq)) { - printk(KERN_WARNING "%s: failed to create CQ\n", ca->name); + priv->rcq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); + if (IS_ERR(priv->rcq)) { + printk(KERN_WARNING "%s: failed to create receive CQ\n", ca->name); goto out_free_mr; } - if (ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP)) - goto out_free_cq; + priv->scq = ib_create_cq(priv->ca, NULL, NULL, dev, ipoib_sendq_size, 0); + if (IS_ERR(priv->scq)) { + printk(KERN_WARNING "%s: failed to create send CQ\n", ca->name); + goto out_free_rcq; + } + + if (ib_req_notify_cq(priv->rcq, IB_CQ_NEXT_COMP)) + goto out_free_scq; - init_attr.send_cq = priv->cq; - init_attr.recv_cq = priv->cq; + init_attr.send_cq = priv->scq; + init_attr.recv_cq = priv->rcq; if (priv->hca_caps & IB_DEVICE_TCP_TSO) init_attr.create_flags = QP_CREATE_LSO; @@ -201,7 +208,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) priv->qp = ib_create_qp(priv->pd, &init_attr); if (IS_ERR(priv->qp)) { printk(KERN_WARNING "%s: failed to create QP\n", ca->name); - goto out_free_cq; + goto out_free_scq; } priv->dev->dev_addr[1] = (priv->qp->qp_num >> 16) & 0xff; @@ -217,8 +224,11 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) return 0; -out_free_cq: - ib_destroy_cq(priv->cq); +out_free_scq: + ib_destroy_cq(priv->scq); + +out_free_rcq: + ib_destroy_cq(priv->rcq); out_free_mr: ib_dereg_mr(priv->mr); @@ -241,8 +251,11 @@ void ipoib_transport_dev_cleanup(struct net_device *dev) clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags); } - if (ib_destroy_cq(priv->cq)) - ipoib_warn(priv, "ib_cq_destroy failed\n"); + if (ib_destroy_cq(priv->scq)) + ipoib_warn(priv, "ib_cq_destroy (send) failed\n"); + + if (ib_destroy_cq(priv->rcq)) + ipoib_warn(priv, "ib_cq_destroy (recv) failed\n"); ipoib_cm_dev_cleanup(dev); -- 1.5.4.4 From ralph.campbell at qlogic.com Tue Mar 25 11:06:07 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 25 Mar 2008 11:06:07 -0700 Subject: [ofa-general] [PATCH 0/6] IB/ipath -- patches in for-roland for 2.6.26 Message-ID: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> The following patches make more PIO buffers available for the kernel to use, clean up the link state changes, and add support for newer EEPROMs and temperature sensors. I think these are appropriate for 2.6.26. These can also be pulled into Roland's infiniband.git for-2.6.26 repo using: git pull git://git.qlogic.com/ipath-linux-2.6 for-roland From ralph.campbell at qlogic.com Tue Mar 25 11:06:12 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 25 Mar 2008 11:06:12 -0700 Subject: [ofa-general] [PATCH 1/6] IB/ipath - Change the module author In-Reply-To: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> References: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> Message-ID: <20080325180612.27615.19836.stgit@eng-46.mv.qlogic.com> Update the module author to the current email address. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_driver.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index b1613a7..bd98b86 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -87,7 +87,7 @@ module_param_named(linkrecovery, ipath_linkrecovery, uint, S_IWUSR | S_IRUGO); MODULE_PARM_DESC(linkrecovery, "enable workaround for link recovery issue"); MODULE_LICENSE("GPL"); -MODULE_AUTHOR("QLogic "); +MODULE_AUTHOR("QLogic "); MODULE_DESCRIPTION("QLogic InfiniPath driver"); const char *ipath_ibcstatus_str[] = { From ralph.campbell at qlogic.com Tue Mar 25 11:06:17 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 25 Mar 2008 11:06:17 -0700 Subject: [ofa-general] [PATCH 2/6] IB/ipath - remove some useless (void) casts In-Reply-To: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> References: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> Message-ID: <20080325180617.27615.35403.stgit@eng-46.mv.qlogic.com> This just removes some useless (void) casts. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_driver.c | 6 +++--- drivers/infiniband/hw/ipath/ipath_init_chip.c | 12 ++++++------ 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index bd98b86..02c468b 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -1243,10 +1243,10 @@ reloop: lval = dd->ipath_rhdrhead_intr_off | l; else lval = l; - (void)ipath_write_ureg(dd, ur_rcvhdrhead, lval, 0); + ipath_write_ureg(dd, ur_rcvhdrhead, lval, 0); if (updegr) { - (void)ipath_write_ureg(dd, ur_rcvegrindexhead, - etail, 0); + ipath_write_ureg(dd, ur_rcvegrindexhead, + etail, 0); updegr = 0; } } diff --git a/drivers/infiniband/hw/ipath/ipath_init_chip.c b/drivers/infiniband/hw/ipath/ipath_init_chip.c index bed0927..524fdf7 100644 --- a/drivers/infiniband/hw/ipath/ipath_init_chip.c +++ b/drivers/infiniband/hw/ipath/ipath_init_chip.c @@ -495,12 +495,12 @@ static void enable_chip(struct ipath_devdata *dd, * head values to match. */ val = ipath_read_ureg32(dd, ur_rcvegrindextail, 0); - (void)ipath_write_ureg(dd, ur_rcvegrindexhead, val, 0); + ipath_write_ureg(dd, ur_rcvegrindexhead, val, 0); /* Initialize so we interrupt on next packet received */ - (void)ipath_write_ureg(dd, ur_rcvhdrhead, - dd->ipath_rhdrhead_intr_off | - dd->ipath_pd[0]->port_head, 0); + ipath_write_ureg(dd, ur_rcvhdrhead, + dd->ipath_rhdrhead_intr_off | + dd->ipath_pd[0]->port_head, 0); /* * by now pioavail updates to memory should have occurred, so @@ -769,8 +769,8 @@ int ipath_init_chip(struct ipath_devdata *dd, int reinit) goto done; } - (void)ipath_write_kreg(dd, dd->ipath_kregs->kr_sendpioavailaddr, - dd->ipath_pioavailregs_phys); + ipath_write_kreg(dd, dd->ipath_kregs->kr_sendpioavailaddr, + dd->ipath_pioavailregs_phys); /* * this is to detect s/w errors, which the h/w works around by * ignoring the low 6 bits of address, if it wasn't aligned. From ralph.campbell at qlogic.com Tue Mar 25 11:06:22 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 25 Mar 2008 11:06:22 -0700 Subject: [ofa-general] [PATCH 3/6] IB/ipath -- prevent link-recovery code from negating admin disable In-Reply-To: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> References: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> Message-ID: <20080325180622.27615.59377.stgit@eng-46.mv.qlogic.com> From: Michael Albaugh The link can be put in LINKDOWN_DISABLE state either locally or via a mad. However, the link-recovery code will take it out of that state as a side-effect of attempts to clear SerDes/XGXS issues. We add a flag to indicate "link is down on purpose, leave it alone" Signed-off-by: Michael Albaugh --- drivers/infiniband/hw/ipath/ipath_debug.h | 1 drivers/infiniband/hw/ipath/ipath_driver.c | 70 +++++++++++++++---------- drivers/infiniband/hw/ipath/ipath_init_chip.c | 5 +- drivers/infiniband/hw/ipath/ipath_intr.c | 2 + drivers/infiniband/hw/ipath/ipath_kernel.h | 2 + drivers/infiniband/hw/ipath/ipath_mad.c | 4 + 6 files changed, 55 insertions(+), 29 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_debug.h b/drivers/infiniband/hw/ipath/ipath_debug.h index 7170bd2..65926cd 100644 --- a/drivers/infiniband/hw/ipath/ipath_debug.h +++ b/drivers/infiniband/hw/ipath/ipath_debug.h @@ -90,6 +90,7 @@ #define __IPATH_IPATHERR 0x0 /* Ethernet (IPATH) errors on */ #define __IPATH_IPATHPD 0x0 /* Ethernet (IPATH) packet dump on */ #define __IPATH_IPATHTABLE 0x0 /* Ethernet (IPATH) packet dump on */ +#define __IPATH_LINKVERBDBG 0x0 /* very verbose linkchange debug */ #endif /* _IPATH_DEBUGGING */ diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index 02c468b..dfd19de 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -1664,30 +1664,48 @@ void ipath_cancel_sends(struct ipath_devdata *dd, int restore_sendctrl) ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); } - -static void ipath_set_ib_lstate(struct ipath_devdata *dd, int which) +static void ipath_set_ib_lstate(struct ipath_devdata *dd, int linkcmd, + int linitcmd) { + u64 mod_wd; static const char *what[4] = { [0] = "NOP", [INFINIPATH_IBCC_LINKCMD_DOWN] = "DOWN", [INFINIPATH_IBCC_LINKCMD_ARMED] = "ARMED", [INFINIPATH_IBCC_LINKCMD_ACTIVE] = "ACTIVE" }; - int linkcmd = (which >> INFINIPATH_IBCC_LINKCMD_SHIFT) & - INFINIPATH_IBCC_LINKCMD_MASK; - ipath_cdbg(VERBOSE, "Trying to move unit %u to %s, current ltstate " - "is %s\n", dd->ipath_unit, - what[linkcmd], - ipath_ibcstatus_str[ipath_ib_linktrstate(dd, + if (linitcmd == INFINIPATH_IBCC_LINKINITCMD_DISABLE) { + /* + * If we are told to disable, note that so link-recovery + * code does not attempt to bring us back up. + */ + preempt_disable(); + dd->ipath_flags |= IPATH_IB_LINK_DISABLED; + preempt_enable(); + } else if (linitcmd) { + /* + * Any other linkinitcmd will lead to LINKDOWN and then + * to INIT (if all is well), so clear flag to let + * link-recovery code attempt to bring us back up. + */ + preempt_disable(); + dd->ipath_flags &= ~IPATH_IB_LINK_DISABLED; + preempt_enable(); + } + + mod_wd = (linkcmd << dd->ibcc_lc_shift) | + (linitcmd << INFINIPATH_IBCC_LINKINITCMD_SHIFT); + ipath_cdbg(VERBOSE, + "Moving unit %u to %s (initcmd=0x%x), current ltstate is %s\n", + dd->ipath_unit, what[linkcmd], linitcmd, + ipath_ibcstatus_str[ipath_ib_linktrstate(dd, ipath_read_kreg64(dd, dd->ipath_kregs->kr_ibcstatus))]); - /* flush all queued sends when going to DOWN to be sure that - * they don't block MAD packets */ - if (linkcmd == INFINIPATH_IBCC_LINKCMD_DOWN) - ipath_cancel_sends(dd, 1); ipath_write_kreg(dd, dd->ipath_kregs->kr_ibcctrl, - dd->ipath_ibcctrl | which); + dd->ipath_ibcctrl | mod_wd); + /* read from chip so write is flushed */ + (void) ipath_read_kreg64(dd, dd->ipath_kregs->kr_ibcstatus); } int ipath_set_linkstate(struct ipath_devdata *dd, u8 newstate) @@ -1697,30 +1715,28 @@ int ipath_set_linkstate(struct ipath_devdata *dd, u8 newstate) switch (newstate) { case IPATH_IB_LINKDOWN_ONLY: - ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_DOWN << - INFINIPATH_IBCC_LINKCMD_SHIFT); + ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_DOWN, 0); /* don't wait */ ret = 0; goto bail; case IPATH_IB_LINKDOWN: - ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKINITCMD_POLL << - INFINIPATH_IBCC_LINKINITCMD_SHIFT); + ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_DOWN, + INFINIPATH_IBCC_LINKINITCMD_POLL); /* don't wait */ ret = 0; goto bail; case IPATH_IB_LINKDOWN_SLEEP: - ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKINITCMD_SLEEP << - INFINIPATH_IBCC_LINKINITCMD_SHIFT); + ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_DOWN, + INFINIPATH_IBCC_LINKINITCMD_SLEEP); /* don't wait */ ret = 0; goto bail; case IPATH_IB_LINKDOWN_DISABLE: - ipath_set_ib_lstate(dd, - INFINIPATH_IBCC_LINKINITCMD_DISABLE << - INFINIPATH_IBCC_LINKINITCMD_SHIFT); + ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_DOWN, + INFINIPATH_IBCC_LINKINITCMD_DISABLE); /* don't wait */ ret = 0; goto bail; @@ -1735,8 +1751,8 @@ int ipath_set_linkstate(struct ipath_devdata *dd, u8 newstate) ret = -EINVAL; goto bail; } - ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_ARMED << - INFINIPATH_IBCC_LINKCMD_SHIFT); + ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_ARMED, 0); + /* * Since the port can transition to ACTIVE by receiving * a non VL 15 packet, wait for either state. @@ -1753,8 +1769,7 @@ int ipath_set_linkstate(struct ipath_devdata *dd, u8 newstate) ret = -EINVAL; goto bail; } - ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_ACTIVE << - INFINIPATH_IBCC_LINKCMD_SHIFT); + ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKCMD_ACTIVE, 0); lstate = IPATH_LINKACTIVE; break; @@ -2026,8 +2041,7 @@ void ipath_shutdown_device(struct ipath_devdata *dd) */ udelay(5); - ipath_set_ib_lstate(dd, INFINIPATH_IBCC_LINKINITCMD_DISABLE << - INFINIPATH_IBCC_LINKINITCMD_SHIFT); + ipath_set_ib_lstate(dd, 0, INFINIPATH_IBCC_LINKINITCMD_DISABLE); ipath_cancel_sends(dd, 0); signal_ib_event(dd, IB_EVENT_PORT_ERR); diff --git a/drivers/infiniband/hw/ipath/ipath_init_chip.c b/drivers/infiniband/hw/ipath/ipath_init_chip.c index 524fdf7..46c7065 100644 --- a/drivers/infiniband/hw/ipath/ipath_init_chip.c +++ b/drivers/infiniband/hw/ipath/ipath_init_chip.c @@ -180,10 +180,13 @@ static int bringup_link(struct ipath_devdata *dd) /* * Want to start out with both LINKCMD and LINKINITCMD in NOP * (0 and 0). Don't put linkinitcmd in ipath_ibcctrl, want that - * to stay a NOP + * to stay a NOP. Flag that we are disabled, for the (unlikely) + * case that some recovery path is trying to bring the link up + * before we are ready. */ ibc |= INFINIPATH_IBCC_LINKINITCMD_DISABLE << INFINIPATH_IBCC_LINKINITCMD_SHIFT; + dd->ipath_flags |= IPATH_IB_LINK_DISABLED; ipath_cdbg(VERBOSE, "Writing 0x%llx to ibcctrl\n", (unsigned long long) ibc); ipath_write_kreg(dd, dd->ipath_kregs->kr_ibcctrl, ibc); diff --git a/drivers/infiniband/hw/ipath/ipath_intr.c b/drivers/infiniband/hw/ipath/ipath_intr.c index ed82ecb..5608e32 100644 --- a/drivers/infiniband/hw/ipath/ipath_intr.c +++ b/drivers/infiniband/hw/ipath/ipath_intr.c @@ -309,6 +309,8 @@ static void handle_e_ibstatuschanged(struct ipath_devdata *dd, lastlstate == INFINIPATH_IBCS_L_STATE_DOWN) { /* transitioned to UP */ if (dd->ipath_f_ib_updown(dd, 1, ibcs)) { + /* link came up, so we must no longer be disabled */ + dd->ipath_flags &= ~IPATH_IB_LINK_DISABLED; ipath_cdbg(LINKVERB, "LinkUp handled, skipped\n"); goto skip_ibchange; /* chip-code handled */ } diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 7a9121f..f7705fa 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -849,6 +849,8 @@ void ipath_hol_event(unsigned long); /* Suppress heartbeat, even if turning off loopback */ #define IPATH_NO_HRTBT 0x1000000 #define IPATH_HAS_MULT_IB_SPEED 0x8000000 + /* Linkdown-disable intentionally, Do not attempt to bring up */ +#define IPATH_IB_LINK_DISABLED 0x40000000 #define IPATH_IB_FORCE_NOTIFY 0x80000000 /* force notify on next ib change */ /* Bits in GPIO for the added interrupts */ diff --git a/drivers/infiniband/hw/ipath/ipath_mad.c b/drivers/infiniband/hw/ipath/ipath_mad.c index 7516a26..30b2f44 100644 --- a/drivers/infiniband/hw/ipath/ipath_mad.c +++ b/drivers/infiniband/hw/ipath/ipath_mad.c @@ -590,6 +590,10 @@ static int recv_subn_set_portinfo(struct ib_smp *smp, else goto err; ipath_set_linkstate(dd, lstate); + if (lstate == IPATH_IB_LINKDOWN_DISABLE) { + ret = IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_CONSUMED; + goto done; + } ipath_wait_linkstate(dd, IPATH_LINKINIT | IPATH_LINKARMED | IPATH_LINKACTIVE, 1000); break; From ralph.campbell at qlogic.com Tue Mar 25 11:06:33 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 25 Mar 2008 11:06:33 -0700 Subject: [ofa-general] [PATCH 5/6] IB/ipath - Use PIO buffer for RC ACKs In-Reply-To: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> References: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> Message-ID: <20080325180632.27615.62405.stgit@eng-46.mv.qlogic.com> This reduces the latency for RC ACKs when a PIO buffer is available. Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_rc.c | 57 ++++++++++++++++++++++---------- 1 files changed, 39 insertions(+), 18 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_rc.c b/drivers/infiniband/hw/ipath/ipath_rc.c index 40f3e37..f765d48 100644 --- a/drivers/infiniband/hw/ipath/ipath_rc.c +++ b/drivers/infiniband/hw/ipath/ipath_rc.c @@ -31,6 +31,8 @@ * SOFTWARE. */ +#include + #include "ipath_verbs.h" #include "ipath_kernel.h" @@ -585,19 +587,39 @@ bail: static void send_rc_ack(struct ipath_qp *qp) { struct ipath_ibdev *dev = to_idev(qp->ibqp.device); + struct ipath_devdata *dd; u16 lrh0; u32 bth0; u32 hwords; + u32 __iomem *piobuf; struct ipath_ib_header hdr; struct ipath_other_headers *ohdr; unsigned long flags; + spin_lock_irqsave(&qp->s_lock, flags); + /* Don't send ACK or NAK if a RDMA read or atomic is pending. */ if (qp->r_head_ack_queue != qp->s_tail_ack_queue || (qp->s_flags & IPATH_S_ACK_PENDING) || qp->s_ack_state != OP(ACKNOWLEDGE)) goto queue_ack; + spin_unlock_irqrestore(&qp->s_lock, flags); + + dd = dev->dd; + piobuf = ipath_getpiobuf(dd, 0, NULL); + if (!piobuf) { + /* + * We are out of PIO buffers at the moment. + * Pass responsibility for sending the ACK to the + * send tasklet so that when a PIO buffer becomes + * available, the ACK is sent ahead of other outgoing + * packets. + */ + spin_lock_irqsave(&qp->s_lock, flags); + goto queue_ack; + } + /* Construct the header. */ ohdr = &hdr.u.oth; lrh0 = IPATH_LRH_BTH; @@ -611,7 +633,7 @@ static void send_rc_ack(struct ipath_qp *qp) lrh0 = IPATH_LRH_GRH; } /* read pkey_index w/o lock (its atomic) */ - bth0 = ipath_get_pkey(dev->dd, qp->s_pkey_index) | + bth0 = ipath_get_pkey(dd, qp->s_pkey_index) | (OP(ACKNOWLEDGE) << 24) | (1 << 22); if (qp->r_nak_state) ohdr->u.aeth = cpu_to_be32((qp->r_msn & IPATH_MSN_MASK) | @@ -623,30 +645,29 @@ static void send_rc_ack(struct ipath_qp *qp) hdr.lrh[0] = cpu_to_be16(lrh0); hdr.lrh[1] = cpu_to_be16(qp->remote_ah_attr.dlid); hdr.lrh[2] = cpu_to_be16(hwords + SIZE_OF_CRC); - hdr.lrh[3] = cpu_to_be16(dev->dd->ipath_lid); + hdr.lrh[3] = cpu_to_be16(dd->ipath_lid); ohdr->bth[0] = cpu_to_be32(bth0); ohdr->bth[1] = cpu_to_be32(qp->remote_qpn); ohdr->bth[2] = cpu_to_be32(qp->r_ack_psn & IPATH_PSN_MASK); - /* - * If we can send the ACK, clear the ACK state. - */ - if (ipath_verbs_send(qp, &hdr, hwords, NULL, 0) == 0) { - dev->n_unicast_xmit++; - goto done; - } + writeq(hwords + 1, piobuf); - /* - * We are out of PIO buffers at the moment. - * Pass responsibility for sending the ACK to the - * send tasklet so that when a PIO buffer becomes - * available, the ACK is sent ahead of other outgoing - * packets. - */ - dev->n_rc_qacks++; + if (dd->ipath_flags & IPATH_PIO_FLUSH_WC) { + u32 *hdrp = (u32 *) &hdr; + + ipath_flush_wc(); + __iowrite32_copy(piobuf + 2, hdrp, hwords - 1); + ipath_flush_wc(); + __raw_writel(hdrp[hwords - 1], piobuf + hwords + 1); + } else + __iowrite32_copy(piobuf + 2, (u32 *) &hdr, hwords); + + ipath_flush_wc(); + + dev->n_unicast_xmit++; + goto done; queue_ack: - spin_lock_irqsave(&qp->s_lock, flags); dev->n_rc_qacks++; qp->s_flags |= IPATH_S_ACK_PENDING; qp->s_nak_state = qp->r_nak_state; From ralph.campbell at qlogic.com Tue Mar 25 11:06:38 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 25 Mar 2008 11:06:38 -0700 Subject: [ofa-general] [PATCH 6/6] IB/ipath - eeprom support for 7220 devices, robustness improvements, cleanup In-Reply-To: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> References: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> Message-ID: <20080325180638.27615.7220.stgit@eng-46.mv.qlogic.com> From: Michael Albaugh Add support for reading newer card's EEPROMs while continuing to support older EEPROMs. Also, add support for the temperature sensor if present. Signed-off-by: Michael Albaugh --- drivers/infiniband/hw/ipath/ipath_eeprom.c | 426 ++++++++++++++++++++++++---- drivers/infiniband/hw/ipath/ipath_kernel.h | 7 drivers/infiniband/hw/ipath/ipath_sysfs.c | 73 +++++ 3 files changed, 439 insertions(+), 67 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_eeprom.c b/drivers/infiniband/hw/ipath/ipath_eeprom.c index e28a42f..72f90e8 100644 --- a/drivers/infiniband/hw/ipath/ipath_eeprom.c +++ b/drivers/infiniband/hw/ipath/ipath_eeprom.c @@ -62,6 +62,33 @@ * accessing eeprom contents from within the kernel, only via sysfs. */ +/* Added functionality for IBA7220-based cards */ +#define IPATH_EEPROM_DEV_V1 0xA0 +#define IPATH_EEPROM_DEV_V2 0xA2 +#define IPATH_TEMP_DEV 0x98 +#define IPATH_BAD_DEV (IPATH_EEPROM_DEV_V2+2) +#define IPATH_NO_DEV (0xFF) + +/* + * The number of I2C chains is proliferating. Table below brings + * some order to the madness. The basic principle is that the + * table is scanned from the top, and a "probe" is made to the + * device probe_dev. If that succeeds, the chain is considered + * to be of that type, and dd->i2c_chain_type is set to the index+1 + * of the entry. + * The +1 is so static initialization can mean "unknown, do probe." + */ +static struct i2c_chain_desc { + u8 probe_dev; /* If seen at probe, chain is this type */ + u8 eeprom_dev; /* Dev addr (if any) for EEPROM */ + u8 temp_dev; /* Dev Addr (if any) for Temp-sense */ +} i2c_chains[] = { + { IPATH_BAD_DEV, IPATH_NO_DEV, IPATH_NO_DEV }, /* pre-iba7220 bds */ + { IPATH_EEPROM_DEV_V1, IPATH_EEPROM_DEV_V1, IPATH_TEMP_DEV}, /* V1 */ + { IPATH_EEPROM_DEV_V2, IPATH_EEPROM_DEV_V2, IPATH_TEMP_DEV}, /* V2 */ + { IPATH_NO_DEV } +}; + enum i2c_type { i2c_line_scl = 0, i2c_line_sda @@ -75,13 +102,6 @@ enum i2c_state { #define READ_CMD 1 #define WRITE_CMD 0 -static int eeprom_init; - -/* - * The gpioval manipulation really should be protected by spinlocks - * or be converted to use atomic operations. - */ - /** * i2c_gpio_set - set a GPIO line * @dd: the infinipath device @@ -241,6 +261,27 @@ static int i2c_ackrcv(struct ipath_devdata *dd) } /** + * rd_byte - read a byte, leaving ACK, STOP, etc up to caller + * @dd: the infinipath device + * + * Returns byte shifted out of device + */ +static int rd_byte(struct ipath_devdata *dd) +{ + int bit_cntr, data; + + data = 0; + + for (bit_cntr = 7; bit_cntr >= 0; --bit_cntr) { + data <<= 1; + scl_out(dd, i2c_line_high); + data |= sda_in(dd, 0); + scl_out(dd, i2c_line_low); + } + return data; +} + +/** * wr_byte - write a byte, one bit at a time * @dd: the infinipath device * @data: the byte to write @@ -331,7 +372,6 @@ static int eeprom_reset(struct ipath_devdata *dd) ipath_cdbg(VERBOSE, "Resetting i2c eeprom; initial gpioout reg " "is %llx\n", (unsigned long long) *gpioval); - eeprom_init = 1; /* * This is to get the i2c into a known state, by first going low, * then tristate sda (and then tristate scl as first thing @@ -340,12 +380,17 @@ static int eeprom_reset(struct ipath_devdata *dd) scl_out(dd, i2c_line_low); sda_out(dd, i2c_line_high); + /* Clock up to 9 cycles looking for SDA hi, then issue START and STOP */ while (clock_cycles_left--) { scl_out(dd, i2c_line_high); + /* SDA seen high, issue START by dropping it while SCL high */ if (sda_in(dd, 0)) { sda_out(dd, i2c_line_low); scl_out(dd, i2c_line_low); + /* ATMEL spec says must be followed by STOP. */ + scl_out(dd, i2c_line_high); + sda_out(dd, i2c_line_high); ret = 0; goto bail; } @@ -359,29 +404,121 @@ bail: return ret; } -/** - * ipath_eeprom_read - receives bytes from the eeprom via I2C - * @dd: the infinipath device - * @eeprom_offset: address to read from - * @buffer: where to store result - * @len: number of bytes to receive +/* + * Probe for I2C device at specified address. Returns 0 for "success" + * to match rest of this file. + * Leave bus in "reasonable" state for further commands. */ +static int i2c_probe(struct ipath_devdata *dd, int devaddr) +{ + int ret = 0; + + ret = eeprom_reset(dd); + if (ret) { + ipath_dev_err(dd, "Failed reset probing device 0x%02X\n", + devaddr); + return ret; + } + /* + * Reset no longer leaves bus in start condition, so normal + * i2c_startcmd() will do. + */ + ret = i2c_startcmd(dd, devaddr | READ_CMD); + if (ret) + ipath_cdbg(VERBOSE, "Failed startcmd for device 0x%02X\n", + devaddr); + else { + /* + * Device did respond. Complete a single-byte read, because some + * devices apparently cannot handle STOP immediately after they + * ACK the start-cmd. + */ + int data; + data = rd_byte(dd); + stop_cmd(dd); + ipath_cdbg(VERBOSE, "Response from device 0x%02X\n", devaddr); + } + return ret; +} + +/* + * Returns the "i2c type". This is a pointer to a struct that describes + * the I2C chain on this board. To minimize impact on struct ipath_devdata, + * the (small integer) index into the table is actually memoized, rather + * then the pointer. + * Memoization is because the type is determined on the first call per chip. + * An alternative would be to move type determination to early + * init code. + */ +static struct i2c_chain_desc *ipath_i2c_type(struct ipath_devdata *dd) +{ + int idx; + + /* Get memoized index, from previous successful probes */ + idx = dd->ipath_i2c_chain_type - 1; + if (idx >= 0 && idx < (ARRAY_SIZE(i2c_chains) - 1)) + goto done; + + idx = 0; + while (i2c_chains[idx].probe_dev != IPATH_NO_DEV) { + /* if probe succeeds, this is type */ + if (!i2c_probe(dd, i2c_chains[idx].probe_dev)) + break; + ++idx; + } + + /* + * Old EEPROM (first entry) may require a reset after probe, + * rather than being able to "start" after "stop" + */ + if (idx == 0) + eeprom_reset(dd); + + if (i2c_chains[idx].probe_dev == IPATH_NO_DEV) + idx = -1; + else + dd->ipath_i2c_chain_type = idx + 1; +done: + return (idx >= 0) ? i2c_chains + idx : NULL; +} static int ipath_eeprom_internal_read(struct ipath_devdata *dd, u8 eeprom_offset, void *buffer, int len) { - /* compiler complains unless initialized */ - u8 single_byte = 0; - int bit_cntr; int ret; + struct i2c_chain_desc *icd; + u8 *bp = buffer; - if (!eeprom_init) - eeprom_reset(dd); - - eeprom_offset = (eeprom_offset << 1) | READ_CMD; + ret = 1; + icd = ipath_i2c_type(dd); + if (!icd) + goto bail; - if (i2c_startcmd(dd, eeprom_offset)) { - ipath_dbg("Failed startcmd\n"); + if (icd->eeprom_dev == IPATH_NO_DEV) { + /* legacy not-really-I2C */ + ipath_cdbg(VERBOSE, "Start command only address\n"); + eeprom_offset = (eeprom_offset << 1) | READ_CMD; + ret = i2c_startcmd(dd, eeprom_offset); + } else { + /* Actual I2C */ + ipath_cdbg(VERBOSE, "Start command uses devaddr\n"); + if (i2c_startcmd(dd, icd->eeprom_dev | WRITE_CMD)) { + ipath_dbg("Failed EEPROM startcmd\n"); + stop_cmd(dd); + ret = 1; + goto bail; + } + ret = wr_byte(dd, eeprom_offset); + stop_cmd(dd); + if (ret) { + ipath_dev_err(dd, "Failed to write EEPROM address\n"); + ret = 1; + goto bail; + } + ret = i2c_startcmd(dd, icd->eeprom_dev | READ_CMD); + } + if (ret) { + ipath_dbg("Failed startcmd for dev %02X\n", icd->eeprom_dev); stop_cmd(dd); ret = 1; goto bail; @@ -392,22 +529,11 @@ static int ipath_eeprom_internal_read(struct ipath_devdata *dd, * incrementing the address. */ while (len-- > 0) { - /* get data */ - single_byte = 0; - for (bit_cntr = 8; bit_cntr; bit_cntr--) { - u8 bit; - scl_out(dd, i2c_line_high); - bit = sda_in(dd, 0); - single_byte |= bit << (bit_cntr - 1); - scl_out(dd, i2c_line_low); - } - + /* get and store data */ + *bp++ = rd_byte(dd); /* send ack if not the last byte */ if (len) send_ack(dd); - - *((u8 *) buffer) = single_byte; - buffer++; } stop_cmd(dd); @@ -418,31 +544,40 @@ bail: return ret; } - -/** - * ipath_eeprom_write - writes data to the eeprom via I2C - * @dd: the infinipath device - * @eeprom_offset: where to place data - * @buffer: data to write - * @len: number of bytes to write - */ static int ipath_eeprom_internal_write(struct ipath_devdata *dd, u8 eeprom_offset, const void *buffer, int len) { - u8 single_byte; int sub_len; const u8 *bp = buffer; int max_wait_time, i; int ret; + struct i2c_chain_desc *icd; - if (!eeprom_init) - eeprom_reset(dd); + ret = 1; + icd = ipath_i2c_type(dd); + if (!icd) + goto bail; while (len > 0) { - if (i2c_startcmd(dd, (eeprom_offset << 1) | WRITE_CMD)) { - ipath_dbg("Failed to start cmd offset %u\n", - eeprom_offset); - goto failed_write; + if (icd->eeprom_dev == IPATH_NO_DEV) { + if (i2c_startcmd(dd, + (eeprom_offset << 1) | WRITE_CMD)) { + ipath_dbg("Failed to start cmd offset %u\n", + eeprom_offset); + goto failed_write; + } + } else { + /* Real I2C */ + if (i2c_startcmd(dd, icd->eeprom_dev | WRITE_CMD)) { + ipath_dbg("Failed EEPROM startcmd\n"); + goto failed_write; + } + ret = wr_byte(dd, eeprom_offset); + if (ret) { + ipath_dev_err(dd, "Failed to write EEPROM " + "address\n"); + goto failed_write; + } } sub_len = min(len, 4); @@ -468,9 +603,11 @@ static int ipath_eeprom_internal_write(struct ipath_devdata *dd, u8 eeprom_offse * the writes have completed. We do this inline to avoid * the debug prints that are in the real read routine * if the startcmd fails. + * We also use the proper device address, so it doesn't matter + * whether we have real eeprom_dev. legacy likes any address. */ max_wait_time = 100; - while (i2c_startcmd(dd, READ_CMD)) { + while (i2c_startcmd(dd, icd->eeprom_dev | READ_CMD)) { stop_cmd(dd); if (!--max_wait_time) { ipath_dbg("Did not get successful read to " @@ -478,15 +615,8 @@ static int ipath_eeprom_internal_write(struct ipath_devdata *dd, u8 eeprom_offse goto failed_write; } } - /* now read the zero byte */ - for (i = single_byte = 0; i < 8; i++) { - u8 bit; - scl_out(dd, i2c_line_high); - bit = sda_in(dd, 0); - scl_out(dd, i2c_line_low); - single_byte <<= 1; - single_byte |= bit; - } + /* now read (and ignore) the resulting byte */ + rd_byte(dd); stop_cmd(dd); } @@ -501,9 +631,12 @@ bail: return ret; } -/* - * The public entry-points ipath_eeprom_read() and ipath_eeprom_write() - * are now just wrappers around the internal functions. +/** + * ipath_eeprom_read - receives bytes from the eeprom via I2C + * @dd: the infinipath device + * @eeprom_offset: address to read from + * @buffer: where to store result + * @len: number of bytes to receive */ int ipath_eeprom_read(struct ipath_devdata *dd, u8 eeprom_offset, void *buff, int len) @@ -519,6 +652,13 @@ int ipath_eeprom_read(struct ipath_devdata *dd, u8 eeprom_offset, return ret; } +/** + * ipath_eeprom_write - writes data to the eeprom via I2C + * @dd: the infinipath device + * @eeprom_offset: where to place data + * @buffer: data to write + * @len: number of bytes to write + */ int ipath_eeprom_write(struct ipath_devdata *dd, u8 eeprom_offset, const void *buff, int len) { @@ -820,7 +960,7 @@ int ipath_update_eeprom_log(struct ipath_devdata *dd) * if we log an hour at 31 minutes, then we would need to set * active_time to -29 to accurately count the _next_ hour. */ - if (new_time > 3600) { + if (new_time >= 3600) { new_hrs = new_time / 3600; atomic_sub((new_hrs * 3600), &dd->ipath_active_time); new_hrs += dd->ipath_eep_hrs; @@ -885,3 +1025,159 @@ void ipath_inc_eeprom_err(struct ipath_devdata *dd, u32 eidx, u32 incr) spin_unlock_irqrestore(&dd->ipath_eep_st_lock, flags); return; } + +static int ipath_tempsense_internal_read(struct ipath_devdata *dd, u8 regnum) +{ + int ret; + struct i2c_chain_desc *icd; + + ret = -ENOENT; + + icd = ipath_i2c_type(dd); + if (!icd) + goto bail; + + if (icd->temp_dev == IPATH_NO_DEV) { + /* tempsense only exists on new, real-I2C boards */ + ret = -ENXIO; + goto bail; + } + + if (i2c_startcmd(dd, icd->temp_dev | WRITE_CMD)) { + ipath_dbg("Failed tempsense startcmd\n"); + stop_cmd(dd); + ret = -ENXIO; + goto bail; + } + ret = wr_byte(dd, regnum); + stop_cmd(dd); + if (ret) { + ipath_dev_err(dd, "Failed tempsense WR command %02X\n", + regnum); + ret = -ENXIO; + goto bail; + } + if (i2c_startcmd(dd, icd->temp_dev | READ_CMD)) { + ipath_dbg("Failed tempsense RD startcmd\n"); + stop_cmd(dd); + ret = -ENXIO; + goto bail; + } + /* + * We can only clock out one byte per command, sensibly + */ + ret = rd_byte(dd); + stop_cmd(dd); + +bail: + return ret; +} + +#define VALID_TS_RD_REG_MASK 0xBF + +/** + * ipath_tempsense_read - read register of temp sensor via I2C + * @dd: the infinipath device + * @regnum: register to read from + * + * returns reg contents (0..255) or < 0 for error + */ +int ipath_tempsense_read(struct ipath_devdata *dd, u8 regnum) +{ + int ret; + + if (regnum > 7) + return -EINVAL; + + /* return a bogus value for (the one) register we do not have */ + if (!((1 << regnum) & VALID_TS_RD_REG_MASK)) + return 0; + + ret = mutex_lock_interruptible(&dd->ipath_eep_lock); + if (!ret) { + ret = ipath_tempsense_internal_read(dd, regnum); + mutex_unlock(&dd->ipath_eep_lock); + } + + /* + * There are three possibilities here: + * ret is actual value (0..255) + * ret is -ENXIO or -EINVAL from code in this file + * ret is -EINTR from mutex_lock_interruptible. + */ + return ret; +} + +static int ipath_tempsense_internal_write(struct ipath_devdata *dd, + u8 regnum, u8 data) +{ + int ret = -ENOENT; + struct i2c_chain_desc *icd; + + icd = ipath_i2c_type(dd); + if (!icd) + goto bail; + + if (icd->temp_dev == IPATH_NO_DEV) { + /* tempsense only exists on new, real-I2C boards */ + ret = -ENXIO; + goto bail; + } + if (i2c_startcmd(dd, icd->temp_dev | WRITE_CMD)) { + ipath_dbg("Failed tempsense startcmd\n"); + stop_cmd(dd); + ret = -ENXIO; + goto bail; + } + ret = wr_byte(dd, regnum); + if (ret) { + stop_cmd(dd); + ipath_dev_err(dd, "Failed to write tempsense command %02X\n", + regnum); + ret = -ENXIO; + goto bail; + } + ret = wr_byte(dd, data); + stop_cmd(dd); + ret = i2c_startcmd(dd, icd->temp_dev | READ_CMD); + if (ret) { + ipath_dev_err(dd, "Failed tempsense data wrt to %02X\n", + regnum); + ret = -ENXIO; + } + +bail: + return ret; +} + +#define VALID_TS_WR_REG_MASK ((1 << 9) | (1 << 0xB) | (1 << 0xD)) + +/** + * ipath_tempsense_write - write register of temp sensor via I2C + * @dd: the infinipath device + * @regnum: register to write + * @data: data to write + * + * returns 0 for success or < 0 for error + */ +int ipath_tempsense_write(struct ipath_devdata *dd, u8 regnum, u8 data) +{ + int ret; + + if (regnum > 15 || !((1 << regnum) & VALID_TS_WR_REG_MASK)) + return -EINVAL; + + ret = mutex_lock_interruptible(&dd->ipath_eep_lock); + if (!ret) { + ret = ipath_tempsense_internal_write(dd, regnum, data); + mutex_unlock(&dd->ipath_eep_lock); + } + + /* + * There are three possibilities here: + * ret is 0 for success + * ret is -ENXIO or -EINVAL from code in this file + * ret is -EINTR from mutex_lock_interruptible. + */ + return ret; +} diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index 069a9db..f10442f 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -654,8 +654,9 @@ struct ipath_devdata { * Register bits for selecting i2c direction and values, used for * I2C serial flash. */ - u16 ipath_gpio_sda_num; - u16 ipath_gpio_scl_num; + u8 ipath_gpio_sda_num; + u8 ipath_gpio_scl_num; + u8 ipath_i2c_chain_type; u64 ipath_gpio_sda; u64 ipath_gpio_scl; @@ -907,6 +908,8 @@ void ipath_release_user_pages(struct page **, size_t); void ipath_release_user_pages_on_close(struct page **, size_t); int ipath_eeprom_read(struct ipath_devdata *, u8, void *, int); int ipath_eeprom_write(struct ipath_devdata *, u8, const void *, int); +int ipath_tempsense_read(struct ipath_devdata *, u8 regnum); +int ipath_tempsense_write(struct ipath_devdata *, u8 regnum, u8 data); /* these are used for the registers that vary with port */ void ipath_write_kreg_port(const struct ipath_devdata *, ipath_kreg, diff --git a/drivers/infiniband/hw/ipath/ipath_sysfs.c b/drivers/infiniband/hw/ipath/ipath_sysfs.c index bb41c3f..7961d26 100644 --- a/drivers/infiniband/hw/ipath/ipath_sysfs.c +++ b/drivers/infiniband/hw/ipath/ipath_sysfs.c @@ -943,6 +943,7 @@ invalid: bail: return ret; } + /* * Get/Set RX lane-reversal enable. 0=no, 1=yes. */ @@ -997,6 +998,75 @@ static struct attribute_group driver_attr_group = { .attrs = driver_attributes }; +static ssize_t store_tempsense(struct device *dev, + struct device_attribute *attr, + const char *buf, + size_t count) +{ + struct ipath_devdata *dd = dev_get_drvdata(dev); + int ret, stat; + u16 val; + + ret = ipath_parse_ushort(buf, &val); + if (ret <= 0) { + ipath_dev_err(dd, "attempt to set invalid tempsense config\n"); + goto bail; + } + /* If anything but the highest limit, enable T_CRIT_A "interrupt" */ + stat = ipath_tempsense_write(dd, 9, (val == 0x7f7f) ? 0x80 : 0); + if (stat) { + ipath_dev_err(dd, "Unable to set tempsense config\n"); + ret = -1; + goto bail; + } + stat = ipath_tempsense_write(dd, 0xB, (u8) (val & 0xFF)); + if (stat) { + ipath_dev_err(dd, "Unable to set local Tcrit\n"); + ret = -1; + goto bail; + } + stat = ipath_tempsense_write(dd, 0xD, (u8) (val >> 8)); + if (stat) { + ipath_dev_err(dd, "Unable to set remote Tcrit\n"); + ret = -1; + goto bail; + } + +bail: + return ret; +} + +/* + * dump tempsense regs. in decimal, to ease shell-scripts. + */ +static ssize_t show_tempsense(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct ipath_devdata *dd = dev_get_drvdata(dev); + int ret; + int idx; + u8 regvals[8]; + + ret = -ENXIO; + for (idx = 0; idx < 8; ++idx) { + if (idx == 6) + continue; + ret = ipath_tempsense_read(dd, idx); + if (ret < 0) + break; + regvals[idx] = ret; + } + if (idx == 8) + ret = scnprintf(buf, PAGE_SIZE, "%d %d %02X %02X %d %d\n", + *(signed char *)(regvals), + *(signed char *)(regvals + 1), + regvals[2], regvals[3], + *(signed char *)(regvals + 5), + *(signed char *)(regvals + 7)); + return ret; +} + struct attribute_group *ipath_driver_attr_groups[] = { &driver_attr_group, NULL, @@ -1025,6 +1095,8 @@ static DEVICE_ATTR(jint_max_packets, S_IWUSR | S_IRUGO, show_jint_max_packets, store_jint_max_packets); static DEVICE_ATTR(jint_idle_ticks, S_IWUSR | S_IRUGO, show_jint_idle_ticks, store_jint_idle_ticks); +static DEVICE_ATTR(tempsense, S_IWUSR | S_IRUGO, + show_tempsense, store_tempsense); static struct attribute *dev_attributes[] = { &dev_attr_guid.attr, @@ -1044,6 +1116,7 @@ static struct attribute *dev_attributes[] = { &dev_attr_rx_pol_inv.attr, &dev_attr_led_override.attr, &dev_attr_logged_errors.attr, + &dev_attr_tempsense.attr, &dev_attr_localbus_info.attr, NULL }; From ralph.campbell at qlogic.com Tue Mar 25 11:06:27 2008 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 25 Mar 2008 11:06:27 -0700 Subject: [ofa-general] [PATCH 4/6] IB/ipath - make send buffers available for kernel if not allocated to user In-Reply-To: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> References: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> Message-ID: <20080325180627.27615.80469.stgit@eng-46.mv.qlogic.com> A fixed partitioning of send buffers is determined at driver load time for user processes and kernel use. Since send buffers are a scarce resource, it makes sense to allow the kernel to use the buffers if they are not in use by a user process. Also, eliminate code duplication for ipath_force_pio_avail_update(). Signed-off-by: Ralph Campbell --- drivers/infiniband/hw/ipath/ipath_diag.c | 6 drivers/infiniband/hw/ipath/ipath_driver.c | 306 ++++++++++++++++--------- drivers/infiniband/hw/ipath/ipath_file_ops.c | 21 -- drivers/infiniband/hw/ipath/ipath_init_chip.c | 8 - drivers/infiniband/hw/ipath/ipath_intr.c | 14 - drivers/infiniband/hw/ipath/ipath_kernel.h | 11 + drivers/infiniband/hw/ipath/ipath_registers.h | 2 drivers/infiniband/hw/ipath/ipath_verbs.c | 2 8 files changed, 225 insertions(+), 145 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_diag.c b/drivers/infiniband/hw/ipath/ipath_diag.c index 96a1c41..af59bf3 100644 --- a/drivers/infiniband/hw/ipath/ipath_diag.c +++ b/drivers/infiniband/hw/ipath/ipath_diag.c @@ -439,7 +439,9 @@ static ssize_t ipath_diagpkt_write(struct file *fp, goto bail; } - piobuf = ipath_getpiobuf(dd, &pbufn); + plen >>= 2; /* in dwords */ + + piobuf = ipath_getpiobuf(dd, plen, &pbufn); if (!piobuf) { ipath_cdbg(VERBOSE, "No PIO buffers avail unit for %u\n", dd->ipath_unit); @@ -449,8 +451,6 @@ static ssize_t ipath_diagpkt_write(struct file *fp, /* disarm it just to be extra sure */ ipath_disarm_piobufs(dd, pbufn, 1); - plen >>= 2; /* in dwords */ - if (ipath_debug & __IPATH_PKTDBG) ipath_cdbg(VERBOSE, "unit %u 0x%x+1w pio%d\n", dd->ipath_unit, plen - 1, pbufn); diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index dfd19de..dfa009a 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c @@ -317,7 +317,7 @@ static void ipath_verify_pioperf(struct ipath_devdata *dd) u32 *addr; u64 msecs, emsecs; - piobuf = ipath_getpiobuf(dd, &pbnum); + piobuf = ipath_getpiobuf(dd, 0, &pbnum); if (!piobuf) { dev_info(&dd->pcidev->dev, "No PIObufs for checking perf, skipping\n"); @@ -836,20 +836,8 @@ void ipath_disarm_piobufs(struct ipath_devdata *dd, unsigned first, ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); spin_unlock_irqrestore(&dd->ipath_sendctrl_lock, flags); } - - /* - * Disable PIOAVAILUPD, then re-enable, reading scratch in - * between. This seems to avoid a chip timing race that causes - * pioavail updates to memory to stop. We xor as we don't - * know the state of the bit when we're called. - */ - spin_lock_irqsave(&dd->ipath_sendctrl_lock, flags); - ipath_write_kreg(dd, dd->ipath_kregs->kr_sendctrl, - dd->ipath_sendctrl ^ INFINIPATH_S_PIOBUFAVAILUPD); - ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); - ipath_write_kreg(dd, dd->ipath_kregs->kr_sendctrl, - dd->ipath_sendctrl); - spin_unlock_irqrestore(&dd->ipath_sendctrl_lock, flags); + /* on some older chips, update may not happen after cancel */ + ipath_force_pio_avail_update(dd); } /** @@ -1314,7 +1302,6 @@ static void ipath_update_pio_bufs(struct ipath_devdata *dd) * happens when all buffers are in use, so only cpu overhead, not * latency or bandwidth is affected. */ -#define _IPATH_ALL_CHECKBITS 0x5555555555555555ULL if (!dd->ipath_pioavailregs_dma) { ipath_dbg("Update shadow pioavail, but regs_dma NULL!\n"); return; @@ -1359,7 +1346,7 @@ static void ipath_update_pio_bufs(struct ipath_devdata *dd) piov = le64_to_cpu(dd->ipath_pioavailregs_dma[i ^ 1]); else piov = le64_to_cpu(dd->ipath_pioavailregs_dma[i]); - pchg = _IPATH_ALL_CHECKBITS & + pchg = dd->ipath_pioavailkernel[i] & ~(dd->ipath_pioavailshadow[i] ^ piov); pchbusy = pchg << INFINIPATH_SENDPIOAVAIL_BUSY_SHIFT; if (pchg && (pchbusy & dd->ipath_pioavailshadow[i])) { @@ -1410,27 +1397,63 @@ int ipath_setrcvhdrsize(struct ipath_devdata *dd, unsigned rhdrsize) return ret; } -/** - * ipath_getpiobuf - find an available pio buffer - * @dd: the infinipath device - * @pbufnum: the buffer number is placed here +/* + * debugging code and stats updates if no pio buffers available. + */ +static noinline void no_pio_bufs(struct ipath_devdata *dd) +{ + unsigned long *shadow = dd->ipath_pioavailshadow; + __le64 *dma = (__le64 *)dd->ipath_pioavailregs_dma; + + dd->ipath_upd_pio_shadow = 1; + + /* + * not atomic, but if we lose a stat count in a while, that's OK + */ + ipath_stats.sps_nopiobufs++; + if (!(++dd->ipath_consec_nopiobuf % 100000)) { + ipath_dbg("%u pio sends with no bufavail; dmacopy: " + "%llx %llx %llx %llx; shadow: %lx %lx %lx %lx\n", + dd->ipath_consec_nopiobuf, + (unsigned long long) le64_to_cpu(dma[0]), + (unsigned long long) le64_to_cpu(dma[1]), + (unsigned long long) le64_to_cpu(dma[2]), + (unsigned long long) le64_to_cpu(dma[3]), + shadow[0], shadow[1], shadow[2], shadow[3]); + /* + * 4 buffers per byte, 4 registers above, cover rest + * below + */ + if ((dd->ipath_piobcnt2k + dd->ipath_piobcnt4k) > + (sizeof(shadow[0]) * 4 * 4)) + ipath_dbg("2nd group: dmacopy: %llx %llx " + "%llx %llx; shadow: %lx %lx %lx %lx\n", + (unsigned long long)le64_to_cpu(dma[4]), + (unsigned long long)le64_to_cpu(dma[5]), + (unsigned long long)le64_to_cpu(dma[6]), + (unsigned long long)le64_to_cpu(dma[7]), + shadow[4], shadow[5], shadow[6], + shadow[7]); + } +} + +/* + * common code for normal driver pio buffer allocation, and reserved + * allocation. * * do appropriate marking as busy, etc. * returns buffer number if one found (>=0), negative number is error. - * Used by ipath_layer_send */ -u32 __iomem *ipath_getpiobuf(struct ipath_devdata *dd, u32 * pbufnum) +static u32 __iomem *ipath_getpiobuf_range(struct ipath_devdata *dd, + u32 *pbufnum, u32 first, u32 last, u32 firsti) { - int i, j, starti, updated = 0; - unsigned piobcnt, iter; + int i, j, updated = 0; + unsigned piobcnt; unsigned long flags; unsigned long *shadow = dd->ipath_pioavailshadow; u32 __iomem *buf; - piobcnt = (unsigned)(dd->ipath_piobcnt2k - + dd->ipath_piobcnt4k); - starti = dd->ipath_lastport_piobuf; - iter = piobcnt - starti; + piobcnt = last - first; if (dd->ipath_upd_pio_shadow) { /* * Minor optimization. If we had no buffers on last call, @@ -1438,12 +1461,10 @@ u32 __iomem *ipath_getpiobuf(struct ipath_devdata *dd, u32 * pbufnum) * if no buffers were updated, to be paranoid */ ipath_update_pio_bufs(dd); - /* we scanned here, don't do it at end of scan */ - updated = 1; - i = starti; + updated++; + i = first; } else - i = dd->ipath_lastpioindex; - + i = firsti; rescan: /* * while test_and_set_bit() is atomic, we do that and then the @@ -1451,104 +1472,141 @@ rescan: * of the remaining armlaunch errors. */ spin_lock_irqsave(&ipath_pioavail_lock, flags); - for (j = 0; j < iter; j++, i++) { - if (i >= piobcnt) - i = starti; - /* - * To avoid bus lock overhead, we first find a candidate - * buffer, then do the test and set, and continue if that - * fails. - */ - if (test_bit((2 * i) + 1, shadow) || - test_and_set_bit((2 * i) + 1, shadow)) + for (j = 0; j < piobcnt; j++, i++) { + if (i >= last) + i = first; + if (__test_and_set_bit((2 * i) + 1, shadow)) continue; /* flip generation bit */ - change_bit(2 * i, shadow); + __change_bit(2 * i, shadow); break; } spin_unlock_irqrestore(&ipath_pioavail_lock, flags); - if (j == iter) { - volatile __le64 *dma = dd->ipath_pioavailregs_dma; - - /* - * first time through; shadow exhausted, but may be real - * buffers available, so go see; if any updated, rescan - * (once) - */ + if (j == piobcnt) { if (!updated) { + /* + * first time through; shadow exhausted, but may be + * buffers available, try an update and then rescan. + */ ipath_update_pio_bufs(dd); - updated = 1; - i = starti; + updated++; + i = first; goto rescan; - } - dd->ipath_upd_pio_shadow = 1; - /* - * not atomic, but if we lose one once in a while, that's OK - */ - ipath_stats.sps_nopiobufs++; - if (!(++dd->ipath_consec_nopiobuf % 100000)) { - ipath_dbg( - "%u pio sends with no bufavail; dmacopy: " - "%llx %llx %llx %llx; shadow: " - "%lx %lx %lx %lx\n", - dd->ipath_consec_nopiobuf, - (unsigned long long) le64_to_cpu(dma[0]), - (unsigned long long) le64_to_cpu(dma[1]), - (unsigned long long) le64_to_cpu(dma[2]), - (unsigned long long) le64_to_cpu(dma[3]), - shadow[0], shadow[1], shadow[2], - shadow[3]); + } else if (updated == 1 && piobcnt <= + ((dd->ipath_sendctrl + >> INFINIPATH_S_UPDTHRESH_SHIFT) & + INFINIPATH_S_UPDTHRESH_MASK)) { /* - * 4 buffers per byte, 4 registers above, cover rest - * below + * for chips supporting and using the update + * threshold we need to force an update of the + * in-memory copy if the count is less than the + * thershold, then check one more time. */ - if ((dd->ipath_piobcnt2k + dd->ipath_piobcnt4k) > - (sizeof(shadow[0]) * 4 * 4)) - ipath_dbg("2nd group: dmacopy: %llx %llx " - "%llx %llx; shadow: %lx %lx " - "%lx %lx\n", - (unsigned long long) - le64_to_cpu(dma[4]), - (unsigned long long) - le64_to_cpu(dma[5]), - (unsigned long long) - le64_to_cpu(dma[6]), - (unsigned long long) - le64_to_cpu(dma[7]), - shadow[4], shadow[5], - shadow[6], shadow[7]); + ipath_force_pio_avail_update(dd); + ipath_update_pio_bufs(dd); + updated++; + i = first; + goto rescan; } + + no_pio_bufs(dd); buf = NULL; - goto bail; + } else { + if (i < dd->ipath_piobcnt2k) + buf = (u32 __iomem *) (dd->ipath_pio2kbase + + i * dd->ipath_palign); + else + buf = (u32 __iomem *) + (dd->ipath_pio4kbase + + (i - dd->ipath_piobcnt2k) * dd->ipath_4kalign); + if (pbufnum) + *pbufnum = i; } - /* - * set next starting place. Since it's just an optimization, - * it doesn't matter who wins on this, so no locking - */ - dd->ipath_lastpioindex = i + 1; - if (dd->ipath_upd_pio_shadow) - dd->ipath_upd_pio_shadow = 0; - if (dd->ipath_consec_nopiobuf) - dd->ipath_consec_nopiobuf = 0; - if (i < dd->ipath_piobcnt2k) - buf = (u32 __iomem *) (dd->ipath_pio2kbase + - i * dd->ipath_palign); - else - buf = (u32 __iomem *) - (dd->ipath_pio4kbase + - (i - dd->ipath_piobcnt2k) * dd->ipath_4kalign); - ipath_cdbg(VERBOSE, "Return piobuf%u %uk @ %p\n", - i, (i < dd->ipath_piobcnt2k) ? 2 : 4, buf); - if (pbufnum) - *pbufnum = i; + return buf; +} -bail: +/** + * ipath_getpiobuf - find an available pio buffer + * @dd: the infinipath device + * @plen: the size of the PIO buffer needed in 32-bit words + * @pbufnum: the buffer number is placed here + */ +u32 __iomem *ipath_getpiobuf(struct ipath_devdata *dd, u32 plen, u32 *pbufnum) +{ + u32 __iomem *buf; + u32 pnum, nbufs; + u32 first, lasti; + + if (plen + 1 >= IPATH_SMALLBUF_DWORDS) { + first = dd->ipath_piobcnt2k; + lasti = dd->ipath_lastpioindexl; + } else { + first = 0; + lasti = dd->ipath_lastpioindex; + } + nbufs = dd->ipath_piobcnt2k + dd->ipath_piobcnt4k; + buf = ipath_getpiobuf_range(dd, &pnum, first, nbufs, lasti); + + if (buf) { + /* + * Set next starting place. It's just an optimization, + * it doesn't matter who wins on this, so no locking + */ + if (plen + 1 >= IPATH_SMALLBUF_DWORDS) + dd->ipath_lastpioindexl = pnum + 1; + else + dd->ipath_lastpioindex = pnum + 1; + if (dd->ipath_upd_pio_shadow) + dd->ipath_upd_pio_shadow = 0; + if (dd->ipath_consec_nopiobuf) + dd->ipath_consec_nopiobuf = 0; + ipath_cdbg(VERBOSE, "Return piobuf%u %uk @ %p\n", + pnum, (pnum < dd->ipath_piobcnt2k) ? 2 : 4, buf); + if (pbufnum) + *pbufnum = pnum; + + } return buf; } /** + * ipath_chg_pioavailkernel - change which send buffers are available for kernel + * @dd: the infinipath device + * @start: the starting send buffer number + * @len: the number of send buffers + * @avail: true if the buffers are available for kernel use, false otherwise + */ +void ipath_chg_pioavailkernel(struct ipath_devdata *dd, unsigned start, + unsigned len, int avail) +{ + unsigned long flags; + unsigned end; + + /* There are two bits per send buffer (busy and generation) */ + start *= 2; + len *= 2; + end = start + len; + + /* Set or clear the generation bits. */ + spin_lock_irqsave(&ipath_pioavail_lock, flags); + while (start < end) { + if (avail) { + __clear_bit(start + INFINIPATH_SENDPIOAVAIL_BUSY_SHIFT, + dd->ipath_pioavailshadow); + __set_bit(start, dd->ipath_pioavailkernel); + } else { + __set_bit(start + INFINIPATH_SENDPIOAVAIL_BUSY_SHIFT, + dd->ipath_pioavailshadow); + __clear_bit(start, dd->ipath_pioavailkernel); + } + start += 2; + } + spin_unlock_irqrestore(&ipath_pioavail_lock, flags); +} + +/** * ipath_create_rcvhdrq - create a receive header queue * @dd: the infinipath device * @pd: the port data @@ -1664,6 +1722,30 @@ void ipath_cancel_sends(struct ipath_devdata *dd, int restore_sendctrl) ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); } +/* + * Force an update of in-memory copy of the pioavail registers, when + * needed for any of a variety of reasons. We read the scratch register + * to make it highly likely that the update will have happened by the + * time we return. If already off (as in cancel_sends above), this + * routine is a nop, on the assumption that the caller will "do the + * right thing". + */ +void ipath_force_pio_avail_update(struct ipath_devdata *dd) +{ + unsigned long flags; + + spin_lock_irqsave(&dd->ipath_sendctrl_lock, flags); + if (dd->ipath_sendctrl & INFINIPATH_S_PIOBUFAVAILUPD) { + ipath_write_kreg(dd, dd->ipath_kregs->kr_sendctrl, + dd->ipath_sendctrl & ~INFINIPATH_S_PIOBUFAVAILUPD); + ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); + ipath_write_kreg(dd, dd->ipath_kregs->kr_sendctrl, + dd->ipath_sendctrl); + ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); + } + spin_unlock_irqrestore(&dd->ipath_sendctrl_lock, flags); +} + static void ipath_set_ib_lstate(struct ipath_devdata *dd, int linkcmd, int linitcmd) { diff --git a/drivers/infiniband/hw/ipath/ipath_file_ops.c b/drivers/infiniband/hw/ipath/ipath_file_ops.c index cddf29b..1b232b2 100644 --- a/drivers/infiniband/hw/ipath/ipath_file_ops.c +++ b/drivers/infiniband/hw/ipath/ipath_file_ops.c @@ -1603,6 +1603,9 @@ static int try_alloc_port(struct ipath_devdata *dd, int port, port_fp(fp) = pd; pd->port_pid = current->pid; strncpy(pd->port_comm, current->comm, sizeof(pd->port_comm)); + ipath_chg_pioavailkernel(dd, + dd->ipath_pbufsport * (pd->port_port - 1), + dd->ipath_pbufsport, 0); ipath_stats.sps_ports++; ret = 0; } else @@ -2081,6 +2084,7 @@ static int ipath_close(struct inode *in, struct file *fp) i = dd->ipath_pbufsport * (port - 1); ipath_disarm_piobufs(dd, i, dd->ipath_pbufsport); + ipath_chg_pioavailkernel(dd, i, dd->ipath_pbufsport, 1); dd->ipath_f_clear_tids(dd, pd->port_port); @@ -2145,21 +2149,6 @@ static int ipath_get_slave_info(struct ipath_portdata *pd, return ret; } -static int ipath_force_pio_avail_update(struct ipath_devdata *dd) -{ - unsigned long flags; - - spin_lock_irqsave(&dd->ipath_sendctrl_lock, flags); - ipath_write_kreg(dd, dd->ipath_kregs->kr_sendctrl, - dd->ipath_sendctrl & ~INFINIPATH_S_PIOBUFAVAILUPD); - ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); - ipath_write_kreg(dd, dd->ipath_kregs->kr_sendctrl, dd->ipath_sendctrl); - ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); - spin_unlock_irqrestore(&dd->ipath_sendctrl_lock, flags); - - return 0; -} - static ssize_t ipath_write(struct file *fp, const char __user *data, size_t count, loff_t *off) { @@ -2304,7 +2293,7 @@ static ssize_t ipath_write(struct file *fp, const char __user *data, cmd.cmd.slave_mask_addr); break; case IPATH_CMD_PIOAVAILUPD: - ret = ipath_force_pio_avail_update(pd->port_dd); + ipath_force_pio_avail_update(pd->port_dd); break; case IPATH_CMD_POLL_TYPE: pd->poll_type = cmd.cmd.poll_type; diff --git a/drivers/infiniband/hw/ipath/ipath_init_chip.c b/drivers/infiniband/hw/ipath/ipath_init_chip.c index 46c7065..786a5e0 100644 --- a/drivers/infiniband/hw/ipath/ipath_init_chip.c +++ b/drivers/infiniband/hw/ipath/ipath_init_chip.c @@ -521,7 +521,9 @@ static void enable_chip(struct ipath_devdata *dd, pioavail = dd->ipath_pioavailregs_dma[i ^ 1]; else pioavail = dd->ipath_pioavailregs_dma[i]; - dd->ipath_pioavailshadow[i] = le64_to_cpu(pioavail); + dd->ipath_pioavailshadow[i] = le64_to_cpu(pioavail) | + (~dd->ipath_pioavailkernel[i] << + INFINIPATH_SENDPIOAVAIL_BUSY_SHIFT); } /* can get counters, stats, etc. */ dd->ipath_flags |= IPATH_PRESENT; @@ -743,7 +745,9 @@ int ipath_init_chip(struct ipath_devdata *dd, int reinit) ipath_dbg("%u pbufs/port leaves %u unused, add to kernel\n", dd->ipath_pbufsport, val32); } - dd->ipath_lastpioindex = dd->ipath_lastport_piobuf; + dd->ipath_lastpioindex = 0; + dd->ipath_lastpioindexl = dd->ipath_piobcnt2k; + ipath_chg_pioavailkernel(dd, 0, piobufs, 1); ipath_cdbg(VERBOSE, "%d PIO bufs for kernel out of %d total %u " "each for %u user ports\n", kpiobufs, piobufs, dd->ipath_pbufsport, uports); diff --git a/drivers/infiniband/hw/ipath/ipath_intr.c b/drivers/infiniband/hw/ipath/ipath_intr.c index 5608e32..d1e13a4 100644 --- a/drivers/infiniband/hw/ipath/ipath_intr.c +++ b/drivers/infiniband/hw/ipath/ipath_intr.c @@ -804,7 +804,6 @@ void ipath_clear_freeze(struct ipath_devdata *dd) { int i, im; u64 val; - unsigned long flags; /* disable error interrupts, to avoid confusion */ ipath_write_kreg(dd, dd->ipath_kregs->kr_errormask, 0ULL); @@ -823,14 +822,7 @@ void ipath_clear_freeze(struct ipath_devdata *dd) dd->ipath_control); /* ensure pio avail updates continue */ - spin_lock_irqsave(&dd->ipath_sendctrl_lock, flags); - ipath_write_kreg(dd, dd->ipath_kregs->kr_sendctrl, - dd->ipath_sendctrl & ~INFINIPATH_S_PIOBUFAVAILUPD); - ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); - ipath_write_kreg(dd, dd->ipath_kregs->kr_sendctrl, - dd->ipath_sendctrl); - ipath_read_kreg64(dd, dd->ipath_kregs->kr_scratch); - spin_unlock_irqrestore(&dd->ipath_sendctrl_lock, flags); + ipath_force_pio_avail_update(dd); /* * We just enabled pioavailupdate, so dma copy is almost certainly @@ -842,7 +834,9 @@ void ipath_clear_freeze(struct ipath_devdata *dd) i ^ 1 : i; val = ipath_read_kreg64(dd, (0x1000 / sizeof(u64)) + im); dd->ipath_pioavailregs_dma[i] = cpu_to_le64(val); - dd->ipath_pioavailshadow[i] = val; + dd->ipath_pioavailshadow[i] = val | + (~dd->ipath_pioavailkernel[i] << + INFINIPATH_SENDPIOAVAIL_BUSY_SHIFT); } /* diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h index f7705fa..069a9db 100644 --- a/drivers/infiniband/hw/ipath/ipath_kernel.h +++ b/drivers/infiniband/hw/ipath/ipath_kernel.h @@ -191,6 +191,9 @@ struct ipath_skbinfo { dma_addr_t phys; }; +/* max dwords in small buffer packet */ +#define IPATH_SMALLBUF_DWORDS (dd->ipath_piosize2k >> 2) + /* * Possible IB config parameters for ipath_f_get/set_ib_cfg() */ @@ -366,6 +369,7 @@ struct ipath_devdata { * get to multiple devices */ u32 ipath_lastpioindex; + u32 ipath_lastpioindexl; /* max length of freezemsg */ u32 ipath_freezelen; /* @@ -453,6 +457,8 @@ struct ipath_devdata { * init time. */ unsigned long ipath_pioavailshadow[8]; + /* bitmap of send buffers available for the kernel to use with PIO. */ + unsigned long ipath_pioavailkernel[8]; /* shadow of kr_gpio_out, for rmw ops */ u64 ipath_gpio_out; /* shadow the gpio mask register */ @@ -870,13 +876,16 @@ void ipath_hol_event(unsigned long); /* free up any allocated data at closes */ void ipath_free_data(struct ipath_portdata *dd); -u32 __iomem *ipath_getpiobuf(struct ipath_devdata *, u32 *); +u32 __iomem *ipath_getpiobuf(struct ipath_devdata *, u32, u32 *); +void ipath_chg_pioavailkernel(struct ipath_devdata *dd, unsigned start, + unsigned len, int avail); void ipath_init_iba6120_funcs(struct ipath_devdata *); void ipath_init_iba6110_funcs(struct ipath_devdata *); void ipath_get_eeprom_info(struct ipath_devdata *); int ipath_update_eeprom_log(struct ipath_devdata *dd); void ipath_inc_eeprom_err(struct ipath_devdata *dd, u32 eidx, u32 incr); u64 ipath_snap_cntr(struct ipath_devdata *, ipath_creg); +void ipath_force_pio_avail_update(struct ipath_devdata *); void signal_ib_event(struct ipath_devdata *dd, enum ib_event_type ev); /* diff --git a/drivers/infiniband/hw/ipath/ipath_registers.h b/drivers/infiniband/hw/ipath/ipath_registers.h index 16d0d74..61e5621 100644 --- a/drivers/infiniband/hw/ipath/ipath_registers.h +++ b/drivers/infiniband/hw/ipath/ipath_registers.h @@ -66,6 +66,8 @@ /* kr_sendctrl bits */ #define INFINIPATH_S_DISARMPIOBUF_SHIFT 16 +#define INFINIPATH_S_UPDTHRESH_SHIFT 24 +#define INFINIPATH_S_UPDTHRESH_MASK 0x1f #define IPATH_S_ABORT 0 #define IPATH_S_PIOINTBUFAVAIL 1 diff --git a/drivers/infiniband/hw/ipath/ipath_verbs.c b/drivers/infiniband/hw/ipath/ipath_verbs.c index 2f9bc29..2e6b6f6 100644 --- a/drivers/infiniband/hw/ipath/ipath_verbs.c +++ b/drivers/infiniband/hw/ipath/ipath_verbs.c @@ -875,7 +875,7 @@ static int ipath_verbs_send_pio(struct ipath_qp *qp, u32 *hdr, u32 hdrwords, unsigned flush_wc; int ret; - piobuf = ipath_getpiobuf(dd, NULL); + piobuf = ipath_getpiobuf(dd, plen, NULL); if (unlikely(piobuf == NULL)) { ret = -EBUSY; goto bail; From weiny2 at llnl.gov Tue Mar 25 11:40:56 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 25 Mar 2008 11:40:56 -0700 Subject: [ofa-general] Re: [PATCH 5/5] Move extern declarations for node description processing to a new header file. In-Reply-To: <20080324021413.GF2102@sashak.voltaire.com> References: <20080321155125.170f629c.weiny2@llnl.gov> <20080324021413.GF2102@sashak.voltaire.com> Message-ID: <20080325114056.3e3e42e5.weiny2@llnl.gov> On Mon, 24 Mar 2008 02:14:13 +0000 Sasha Khapyorsky wrote: > Hi Ira, > > On 15:51 Fri 21 Mar , Ira Weiny wrote: > > From 93479e6cac589167f79f09d42900994e947c90f4 Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Fri, 21 Mar 2008 15:23:16 -0700 > > Subject: [PATCH] Move extern declarations for node description processing to a new header file. > > > > > > Signed-off-by: Ira K. Weiny > > --- > > opensm/include/opensm/osm_node_desc.h | 54 +++++++++++++++++++++++++++++++++ > > opensm/opensm/osm_node_desc_rcv.c | 27 ++++++++++++++++ > > opensm/opensm/osm_node_info_rcv.c | 26 ---------------- > > opensm/opensm/osm_sm.c | 2 +- > > opensm/opensm/osm_trap_rcv.c | 3 +- > > 5 files changed, 83 insertions(+), 29 deletions(-) > > create mode 100644 opensm/include/opensm/osm_node_desc.h > > > > diff --git a/opensm/include/opensm/osm_node_desc.h b/opensm/include/opensm/osm_node_desc.h > > new file mode 100644 > > index 0000000..d86f6ba > > --- /dev/null > > +++ b/opensm/include/opensm/osm_node_desc.h > > When new file is added it should be also added to EXTRA_DIST list in > include/Makefile.am so 'make dist' will work. My bad, I should have checked the dist and rpm builds... Sorry. > > > @@ -0,0 +1,54 @@ > > +/* > > + * Copyright (c) 2008 Lawrence Livermore National Security > > + * > > + * This software is available to you under a choice of one of two > > + * licenses. You may choose to be licensed under the terms of the GNU > > + * General Public License (GPL) Version 2, available from the file > > + * COPYING in the main directory of this source tree, or the > > + * OpenIB.org BSD license below: > > + * > > + * Redistribution and use in source and binary forms, with or > > + * without modification, are permitted provided that the following > > + * conditions are met: > > + * > > + * - Redistributions of source code must retain the above > > + * copyright notice, this list of conditions and the following > > + * disclaimer. > > + * > > + * - Redistributions in binary form must reproduce the above > > + * copyright notice, this list of conditions and the following > > + * disclaimer in the documentation and/or other materials > > + * provided with the distribution. > > + * > > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > > + * SOFTWARE. > > + * > > + */ > > + > > +#ifndef _OSM_NODE_DESC_H_ > > +#define _OSM_NODE_DESC_H_ > > + > > +#include > > +#include > > + > > +#ifdef __cplusplus > > +# define BEGIN_C_DECLS extern "C" { > > +# define END_C_DECLS } > > +#else /* !__cplusplus */ > > +# define BEGIN_C_DECLS > > +# define END_C_DECLS > > +#endif /* __cplusplus */ > > + > > +BEGIN_C_DECLS > > + > > +void osm_nd_rcv_process(void *context, void *data); > > +void osm_req_get_node_desc(osm_sm_t * sm, osm_physp_t *p_physp); > > + > > +END_C_DECLS > > +#endif /* _OSM_NODE_DESC_H_ */ > > diff --git a/opensm/opensm/osm_node_desc_rcv.c b/opensm/opensm/osm_node_desc_rcv.c > > index 4a22aab..a7266d5 100644 > > --- a/opensm/opensm/osm_node_desc_rcv.c > > +++ b/opensm/opensm/osm_node_desc_rcv.c > > @@ -134,3 +134,30 @@ void osm_nd_rcv_process(IN void *context, IN void *data) > > CL_PLOCK_RELEASE(sm->p_lock); > > OSM_LOG_EXIT(sm->p_log); > > } > > + > > +/********************************************************************** > > + The plock must be held before calling this function. > > +**********************************************************************/ > > +void > > +osm_req_get_node_desc(IN osm_sm_t * sm, > > + osm_physp_t *p_physp) > > +{ > > + ib_api_status_t status = IB_SUCCESS; > > + osm_madw_context_t context; > > + > > + OSM_LOG_ENTER(sm->p_log); > > + > > + context.nd_context.node_guid = > > + osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); > > + > > + status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), > > + IB_MAD_ATTR_NODE_DESC, > > + 0, CL_DISP_MSGID_NONE, &context); > > + if (status != IB_SUCCESS) > > + OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " > > + "Failure initiating NodeDescription request (%s)\n", > > + ib_get_err_str(status)); > > + > > + OSM_LOG_EXIT(sm->p_log); > > +} > > + > > diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c > > index 6818e05..9a3eb7b 100644 > > --- a/opensm/opensm/osm_node_info_rcv.c > > +++ b/opensm/opensm/osm_node_info_rcv.c > > @@ -306,32 +306,6 @@ __osm_ni_rcv_process_new_node(IN osm_sm_t * sm, > > /********************************************************************** > > The plock must be held before calling this function. > > **********************************************************************/ > > -void > > -osm_req_get_node_desc(IN osm_sm_t * sm, > > - osm_physp_t *p_physp) > > -{ > > - ib_api_status_t status = IB_SUCCESS; > > - osm_madw_context_t context; > > - > > - OSM_LOG_ENTER(sm->p_log); > > - > > - context.nd_context.node_guid = > > - osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); > > - > > - status = osm_req_get(sm, osm_physp_get_dr_path_ptr(p_physp), > > - IB_MAD_ATTR_NODE_DESC, > > - 0, CL_DISP_MSGID_NONE, &context); > > - if (status != IB_SUCCESS) > > - OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 0D03: " > > - "Failure initiating NodeDescription request (%s)\n", > > - ib_get_err_str(status)); > > - > > - OSM_LOG_EXIT(sm->p_log); > > -} > > - > > -/********************************************************************** > > - The plock must be held before calling this function. > > -**********************************************************************/ > > Do we need this move? I think this is appropriate to move all the node description functionality to one file. I think it is awkward to have a file called osm_node_desc_rcv.c and then have a function called osm_req_get_node_desc in a file with the name *node_info*. Obviously grep finds all, but I thought this would be better for the organization of the code. > > > static void > > __osm_ni_rcv_get_node_desc(IN osm_sm_t * sm, > > IN osm_node_t * const p_node, > > diff --git a/opensm/opensm/osm_sm.c b/opensm/opensm/osm_sm.c > > index 32525ba..69aafb4 100644 > > --- a/opensm/opensm/osm_sm.c > > +++ b/opensm/opensm/osm_sm.c > > @@ -64,12 +64,12 @@ > > #include > > #include > > #include > > +#include > > > > #define OSM_SM_INITIAL_TID_VALUE 0x1233 > > > > extern void osm_lft_rcv_process(IN void *context, IN void *data); > > extern void osm_mft_rcv_process(IN void *context, IN void *data); > > -extern void osm_nd_rcv_process(IN void *context, IN void *data); > > extern void osm_ni_rcv_process(IN void *context, IN void *data); > > extern void osm_pkey_rcv_process(IN void *context, IN void *data); > > extern void osm_pi_rcv_process(IN void *context, IN void *data); > > Isn't it would be better to put all those declarations to n one place > (let's say osm_sm.h) instead of creating new - one function prototype > header file? I'm not sure... But there is already an osm_sm.h file and it has the functions for the osm_sm_t object. For all these functions I think I would call the file something like osm_rcv_callback.h. Since these all define callbacks used when MADs of those types are received. However, the problem I was trying to solve is where to put the declaration of osm_req_get_node_desc. At the time it seemed best to put it with other node_desc type functions but perhaps it would be best to create another c file with generic MAD sends in it? I am not sure what it would be called at this time. Ira From sashak at voltaire.com Tue Mar 25 12:16:23 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 25 Mar 2008 19:16:23 +0000 Subject: [ofa-general] Re: [PATCH 5/5] Move extern declarations for node description processing to a new header file. In-Reply-To: <20080325114056.3e3e42e5.weiny2@llnl.gov> References: <20080321155125.170f629c.weiny2@llnl.gov> <20080324021413.GF2102@sashak.voltaire.com> <20080325114056.3e3e42e5.weiny2@llnl.gov> Message-ID: <20080325191623.GC6773@sashak.voltaire.com> Hi Ira, On 11:40 Tue 25 Mar , Ira Weiny wrote: > > I think this is appropriate to move all the node description functionality to > one file. I think it is awkward to have a file called osm_node_desc_rcv.c and > then have a function called osm_req_get_node_desc in a file with the name > *node_info*. osm_node_desc_rcv.c contains node description receiving things and osm_req_get_node_desc() is sending stuff. So I'm not sure that is is perfect mix. All this is pretty minor although. > > > > > > extern void osm_lft_rcv_process(IN void *context, IN void *data); > > > extern void osm_mft_rcv_process(IN void *context, IN void *data); > > > -extern void osm_nd_rcv_process(IN void *context, IN void *data); > > > extern void osm_ni_rcv_process(IN void *context, IN void *data); > > > extern void osm_pkey_rcv_process(IN void *context, IN void *data); > > > extern void osm_pi_rcv_process(IN void *context, IN void *data); > > > > Isn't it would be better to put all those declarations to n one place > > (let's say osm_sm.h) instead of creating new - one function prototype > > header file? > > I'm not sure... But there is already an osm_sm.h file and it has the functions > for the osm_sm_t object. For all these functions I think I would call the file > something like osm_rcv_callback.h. Since these all define callbacks used when > MADs of those types are received. However, the problem I was trying to solve > is where to put the declaration of osm_req_get_node_desc. Yes, I see. But the idea behind local prototyping was that all those functions could not be used easily in places other than osm_sm.c. This is the case with osm_nd_rcv_process() function, but obviously not with osm_req_get_node_desc(). > At the time it > seemed best to put it with other node_desc type functions but perhaps it would > be best to create another c file with generic MAD sends in it? It could be interesting, we need to see how it will look, if it consolidates the code it should be good. Sasha From sashak at voltaire.com Tue Mar 25 12:39:39 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 25 Mar 2008 19:39:39 +0000 Subject: [ofa-general] Re: [PATCH V2][TRIVIAL] management: Remove extraneous semicolon from several files In-Reply-To: <200803251354.48321.dotanb@dev.mellanox.co.il> References: <200803251354.48321.dotanb@dev.mellanox.co.il> Message-ID: <20080325193939.GE6773@sashak.voltaire.com> On 13:54 Tue 25 Mar , Dotan Barak wrote: > Remove extraneous semicolon from several components in the management. > > Signed-off-by: Dotan Barak Applied. Thanks. Sasha From panda at cse.ohio-state.edu Tue Mar 25 12:49:30 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Tue, 25 Mar 2008 15:49:30 -0400 (EDT) Subject: [ofa-general] RE: [ewg] OFED March 24 meeting summary on OFED 1.4 plans In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9039C9215@mtlexch01.mtl.com> Message-ID: Aviram, > Thanks. What about XRC support? Do you plan it for MVAPICH 1.1? Yes, we have a plan for it. Thanks, DK > > -----Original Message----- > From: ewg-bounces at lists.openfabrics.org > [mailto:ewg-bounces at lists.openfabrics.org] On Behalf Of Dhabaleswar > Panda > Sent: Monday, March 24, 2008 11:06 PM > To: Tziporet Koren > Cc: ewg at lists.openfabrics.org; general at lists.openfabrics.org > Subject: Re: [ewg] OFED March 24 meeting summary on OFED 1.4 plans > > Tziporet, > > We had some constraints here today and none of us could attend the > conference call. > > > > * MPI: > > > * Open MPI 1.3 > > > * APM support in MPI > > > * mvapich ??? > > Regarding your question, it will be MVAPICH 1.1 and MVAPICH2 1.1 for > OFED 1.4. APM support is already available in MVAPICH and MVAPICH2 for > quite some time. > > Thanks, > > DK > > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > From gsn at boehl.com Tue Mar 25 13:08:17 2008 From: gsn at boehl.com (Abe Ho) Date: Tue, 25 Mar 2008 21:08:17 +0100 Subject: [ofa-general] Master in bed games Message-ID: <01c88ebc$58827680$0f171453@gsn> Take her to love heaven Look attached details and find MORE! -------------- next part -------------- A non-text attachment was scrubbed... Name: vo.zip Type: application/zip Size: 636 bytes Desc: not available URL: From CareyspireMayberry at saudinf.com Tue Mar 25 15:30:35 2008 From: CareyspireMayberry at saudinf.com (Vilma Waldron) Date: Tue, 25 Mar 2008 21:30:35 -0100 Subject: [ofa-general] Wall Street Alert Message-ID: Medical suuplies is one of the hottest sectors currently And the timing is very opportune. With cutting edge technology and state of the ART sortwares to do both improve complex patient care and improve profits for medical practitioners, we believe we have a winner in ZYTO CORP Symbol : ZYTC. Looks like the selling is over, Read the PR's and research ZYTO Corp and see why this is a value stock at bargain price. It's due and ripe for a breakout. Ride the easy 10 bagger Sym: ZYTC -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Tue Mar 25 13:35:25 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Tue, 25 Mar 2008 13:35:25 -0700 Subject: [ofa-general] [PATCH] infiniband-diags/vendstat.c: Fix port xmit wait handling Message-ID: <1206477325.8099.675.camel@hrosenstock-ws.xsigo.com> Fix config space accesses for port xmit wait Signed-off-by: Hal Rosenstock diff --git a/infiniband-diags/src/vendstat.c b/infiniband-diags/src/vendstat.c index 22a1bed..589318f 100644 --- a/infiniband-diags/src/vendstat.c +++ b/infiniband-diags/src/vendstat.c @@ -103,11 +103,6 @@ typedef struct { } is3_record_t; typedef struct { - uint8_t reserved[40]; - is3_record_t record[18]; -} is3_config_space_req_t; - -typedef struct { uint8_t reserved[8]; is3_record_t record[18]; } is3_config_space_t; @@ -147,7 +142,6 @@ main(int argc, char **argv) int ca_port = 0; ib_vendor_call_t call; is3_general_info_t *gi; - is3_config_space_req_t *csr; is3_config_space_t *cs; int general_info = 0; int xmit_wait = 0; @@ -282,32 +276,32 @@ main(int argc, char **argv) /* Limit of 18 accesses per MAD ? */ call.mod = 2 << 22 | 16 << 16; /* 16 records */ /* Set record addresses for each port */ - csr = (is3_config_space_req_t *)&buf; + cs = (is3_config_space_t *)&buf; for (i = 0; i < 16; i++) - csr->record[i].address = htonl(IB_MLX_IS3_PORT_XMIT_WAIT | (i + 1) << 12); + cs->record[i].address = htonl(IB_MLX_IS3_PORT_XMIT_WAIT + ((i + 1) << 12)); if (!ib_vendor_call(&buf, &portid, &call)) IBERROR("vendstat"); - cs = (is3_config_space_t *)&buf; for (i = 0; i < 16; i++) if (cs->record[i].data) /* PortXmitWait is 32 bit counter */ - printf("Port %d: PortXmitWait 0x%x\n", i + 1, ntohl(cs->record[i].data)); + printf("Port %d: PortXmitWait 0x%x\n", i + 4, ntohl(cs->record[i].data)); /* port 4 is first port */ /* Last 8 ports is another query */ memset(&buf, 0, sizeof(buf)); call.attrid = IB_MLX_IS3_CONFIG_SPACE_ACCESS; call.mod = 2 << 22 | 8 << 16; /* 8 records */ /* Set record addresses for each port */ - csr = (is3_config_space_req_t *)&buf; + cs = (is3_config_space_t *)&buf; for (i = 0; i < 8; i++) - csr->record[i].address = htonl(IB_MLX_IS3_PORT_XMIT_WAIT | (i + 17) << 12); + cs->record[i].address = htonl(IB_MLX_IS3_PORT_XMIT_WAIT + ((i + 17) << 12)); if (!ib_vendor_call(&buf, &portid, &call)) IBERROR("vendstat"); - cs = (is3_config_space_t *)&buf; for (i = 0; i < 8; i++) if (cs->record[i].data) /* PortXmitWait is 32 bit counter */ - printf("Port %d: PortXmitWait 0x%x\n", i + 17, ntohl(cs->record[i].data)); + printf("Port %d: PortXmitWait 0x%x\n", + i < 4 ? i + 21 : i - 3, + ntohl(cs->record[i].data)); } exit(0); From LeocorpsmanPayne at eurekalert.org Tue Mar 25 05:46:25 2008 From: LeocorpsmanPayne at eurekalert.org (Pedro Perkins) Date: Tue, 25 Mar 2008 17:46:25 +0500 Subject: [ofa-general] Investor Insight Message-ID: <2IX649EJXVWDA006@eurekalert.org> An HTML attachment was scrubbed... URL: From LesterguntherHarper at gaynors.com Wed Mar 26 07:55:52 2008 From: LesterguntherHarper at gaynors.com (Sam Kelley) Date: Wed, 26 Mar 2008 06:55:52 -0800 Subject: [ofa-general] Hot Stock Talk Message-ID: <0IX723EJXVWDA698@gaynors.com> Medical suuplies is one of the hottest sectors currently And the timing is very opportune. With cutting edge technology and state of the ART sortwares to do both improve complex patient care and improve profits for medical practitioners, we believe we have a winner in ZYTO CORP Symbol : ZYTC. after today's selloff we believe the stock has found strong support at present range Additionally look for another news release for huge contracts. This is not a fly by night. Real company and real products used by real health care professionals It's due and ripe for a breakout. Ride the easy 10 bagger Sym: ZYTC From misjhlw at bradleyallen.com.au Tue Mar 25 16:15:31 2008 From: misjhlw at bradleyallen.com.au (Lukas Huggins) Date: Wed, 26 Mar 2008 05:15:31 +0600 Subject: [ofa-general] Potenzprobleme? Ab jetzt nicht mehr Message-ID: <01c88f00$6954db80$19002f5c@misjhlw> Online anonym bestellen - original Qualitat - 100% wirksam Fakten von unseren Kunden: - Sex ist befriedigender denn je. Stress und Leistungsdruck verschwinden. Sie ist nie wieder frustriert, ich habe keine Angst mehr zu versagen. Es ist ein wundervolles korperliches Erlebnis, dem ein genauso tiefes Gefuhl folgt. - Die Nebenwirkungen sind minimal: manchmal eine verstopfte Nase, kurzzeitig ein roter Kopf - kein Kopfschmerz, sondern das Gefuhl, als wurde man eine Flasche eiskalte Cola in einem Zug trinken. - Interessanterweise macht eine Vi. allein noch keinen Stander. Man(n) muss wenigstens ein bisschen Lust auf Sex mit der Frau haben. Gegen eine Eiserne Jungfrau im Bett hilft auch die grosste Dosis nichts. Wer aber das erste Kribbeln in den Lenden spurt, wird einen stahlharten Stander haben, und das fur wenigstens vier Stunden. - Eine volle 100-mg-Dosis macht den Schwanz zum Schwert. Wer es ubertreibt, ist Schuld, wenn die Herzallerliebste am Ende einen Y-formigen Sarg braucht. Fur die meisten von uns sind 50 mg mehr als genug, wenn man das gute Stuck zwischen den Hohepunkten auch mal hangen lassen will ... zur Not hilft es da vielleicht, sich ein nacktes Grossmutterchen vorzustellen. - Wer noch Zeit und Lust fur eine schnelle Nummer am nachsten Morgen hat, sollte dafur sorgen, dann noch genug Vi. im Blut zu haben - damit es noch fur ein oder zwei "Stehaufmannchen" reicht. - Das Beste an Vi. ist die Sicherheit, dass man "mit Autopilot fliegt", dass man entspannt und ohne Sorgen zur Sache kommen kann, dass der Stander auch halt, auch wenn man unterbrochen wird (die Kinder klopfen an die Schlafzimmertur, der Hund bellt, das Kondom sitzt schlecht). Wenn man Vi. bewusst anwendet, kann es auch der Partnerin gegenuber ein grosses Geschenk sein. Nur ein Rat: Sagen Sie ihr nicht, dass Sie es verwenden, das weibliche Selbstwertgefuhl ist genauso verletzlich wie das unsere. Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro - kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - kein peinlicher Arztbesuch erforderlich - diskrete Zahlung - keine versteckte Kosten - Visa verifizierter Onlineshop - kostenlose, arztliche Telefon-Beratung - bequem und diskret online bestellen. - diskrete Verpackung Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen http://severalnatural.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Tue Mar 25 17:12:44 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 00:12:44 +0000 Subject: [ofa-general] Re: [PATCH] opensm/QoS: setting SL in the IPoIB MCast groups In-Reply-To: <47E89564.1040608@dev.mellanox.co.il> References: <47E7BF68.7050104@dev.mellanox.co.il> <20080324162930.GJ595@sashak.voltaire.com> <47E89564.1040608@dev.mellanox.co.il> Message-ID: <20080326001244.GK6773@sashak.voltaire.com> On 08:02 Tue 25 Mar , Yevgeny Kliteynik wrote: >> Why to not use 'osm_mgrp_t *mgr' instead of mlid and to not access mcast >> group directly w/out qmap search? > > No special reason. Didn't want to think what would happen if the mcast > group was removed for some reason and the partition will still have the > pointer. Now it is impossible because partition creates only "well known" mc groups. Although this could change some days. I'm fine with your approach. Sasha From sashak at voltaire.com Tue Mar 25 17:17:15 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 00:17:15 +0000 Subject: [ofa-general] Re: [PATCH] opensm/QoS: setting SL in the IPoIB MCast groups In-Reply-To: <47E7BF68.7050104@dev.mellanox.co.il> References: <47E7BF68.7050104@dev.mellanox.co.il> Message-ID: <20080326001715.GL6773@sashak.voltaire.com> On 16:49 Mon 24 Mar , Yevgeny Kliteynik wrote: > Hi Sasha, > > This is a reworked patch that sets SL in the IPoIB mcast groups > Note that I'm using functions here! :) > > Added mcast mlid to the partition structure. > This mlid is used later by the QoS manager for quick access > to the mcast group. > > The patch is for master only. > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From kliteyn at dev.mellanox.co.il Tue Mar 25 17:47:08 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 26 Mar 2008 02:47:08 +0200 Subject: [ofa-general] [PATCH] opensm/man: Adding QoS-related info to opensm man pages Message-ID: <47E99D0C.7040403@dev.mellanox.co.il> Hi Sasha, I've added QoS related info to opensm man pages: enhanced existing part (that was talking about VL arbitration) and added description of QoS manager in accordance with QoS annex. Please apply to ofed_1_3 and master. Signed-off-by: Yevgeny Kliteynik --- opensm/man/opensm.8.in | 501 +++++++++++++++++++++++++++++++++++++++++++----- 1 files changed, 457 insertions(+), 44 deletions(-) diff --git a/opensm/man/opensm.8.in b/opensm/man/opensm.8.in index 5322ab7..1d9c5b7 100644 --- a/opensm/man/opensm.8.in +++ b/opensm/man/opensm.8.in @@ -35,7 +35,8 @@ to initialize the InfiniBand hardware (at least one per each InfiniBand subnet). opensm also now contains an experimental version of a performance -manager as well. +manager and an experimental version QoS manager (in accordance with +IBA QoS Annex). opensm defaults were designed to meet the common case usage on clusters with up to a few hundred nodes. Thus, in this default mode, opensm will scan the IB fabric, initialize it, and sweep occasionally for changes. @@ -433,51 +434,463 @@ partition manager: Default=0x7fff,ipoib:ALL=full; -.SH QOS CONFIGURATION +.SH QUALITY OF SERVICE .PP -There are a set of QoS related low-level configuration parameters. -All these parameter names are prefixed by "qos_" string. Here is a full -list of these parameters: - - qos_max_vls - The maximum number of VLs that will be on the subnet - qos_high_limit - The limit of High Priority component of VL - Arbitration table (IBA 7.6.9) - qos_vlarb_low - Low priority VL Arbitration table (IBA 7.6.9) - template - qos_vlarb_high - High priority VL Arbitration table (IBA 7.6.9) - template - Both VL arbitration templates are pairs of - VL and weight - qos_sl2vl - SL2VL Mapping table (IBA 7.6.6) template. It is - a list of VLs corresponding to SLs 0-15 (Note - that VL15 used here means drop this SL) - -Typical default values (hard-coded in OpenSM initialization) are: - - qos_max_vls=15 - qos_high_limit=0 - qos_vlarb_low=0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4 - qos_vlarb_high=0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0 - qos_sl2vl=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 - -The syntax is compatible with rest of OpenSM configuration options and -values may be stored in OpenSM config file (cached options file). - -In addition to the above, we may define separate QoS configuration -parameters sets for various target types. As targets, we currently support -CAs, routers, switch external ports, and switch's enhanced port 0. The -names of such specialized parameters are prefixed by "qos__" -string. Here is a full list of the currently supported sets: - - qos_ca_ - QoS configuration parameters set for CAs. - qos_rtr_ - parameters set for routers. - qos_sw0_ - parameters set for switches' port 0. - qos_swe_ - parameters set for switches' external ports. +OpenSM QoS support comprises of two parts: -Examples: - qos_sw0_max_vls=2 - qos_ca_sl2vl=0,1,2,3,5,5,5,12,12,0, - qos_swe_high_limit=0 + 1. \fBQoS manager in accordance with IBA QoS Annex\fP (experimental) +.P + 2. \fBSL2VL and VL Arbitration tables configuration\fP +.P +.SS QoS Manager (experimental) +.PP +When Quality of Service in OpenSM is enabled (-Q or --qos), OpenSM looks +for QoS Policy file. The default name of this file is +\fB\%@CONF_DIR@/@QOS_POLICY_FILE@\fP. The default may be changed by using +-Y or --qos_policy_file option with OpenSM. + +During fabric initialization and at every heavy sweep OpenSM parses the +QoS policy file, applies its settings to the discovered fabric elements, +and enforces the provided policy on client requests. The overall flow for +such requests is as follows: + - The request is matched against the defined matching rules such that + the QoS Level definition is found. + - Given the QoS Level, path(s) search is performed with the given + restrictions imposed by that level. + +There are two ways to define QoS policy: + - \fBFull\fP: the full policy file syntax provides the administrator various + ways to match a PathRecord/MultiPathRecord (PR/MPR) request, and to + enforce various QoS constraints on the requested PR/MPR. + - \fBSimplified\fP: the simplified policy file syntax enables the administrator + match PR/MPR requests by various ULPs and applications running on top of + these ULPs. + +While the full policy syntax is very flexible, in many cases the simplified +policy definition would be sufficient. +.PP +.B Full QoS Policy File +.PP +QoS policy file has the following sections: + +.B I) +Port Groups (denoted by port-groups). +This section defines zero or more port groups that can be referred later by +matching rules (see below). Port group lists ports by: + - Port GUID + - Port name, which is a combination of NodeDescription and IB port number + - PKey, which means that all the ports in the subnet that belong to + partition with a given PKey belong to this port group + - Partition name, which means that all the ports in the subnet that belong + to partition with a given name belong to this port group + - Node type, where possible node types are: CA, SWITCH, ROUTER, ALL, and + SELF (SM's port). + +.B II) +QoS Setup (denoted by qos-setup). +This section describes how to set up SL2VL and VL Arbitration tables on +various nodes in the fabric. +However, this is not supported in OFED 1.3. +SL2VL and VLArb tables should be configured in the OpenSM options file. + +.B III) +QoS Levels (denoted by qos-levels). +Each QoS Level defines Service Level (SL) and a few optional fields: + - MTU limit + - Rate limit + - PKey + - Packet lifetime + +When path(s) search is performed, it is done with regards to restriction that +these QoS Level parameters impose. +One QoS level that is mandatory to define is a DEFAULT QoS level. It is +applied to a PR/MPR query that does not match any existing match rule. +Similar to any other QoS Level, it can also be explicitly referred by any +match rule. + +.B IV) +QoS Matching Rules (denoted by qos-match-rules). +Each PathRecord/MultiPathRecord query that OpenSM receives is matched against +the set of matching rules. Rules are scanned in order of appearance in the QoS +policy file such as the first match takes precedence. +Each rule has a name of QoS level that will be applied to the matching query. +A default QoS level is applied to a query that did not match any rule. +Queries can be matched by: + - Source port group (whether a source port is a member of a specified group) + - Destination port group (same as above, only for destination port) + - PKey + - QoS class + - Service ID + +To match a certain matching rule, PR/MPR query has to match ALL the rule's +criteria. However, not all the fields of the PR/MPR query have to appear in +the matching rule. +For instance, if the rule has a single criterion - Service ID, it will match +any query that has this Service ID, disregarding rest of the query fields. +However, if a certain query has only Service ID (which means that this is the +only bit in the PR/MPR component mask that is on), it will not match any rule +that has other matching criteria besides Service ID. +.PP +.B Simplified QoS Policy Definition +.PP +Simplified QoS policy definition comprises of a single section denoted by +qos-ulps. Similar to the full QoS policy, it has a list of match rules and +their QoS Level, but in this case a match rule has only one criterion - its +goal is to match a certain ULP (or a certain application on top of this ULP) +PR/MPR request, and QoS Level has only one constraint - Service Level (SL). +The simplified policy section may appear in the policy file in combine with +the full policy, or as a stand-alone policy definition. +See more details and list of match rule criteria below. +.PP +.B Policy File Syntax Guidelines +.PP +Empty lines are ignored. +Leading and trailing blanks, as well as empty lines, are ignored, so the +indentation in the example is just for better readability. +Comments are started with the pound sign (#) and terminated by EOL. +Any keyword should be the first non-blank in the line, unless it's a comment. +Keywords that denote section/subsection start have matching closing keywords. +Having a QoS Level named "DEFAULT" is a must - it is applied to PR/MPR +requests that didn't match any of the matching rules. +Any section/subsection of the policy file is optional. + +.PP +.B Examples of Full Policy File +.PP +As mentioned earlier, any section of the policy file is optional, and +the only mandatory part of the policy file is a default QoS Level. +Here's an example of the shortest policy file: + + qos-levels + qos-level + name: DEFAULT + sl: 0 + end-qos-level + end-qos-levels + +Port groups section is missing because there are no match rules, which means +that port groups are not referred anywhere, and there is no need defining +them. And since this policy file doesn't have any matching rules, PR/MPR query +won't match any rule, and OpenSM will enforce default QoS level. +Essentially, the above example is equivalent to not having QoS policy file +at all. + +The following example shows all the possible options and keywords in the +policy file and their syntax: + + # + # See the comments in the following example. + # They explain different keywords and their meaning. + # + port-groups + port-group + name: Storage + # "use" is just a description that is used for logging + # Other than that, it is just a comment + use: SRP Targets + port-guid: 0x10000000000001, 0x10000000000005-0x1000000000FFFA + port-guid: 0x1000000000FFFF + end-port-group + + port-group + name: Virtual Servers + # The syntax of the port name is as follows: + # "node_description/Pnum". + # node_description is compared to the NodeDescription of the node, + # and "Pnum" is a port number on that node. + port-name: vs1 HCA-1/P1, vs2 HCA-1/P1 + end-port-group + + # using partitions defined in the partition policy + port-group + name: Partitions + partition: Part1 + pkey: 0x1234 + end-port-group + + # using node types: CA, ROUTER, SWITCH, SELF (for node that runs SM) + # or ALL (for all the nodes in the subnet) + port-group + name: CAs and SM + node-type: CA, SELF + end-port-group + + end-port-groups + + qos-setup + # This section of the policy file describes how to set up SL2VL and VL + # Arbitration tables on various nodes in the fabric. + # However, this is not supported in OFED 1.3 - the section is parsed + # and ignored. SL2VL and VLArb tables should be configured in the + # OpenSM options file (by default - /var/cache/opensm/opensm.opts). + end-qos-setup + + qos-levels + + # Having a QoS Level named "DEFAULT" is a must - it is applied to + # PR/MPR requests that didn't match any of the matching rules. + qos-level + name: DEFAULT + use: default QoS Level + sl: 0 + end-qos-level + + # the whole set: SL, MTU-Limit, Rate-Limit, PKey, Packet Lifetime + qos-level + name: WholeSet + sl: 1 + mtu-limit: 4 + rate-limit: 5 + pkey: 0x1234 + packet-life: 8 + end-qos-level + + end-qos-levels + + # Match rules are scanned in order of their apperance in the policy file. + # First matched rule takes precedence. + qos-match-rules + + # matching by single criteria: QoS class + qos-match-rule + use: by QoS class + qos-class: 7-9,11 + # Name of qos-level to apply to the matching PR/MPR + qos-level-name: WholeSet + end-qos-match-rule + + # show matching by destination group and service id + qos-match-rule + use: Storage targets + destination: Storage + service-id: 0x10000000000001, 0x10000000000008-0x10000000000FFF + qos-level-name: WholeSet + end-qos-match-rule + + qos-match-rule + source: Storage + use: match by source group only + qos-level-name: DEFAULT + end-qos-match-rule + + qos-match-rule + use: match by all parameters + qos-class: 7-9,11 + source: Virtual Servers + destination: Storage + service-id: 0x0000000000010000-0x000000000001FFFF + pkey: 0x0F00-0x0FFF + qos-level-name: WholeSet + end-qos-match-rule + + end-qos-match-rules + +.PP +.B Simplified QoS Policy - Details and Examples +.PP +Simplified QoS policy match rules are tailored for matching ULPs (or +some application on top of a ULP) PR/MPR requests. It has a list of +per-ULP (or per-application) match rules and the SL that should be +enforced on the matched PR/MPR query. + +Match rules include: + - Default match rule that is applied to PR/MPR query that didn't + match any of the other match rules + - SDP + - SDP application with a specific target TCP/IP port range + - SRP with a specific target IB port GUID + - RDS + - iSER + - iSER application with a specific target TCP/IP port range + - IPoIB with a default PKey + - IPoIB with a specific PKey + - any ULP/application with a specific Service ID in the PR/MPR query + - any ULP/application with a specific PKey in the PR/MPR query + - any ULP/application with a specific target IB port GUID in the PR/MPR query + +Since any section of the policy file is optional, as long as basic rules +of the file are kept (such as no referring to nonexisting port group, +having default QoS Level, etc), the simplified policy section (qos-ulps) +can serve as a complete QoS policy file. +The shortest policy file in this case would be as follows: + + qos-ulps + default : 0 #default SL + end-qos-ulps + +It is equivalent to not having policy file at all. + +Below is an example of simplified QoS policy with all the possible keywords: + + qos-ulps + default : 0 # default SL + sdp, port-num 30000 : 0 # SL for application running on top + # of SDP when a destination + # TCP/IPport is 30000 + sdp, port-num 10000-20000 : 0 + sdp : 1 # default SL for any other + # application running on top of SDP + rds : 2 # SL for RDS traffic + iser, port-num 900 : 0 # SL for iSER with a specific target + # port + iser : 3 # default SL for iSER + ipoib, pkey 0x0001 : 0 # SL for IPoIB on partition with + # pkey 0x0001 + ipoib : 4 # default IPoIB partition, + # pkey=0x7FFF + any, service-id 0x6234 : 6 # match any PR/MPR query with a + # specific Service ID + any, pkey 0x0ABC : 6 # match any PR/MPR query with a + # specific PKey + srp, target-port-guid 0x1234 : 5 # SRP when SRP Target is located on + # a specified IB port GUID + any, target-port-guid 0x0ABC-0xFFFFF : 6 # match any PR/MPR query with + # a specific target port GUID + end-qos-ulps + + +Similar to the full policy definition, matching of PR/MPR queries is done in +order of appearance in the QoS policy file such as the first match takes +precedence, except for the "default" rule, which is applied only if the query +didn't match any other rule. + +All other sections of the QoS policy file take precedence over the qos-ulps +section. That is, if a policy file has both qos-match-rules and qos-ulps +sections, then any query is matched first against the rules in the +qos-match-rules section, and only if there was no match, the query is matched +against the rules in qos-ulps section. + +Note that some of these match rules may overlap, so in order to use the +simplified QoS definition effectively, it is important to understand how each +of the ULPs is matched: + +.B IPoIB: +PR query is matched by PKey. Default PKey for IPoIB partition is 0x7fff, so +the following three match rules are equivalent: + + ipoib : + ipoib, pkey 0x7fff : + any, pkey 0x7fff : + +.I Note +: For OFED 1.3, IPoIB partition SL configuration should be done through +partition configuration file only. + +\fBSDP\fP: PR query is matched by Service ID. The Service-ID for SDP is +0x000000000001PPPP, where PPPP are 4 hex digits holding the remote TCP/IP +Port Number to connect to. The following two match rules are equivalent: + + sdp : + any, service-id 0x0000000000010000-0x000000000001ffff : + +\fBRDS\fP: Similar to SDP, RDS PR query is matched by Service ID. The +Service ID for RDS is 0x000000000106PPPP, where PPPP are 4 hex digits +holding the remote TCP/IP Port Number to connect to. Default port number +for RDS is 0x48CA, which makes a default Service-ID 0x00000000010648CA. +The following two match rules are equivalent: + + rds : + any, service-id 0x00000000010648CA : + +\fBiSER\fP: Similar to RDS, iSER query is matched by Service ID, where the +Service ID is also 0x000000000106PPPP. Default port number for iSER is 0x035C, +which makes a default Service-ID 0x000000000106035C. +The following two match rules are equivalent: + + iser : + any, service-id 0x000000000106035C : + +\fBSRP\fP: Service ID for SRP varies from storage vendor to vendor, thus SRP query is +matched by the target IB port GUID. The following two match rules are +equivalent: + + srp, target-port-guid 0x1234 : + any, target-port-guid 0x1234 : + +Note that any of the above ULPs might contain target port GUID in the PR +query, so in order for these queries not to be recognized by the QoS manager +as SRP, the SRP match rule (or any match rule that refers to the target port +guid only) should be placed at the end of the qos-ulps match rules. + +\fBMPI\fP: SL for MPI is manually configured by MPI admin. OpenSM is not +forcing any SL on the MPI traffic, and that's why it is the only ULP that +did not appear in the qos-ulps section. + + +.SS SL2VL Mapping and VL Arbitration +.PP + +OpenSM cached options file has a set of QoS related configuration +parameters, that are used to configure SL2VL mapping and VL arbitration +on IB ports. These parameters are: + - Max VLs: the maximum number of VLs that will be on the subnet. + - High limit: the limit of High Priority component of VL Arbitration + table (IBA 7.6.9). + - VLArb low table: Low priority VL Arbitration table (IBA 7.6.9) template. + - VLArb high table: High priority VL Arbitration table (IBA 7.6.9) template. + - SL2VL: SL2VL Mapping table (IBA 7.6.6) template. It is a list of VLs + corresponding to SLs 0-15 (Note that VL15 used here means drop this SL). + +There are separate QoS configuration parameters sets for various target +types: CAs, routers, switch external ports, and switch's enhanced port 0. +The names of such parameters are prefixed by "qos__" string. +Here is a full list of the currently supported sets: + + qos_ca_ - QoS configuration parameters set for CAs. + qos_rtr_ - parameters set for routers. + qos_sw0_ - parameters set for switches' port 0. + qos_swe_ - parameters set for switches' external ports. + +Here's the example of typical default values for all the ports in the +subnet (hard-coded in OpenSM initialization): + + qos_max_vls=15 + qos_high_limit=0 + qos_vlarb_high=0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0 + qos_vlarb_low=0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4 + qos_sl2vl=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 + + +VL arbitration tables (both high and low) are lists of VL/Weight pairs. +Each list entry contains a VL number (values from 0-14), and a weighting value +(values 0-255), indicating the number of 64 byte units (credits) which may be +transmitted from that VL when its turn in the arbitration occurs. A weight +of 0 indicates that this entry should be skipped. If a list entry is +programmed for VL15 or for a VL that is not supported or is not currently +configured by the port, the port may either skip that entry or send from any +supported VL for that entry. + +Note, that the same VLs may be listed multiple times in the High or Low +priority arbitration tables, and, further, it can be listed in both tables. + +The limit of high-priority VLArb table (qos__high_limit) indicates the +number of high-priority packets that can be transmitted without an opportunity +to send a low-priority packet. Specifically, the number of bytes that can be +sent is high_limit times 4K bytes. + +A high_limit value of 255 indicates that the byte limit is unbounded. +Note: if the 255 value is used, the low priority VLs may be starved. +A value of 0 indicates that only a single packet from the high-priority table +may be sent before an opportunity is given to the low-priority table. + +Keep in mind that ports usually transmit packets of size equal to MTU. +For instance, for 4KB MTU a single packet will require 64 credits, so in order +to achieve effective VL arbitration for packets of 4KB MTU, the weighting +values for each VL should be multiples of 64. + +Below is an example of SL2VL and VL Arbitration configuration on subnet: + + qos_max_vls=15 + qos_high_limit=6 + qos_vlarb_high=0:4 + qos_vlarb_low=0:0,1:64,2:128,3:192,4:0,5:64,6:64,7:64 + qos_sl2vl=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 + +In this example, there are 8 VLs configured on subnet: VL0 to VL7. VL0 is +defined as a high priority VL, and it is limited to 6 x 4KB = 24KB in a single +transmission burst. Such configuration would suilt VL that needs low latency +and uses small MTU when transmitting packets. Rest of VLs are defined as low +priority VLs with different weights, while VL4 is effectively turned off. .SH PREFIX ROUTES .PP -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Tue Mar 25 17:54:25 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 26 Mar 2008 02:54:25 +0200 Subject: [ofa-general] Re: [PATCH] opensm/QoS: setting SL in the IPoIB MCast groups In-Reply-To: <20080326001244.GK6773@sashak.voltaire.com> References: <47E7BF68.7050104@dev.mellanox.co.il> <20080324162930.GJ595@sashak.voltaire.com> <47E89564.1040608@dev.mellanox.co.il> <20080326001244.GK6773@sashak.voltaire.com> Message-ID: <47E99EC1.7000507@dev.mellanox.co.il> Sasha Khapyorsky wrote: > On 08:02 Tue 25 Mar , Yevgeny Kliteynik wrote: >>> Why to not use 'osm_mgrp_t *mgr' instead of mlid and to not access mcast >>> group directly w/out qmap search? >> No special reason. Didn't want to think what would happen if the mcast >> group was removed for some reason and the partition will still have the >> pointer. > > Now it is impossible because partition creates only "well known" mc > groups. True > Although this could change some days. Exactly my point. > I'm fine with your approach. Cool :) > > Sasha > From a-akreut at acloche.com Tue Mar 25 18:01:03 2008 From: a-akreut at acloche.com (Sanford Ferreira) Date: Wed, 26 Mar 2008 09:01:03 +0800 Subject: [ofa-general] did you miss me Message-ID: <01c88f1f$eb083180$2195597a@a-akreut> Hello! I am tired today. I am nice girl that would like to chat with you. Email me at Madeleine at BestGolova.com only, because I am using my friend's email to write this. Will send some of my pictures From changquing.tang at hp.com Tue Mar 25 20:56:11 2008 From: changquing.tang at hp.com (Tang, Changqing) Date: Wed, 26 Mar 2008 03:56:11 +0000 Subject: [ofa-general] error with ibv_poll_cq() call Message-ID: Hi, We are debuging our dynamic process code, when we call ret = ibv_poll_cq(cq_hndl, 1, &compl); The peer process may have destroyed the QP. However, ibv_poll_cq() return -2 in 'ret', 'errno' is still 0 What could be the reason for this error ? There is a posted send pending for completion, so error should be reported via the completion status, not the polling function itself. Thanks for any help. This is OFED 1.3 --CQ Tang, HP-MPI From shahrivar.danby at activatormail.com Tue Mar 25 22:07:42 2008 From: shahrivar.danby at activatormail.com (Ginger Ritter) Date: Tue, 25 Mar 2008 21:07:42 -0800 Subject: [ofa-general] Good luck, dear friend Message-ID: <111646186.12840876820300@activatormail.com> -------------- next part -------------- A non-text attachment was scrubbed... Name: it.gif Type: image/gif Size: 5979 bytes Desc: not available URL: From a-amyhu at aboutpaducah.com Tue Mar 25 23:06:06 2008 From: a-amyhu at aboutpaducah.com (Morgan Greenwood) Date: Wed, 26 Mar 2008 14:06:06 +0800 Subject: [ofa-general] hope you remember me Message-ID: <01c88f4a$88785b00$9642b63a@a-amyhu> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Maria at BestGolova.com only, because I am using my friend's email to write this. Don't miss some of my naughty pictures. From jackm at dev.mellanox.co.il Wed Mar 26 00:01:25 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 09:01:25 +0200 Subject: [ofa-general] error with ibv_poll_cq() call In-Reply-To: References: Message-ID: <200803260901.25918.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 05:56, Tang, Changqing wrote: > > Hi, > We are debuging our dynamic process code, when we call > > ret = ibv_poll_cq(cq_hndl, 1, &compl); > > The peer process may have destroyed the QP. > > However, ibv_poll_cq() return -2 in 'ret', 'errno' is still 0 > > What could be the reason for this error ? > > There is a posted send pending for completion, so error should be > reported via the completion status, not the polling function > itself. > > Thanks for any help. This is OFED 1.3 Roland, It looks like we have a race condition in mlx4_destroy_qp. We clean the cq BEFORE modifying the QP to reset (done in kernel as part of the ibv_cmd_destroy_qp() flow). CQ's problem has exposed this bug. mlx4_cq_clean needs to be invoked **after** the destroy: Index: libmlx4/src/verbs.c =================================================================== --- libmlx4.orig/src/verbs.c 2008-03-26 09:00:08.000000000 +0200 +++ libmlx4/src/verbs.c 2008-03-26 09:00:52.449586000 +0200 @@ -558,11 +558,6 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) struct mlx4_qp *qp = to_mqp(ibqp); int ret; - mlx4_cq_clean(to_mcq(ibqp->recv_cq), ibqp->qp_num, - ibqp->srq ? to_msrq(ibqp->srq) : NULL); - if (ibqp->send_cq != ibqp->recv_cq) - mlx4_cq_clean(to_mcq(ibqp->send_cq), ibqp->qp_num, NULL); - mlx4_lock_cqs(ibqp); mlx4_clear_qp(to_mctx(ibqp->context), ibqp->qp_num); mlx4_unlock_cqs(ibqp); @@ -576,6 +571,11 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) return ret; } + mlx4_cq_clean(to_mcq(ibqp->recv_cq), ibqp->qp_num, + ibqp->srq ? to_msrq(ibqp->srq) : NULL); + if (ibqp->send_cq != ibqp->recv_cq) + mlx4_cq_clean(to_mcq(ibqp->send_cq), ibqp->qp_num, NULL); + if (!ibqp->srq && ibqp->qp_type != IBV_QPT_XRC) mlx4_free_db(to_mctx(ibqp->context), MLX4_DB_TYPE_RQ, qp->db); free(qp->sq.wrid); From changquing.tang at hp.com Wed Mar 26 00:09:25 2008 From: changquing.tang at hp.com (Tang, Changqing) Date: Wed, 26 Mar 2008 07:09:25 +0000 Subject: [ofa-general] error with ibv_poll_cq() call In-Reply-To: <200803260901.25918.jackm@dev.mellanox.co.il> References: <200803260901.25918.jackm@dev.mellanox.co.il> Message-ID: Well, I said "PEER" process may destroyed the remote QP, The process calling ibv_poll_cq() still has the QP in RTS state. And though I use OFED 1.3, the HCA is not connectX. idea ? --CQ > -----Original Message----- > From: Jack Morgenstein [mailto:jackm at dev.mellanox.co.il] > Sent: Wednesday, March 26, 2008 2:01 AM > To: general at lists.openfabrics.org > Cc: Tang, Changqing; Roland Dreier > Subject: Re: [ofa-general] error with ibv_poll_cq() call > > On Wednesday 26 March 2008 05:56, Tang, Changqing wrote: > > > > Hi, > > We are debuging our dynamic process code, when we call > > > > ret = ibv_poll_cq(cq_hndl, 1, &compl); > > > > The peer process may have destroyed the QP. > > > > However, ibv_poll_cq() return -2 in 'ret', 'errno' is still 0 > > > > What could be the reason for this error ? > > > > There is a posted send pending for completion, so error should be > > reported via the completion status, not the polling function itself. > > > > Thanks for any help. This is OFED 1.3 > > Roland, > It looks like we have a race condition in mlx4_destroy_qp. > We clean the cq BEFORE modifying the QP to reset (done in > kernel as part of the ibv_cmd_destroy_qp() flow). > > CQ's problem has exposed this bug. mlx4_cq_clean needs to be invoked > **after** the destroy: > > Index: libmlx4/src/verbs.c > =================================================================== > --- libmlx4.orig/src/verbs.c 2008-03-26 09:00:08.000000000 +0200 > +++ libmlx4/src/verbs.c 2008-03-26 09:00:52.449586000 +0200 > @@ -558,11 +558,6 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) > struct mlx4_qp *qp = to_mqp(ibqp); > int ret; > > - mlx4_cq_clean(to_mcq(ibqp->recv_cq), ibqp->qp_num, > - ibqp->srq ? to_msrq(ibqp->srq) : NULL); > - if (ibqp->send_cq != ibqp->recv_cq) > - mlx4_cq_clean(to_mcq(ibqp->send_cq), > ibqp->qp_num, NULL); > - > mlx4_lock_cqs(ibqp); > mlx4_clear_qp(to_mctx(ibqp->context), ibqp->qp_num); > mlx4_unlock_cqs(ibqp); > @@ -576,6 +571,11 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) > return ret; > } > > + mlx4_cq_clean(to_mcq(ibqp->recv_cq), ibqp->qp_num, > + ibqp->srq ? to_msrq(ibqp->srq) : NULL); > + if (ibqp->send_cq != ibqp->recv_cq) > + mlx4_cq_clean(to_mcq(ibqp->send_cq), ibqp->qp_num, > + NULL); > + > if (!ibqp->srq && ibqp->qp_type != IBV_QPT_XRC) > mlx4_free_db(to_mctx(ibqp->context), > MLX4_DB_TYPE_RQ, qp->db); > free(qp->sq.wrid); > > > > From jackm at dev.mellanox.co.il Wed Mar 26 00:30:12 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 09:30:12 +0200 Subject: [ofa-general] error with ibv_poll_cq() call In-Reply-To: References: <200803260901.25918.jackm@dev.mellanox.co.il> Message-ID: <200803260930.12860.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 09:09, Tang, Changqing wrote: > Well, I said "PEER" process may destroyed the remote QP, The process > calling ibv_poll_cq() still has the QP in RTS state. > > And though I use OFED 1.3, the HCA is not connectX. > > idea ? > > 1. What HCA are you using? 2. Are there other QPs using the CQ? Is the CQ used for sends only? How are you using the CQ? - Jack From changquing.tang at hp.com Wed Mar 26 00:41:32 2008 From: changquing.tang at hp.com (Tang, Changqing) Date: Wed, 26 Mar 2008 07:41:32 +0000 Subject: [ofa-general] error with ibv_poll_cq() call In-Reply-To: <200803260930.12860.jackm@dev.mellanox.co.il> References: <200803260901.25918.jackm@dev.mellanox.co.il> <200803260930.12860.jackm@dev.mellanox.co.il> Message-ID: > -----Original Message----- > From: Jack Morgenstein [mailto:jackm at dev.mellanox.co.il] > Sent: Wednesday, March 26, 2008 2:30 AM > To: Tang, Changqing > Cc: general at lists.openfabrics.org; Roland Dreier > Subject: Re: [ofa-general] error with ibv_poll_cq() call > > On Wednesday 26 March 2008 09:09, Tang, Changqing wrote: > > Well, I said "PEER" process may destroyed the remote QP, > The process > > calling ibv_poll_cq() still has the QP in RTS state. > > > > And though I use OFED 1.3, the HCA is not connectX. > > > > idea ? > > > > > 1. What HCA are you using? hca_id: mthca0 fw_ver: 1.1.0 node_guid: 0019:bbff:fff8:c048 sys_image_guid: 0019:bbff:fff8:c04b vendor_id: 0x02c9 vendor_part_id: 25204 hw_ver: 0xA0 board_id: HP_0010000001 phys_port_cnt: 1 > 2. Are there other QPs using the CQ? yes, there are connections to other processes. The connection to a particular peer process is to be disconnected. the peer process may have passed the handshake and destroy the QP Is the CQ used for sends only? Yes, it is for sends only, we use pure RDMA. > How are you using the CQ? We use selective signal for sends completion, but the last one is signaled. --CQ > > - Jack > > From jackm at dev.mellanox.co.il Wed Mar 26 00:54:39 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 09:54:39 +0200 Subject: [ofa-general] error with ibv_poll_cq() call In-Reply-To: References: <200803260930.12860.jackm@dev.mellanox.co.il> Message-ID: <200803260954.39640.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 09:41, Tang, Changqing wrote: > > > -----Original Message----- > > From: Jack Morgenstein [mailto:jackm at dev.mellanox.co.il] > > Sent: Wednesday, March 26, 2008 2:30 AM > > To: Tang, Changqing > > Cc: general at lists.openfabrics.org; Roland Dreier > > Subject: Re: [ofa-general] error with ibv_poll_cq() call > > > > On Wednesday 26 March 2008 09:09, Tang, Changqing wrote: > > > Well, I said "PEER" process may destroyed the remote QP, > > The process > > > calling ibv_poll_cq() still has the QP in RTS state. > > > > > > And though I use OFED 1.3, the HCA is not connectX. > > > > > > idea ? > > > 1. You are getting a "non-existent QP" error for the poll: The ONLY way this can happen on the send is if you have destroyed some QP which is using that CQ for send completions, and the QP generated a SEND completion during the destroy (-- after cleaning the cq from completions for that QP, but before calling ibv_cmd_destroy_qp() to move the QP to the error state, allowing a new, "uncleaned" CQE to be queued). 2. If you always modify the QP to ERR before destroying it, the above race condition (noted in my previous post) does not apply. ** (ROLAND, libmthca has the same race condition) **. - Jack > > 1. What HCA are you using? > > hca_id: mthca0 > fw_ver: 1.1.0 > node_guid: 0019:bbff:fff8:c048 > sys_image_guid: 0019:bbff:fff8:c04b > vendor_id: 0x02c9 > vendor_part_id: 25204 > hw_ver: 0xA0 > board_id: HP_0010000001 > phys_port_cnt: 1 > > > 2. Are there other QPs using the CQ? > > yes, there are connections to other processes. The connection to a particular peer process is to be disconnected. > the peer process may have passed the handshake and destroy the QP > > Is the CQ used for sends only? > > Yes, it is for sends only, we use pure RDMA. > > > > How are you using the CQ? > > We use selective signal for sends completion, but the last one is signaled. > > > --CQ > > > > > > > - Jack > > > > > From jackm at dev.mellanox.co.il Wed Mar 26 01:12:57 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 10:12:57 +0200 Subject: [ofa-general] error with ibv_poll_cq() call In-Reply-To: References: <200803260930.12860.jackm@dev.mellanox.co.il> Message-ID: <200803261012.57749.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 09:41, Tang, Changqing wrote: > > 1. What HCA are you using? > > hca_id: mthca0 >         fw_ver:                         1.1.0 >         node_guid:                      0019:bbff:fff8:c048 >         sys_image_guid:                 0019:bbff:fff8:c04b >         vendor_id:                      0x02c9 >         vendor_part_id:                 25204 >         hw_ver:                         0xA0 >         board_id:                       HP_0010000001 >         phys_port_cnt:                  1 > The FW revision you are using is old. You should be using FW 1.2.000 (See OFED 1.3 release notes, search for 25204). - Jack From financialdream at operamail.com Wed Mar 26 01:38:07 2008 From: financialdream at operamail.com (financialdream at operamail.com) Date: Wed, 26 Mar 2008 02:38:07 -0600 Subject: [ofa-general] No Recruiting! Fully Automated! Never pay more than $10 per month..and EARN a thousand fold Message-ID: <200803260838.m2Q8c76w020339@unstuck.com> Greetings Sheena Here! You can now earn up to $3,904.00 per MONTH or more!* how about double it? 5,856.00 per MONTH or more!* & so on... All this for just $10.75! Yes!...Yes!..Yes!.. with the This unique system "Ladder Step Earning System with your $10.75 become $1000,000,_ _ _ or add zero if you like"! That's correct, starting with just $10.75, you can easily build an enormous monthly income by gradually climbing the Ladder with your downline. No more is your income limited by small 5-level deep commission payments. No more do you have to promote that secondary "high dollar" program to make the big bucks. Some people say that a 10 dollar program cannot generate a reasonable income in a reasonable amount of time. Not only has this program proven them wrong in the past, the program has just created the ultimate earning opportunity. That now brings you the best of both worlds.. Start small, build fast, climb the ladder, earn BIG! There's NOTHING stopping you from earning a massive monthly income when you see how our new system will work for YOU... NOW! what is $10,000 for just 10$.. You decide and no more thinkable actions! Get in Here Fast for FREE to sign up :) http://ItsJust10.80t.com :) http://ItsJust10.80t.com :) http://ItsJust10.80t.com Never left behind because your not the last.. See more about how to make 10$ to a thousand fold.... http://payplan.v33.org/ Thank you for taking time to read this... TO contact me please email me here secret0ni at gmail.com YM: ren0_x 09185271240 P.S.Once you signup for this program and log in, you'll be presented with your upgrade options and the BONUSES and products associated with each one. I would STRONGLY recommend that you start out at the program PLUS level or higher to maximize your starting income. However, the choice is up to you.. If you wish not to participate please OPT-OUT HERE: http://unsubscribehere.zeroweb.org/ for no problem in the future... Keith Yamashita and Sandra Spataro have written a new book called UNSTUCK. It serves as an in-the-moment tool for the times when you want to go forward, but everything seems to be moving in reverse. You can read more at http://www.unstuck.com. ---------- A friend has sent you this email. If you believe that you got this e-mail in error, please forward this entire e-mail to . (For your information, the IP address of the person who sent you this email is: ) From ShannaconstantinopleCorley at americanheart.org Tue Mar 25 11:35:28 2008 From: ShannaconstantinopleCorley at americanheart.org (Rosario Hagen) Date: Wed, 26 Mar 2008 02:35:28 +0800 Subject: [ofa-general] Market investor alert Message-ID: <5IX698EJXVWDA391@americanheart.org> Medical suuplies is one of the hottest sectors currently And the timing is very opportune. With cutting edge technology and state of the ART sortwares to do both improve complex patient care and improve profits for medical practitioners, we believe we have a winner in ZYTO CORP Symbol : ZYTC. after today's selloff we believe the stock has found strong support at present range Additionally look for another news release for huge contracts. This is not a fly by night. Real company and real products used by real health care professionals It's due and ripe for a breakout. Ride the easy 10 bagger Sym: ZYTC From ogerlitz at voltaire.com Wed Mar 26 04:02:59 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 26 Mar 2008 13:02:59 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: References: <1205767427.25950.137.camel@mtls03> Message-ID: <47EA2D63.7040501@voltaire.com> Roland Dreier wrote: > > @@ -505,6 +509,7 @@ struct ib_qp_init_attr { > > enum ib_sig_type sq_sig_type; > > enum ib_qp_type qp_type; > > u8 port_num; /* special QP types only */ > > + enum qp_create_flags create_flags; > > }; > > I'm dubious about this. It seems to me like everything (including the > mlx4 low-level driver changes for LSO) would be simpler to implement > and use if we just extend the qp_type to include IB_QPT_UD_LSO. Roland, All, How about making the qp_type field a bit mask, such that in the ipoib LSO use case it would be (UD | LSO) and in the ipoib ehca case (UD | LL), etc. The bit mask change would also be propagated up to libibverbs and be defined in a way such that it preserves backward compatibility re the qp_type field for users of libibverbs that did not changed their code. I don't think it would make the XRC merge harder, and it would be very helpful in deploying the "block loopback" feature of the connectx, which in ofed 1.3 was implemented system wide (set or unset, see the patch below) and now can be set per app per qp, so IPoIB would create its qp (UD | BL[=block-loopback] | LSO) when running over connectx. > mlx4: enable discarding/passing multicast loopback packets by FW/HW. > > Multicast (and broadcast) loopback is handled by the network stack > meaning that > all MC or BC packets that need to return into receive sockets on the > machine are > duplicated and put in the rx path handling by the ip network stack. > The HCA loops all multicast outgoing packets so that any attached QP > can get > these multicast packets as well. > > The IPoIB module needs to discard all those self QP loopback packets > and does > so by comparing the SLID and QPN. > > This patch controls the ConnectX HCA multicast packets block loopback > (blck_lb) for self QP. > > The patch is designed to enable or disable blocking of all multicast > packets on self QP > in FW/HW on all QPs created on the ConnectX HCA. > > Inter QP multicast packets on the relevant HCA will still be delivered. > > The /sys/module/mlx4_core/block_loopback attribute controls the policy > flag. > Its default value is blocking-enabled (non-zero). > The flag can be read and set/unset through sysfs. > > Signed-off-by: Alex Rosenbaum > Signed-off-by: Merav Havuv > Signed-off-by: Jack Morgenstein > > Index: ofed_kernel/drivers/net/mlx4/mcg.c > =================================================================== > --- ofed_kernel.orig/drivers/net/mlx4/mcg.c 2007-12-05 > 10:34:53.519969000 +0200 > +++ ofed_kernel/drivers/net/mlx4/mcg.c 2008-02-19 > 08:45:33.257352000 +0200 > @@ -206,13 +206,14 @@ int mlx4_multicast_attach(struct mlx4_de > } > > for (i = 0; i < members_count; ++i) > - if (mgm->qp[i] == cpu_to_be32(qp->qpn)) { > + if ((be32_to_cpu(mgm->qp[i]) & MGM_QPN_MASK) == qp->qpn) { > mlx4_dbg(dev, "QP %06x already a member of MGM\n", qp->qpn); > err = 0; > goto out; > } > > - mgm->qp[members_count++] = cpu_to_be32(qp->qpn); > + mgm->qp[members_count++] = cpu_to_be32((qp->qpn & MGM_QPN_MASK) | > + (!!mlx4_blck_lb << MGM_BLCK_LB_BIT)); > mgm->members_count = cpu_to_be32(members_count); > > err = mlx4_WRITE_MCG(dev, index, mailbox); > @@ -287,7 +288,7 @@ int mlx4_multicast_detach(struct mlx4_de > > members_count = be32_to_cpu(mgm->members_count); > for (loc = -1, i = 0; i < members_count; ++i) > - if (mgm->qp[i] == cpu_to_be32(qp->qpn)) > + if ((be32_to_cpu(mgm->qp[i]) & MGM_QPN_MASK) == qp->qpn) > loc = i; > > if (loc == -1) { > Index: ofed_kernel/drivers/net/mlx4/main.c > =================================================================== > --- ofed_kernel.orig/drivers/net/mlx4/main.c 2008-02-19 > 08:38:33.145870000 +0200 > +++ ofed_kernel/drivers/net/mlx4/main.c 2008-02-19 > 08:42:17.836566000 +0200 > @@ -59,6 +59,10 @@ MODULE_PARM_DESC(debug_level, "Enable de > > #endif /* CONFIG_MLX4_DEBUG */ > > +int mlx4_blck_lb=1; > +module_param_named(block_loopback, mlx4_blck_lb, int, 0644); > +MODULE_PARM_DESC(block_loopback, "Block multicast loopback packets if > > 0"); > + > #ifdef CONFIG_PCI_MSI > > static int msi_x = 1; > Index: ofed_kernel/drivers/net/mlx4/mlx4.h > =================================================================== > --- ofed_kernel.orig/drivers/net/mlx4/mlx4.h 2008-02-19 > 08:38:31.356932000 +0200 > +++ ofed_kernel/drivers/net/mlx4/mlx4.h 2008-02-19 > 08:42:17.840568000 +0200 > @@ -106,6 +106,10 @@ extern int mlx4_debug_level; > #define mlx4_warn(mdev, format, arg...) \ > dev_warn(&mdev->pdev->dev, format, ## arg) > > +#define MGM_QPN_MASK 0x00FFFFFF > +#define MGM_BLCK_LB_BIT 30 > +extern int mlx4_blck_lb; > + > struct mlx4_bitmap { > u32 last; > u32 top; From ldp at borrmanconsulting.com Wed Mar 26 04:32:07 2008 From: ldp at borrmanconsulting.com (Aurora Rivers) Date: Wed, 26 Mar 2008 19:32:07 +0800 Subject: [ofa-general] No weight - no problems Message-ID: <01c88f78$13bc1580$33289a7b@ldp> Never late to try. Watch details attached and know more. -------------- next part -------------- A non-text attachment was scrubbed... Name: mp5.zip Type: application/zip Size: 579 bytes Desc: not available URL: From Vivian661 at mayridge.com Wed Mar 26 04:40:24 2008 From: Vivian661 at mayridge.com (=?koi8-r?B?IuHNzMXSIOIu/i4i?=) Date: Wed, 26 Mar 2008 03:40:24 -0800 Subject: [ofa-general] =?koi8-r?b?8M/MzsHRIMvPzMzFy8PJ0SDTz9fF1NPLycggzdXM?= =?koi8-r?b?2NTGyczYzc/X?= Message-ID: <01c88ef3$1fb1c400$d623614e@Vivian661> Сделайте себе подарок! Доставка курьером в Москве и почтой по России. Коллекция из 500 советских мультфильмов в формате DivX-mpeg4 на 17 DVD-дисках 1940-2000 гг. Полный каталог мультиков, вошедших в коллекцию, пожалуйста, смотрите на странице http://gu3pix.hotmail.ru Все эти мультфильмы старые добрые и поучительные. Они благотворно влияют на детей и взрослых! Теперь запись дисков в еще лучшем цифровом качестве! Вся озвучка только оригинальная! Так же в коллекцию вошли: Более 1500 дестких песенок в формате mp3 - 1 диск DVD (1 сборник). Авторы: Большой Детских Хор, Андрей Усачев, Г. Гладков, М. Яснов. Более 100 детских сказок в формате mp3 - 1 диск DVD (1 сборник). *** Стоимость всей коллекции мультфильмов, сказок, песенок на 19 DVD дисках - 3500 рублей, продаются только вместе. Доставка включена в стоимость. Сделать заказ и просмотреть перечень всех мультфильмов, вошедших в коллекцию можно на странице: http://gu3pix.hotmail.ru -------------- next part -------------- An HTML attachment was scrubbed... URL: From jackm at dev.mellanox.co.il Wed Mar 26 04:38:00 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 13:38:00 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <47EA2D63.7040501@voltaire.com> References: <1205767427.25950.137.camel@mtls03> <47EA2D63.7040501@voltaire.com> Message-ID: <200803261338.01131.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 13:02, Or Gerlitz wrote: > How about making the qp_type field a bit mask, such that in the ipoib > LSO use case it would be (UD | LSO) and in the ipoib ehca case (UD | > LL), etc. The bit mask change would also be propagated up to libibverbs > and be defined in a way such that it preserves backward compatibility re > the qp_type field for users of libibverbs that did not changed their code. > You can't have your cake and eat it too. You can't both use a bitmap and preserve backwards compatibility. The current QP types are: enum ib_qp_type { /* * IB_QPT_SMI and IB_QPT_GSI have to be the first two entries * here (and in that order) since the MAD layer uses them as * indices into a 2-entry table. */ IB_QPT_SMI, IB_QPT_GSI, IB_QPT_RC, IB_QPT_UC, IB_QPT_UD, IB_QPT_XRC, IB_QPT_RAW_IPV6, IB_QPT_RAW_ETY }; This means that we are using the 3 LSBs in an enumerated fashion (integers 0..7), so we can't use a bitmap here. This leads us to a mixed field anyway -- 3 bits for existing QP types (integers 0..6), and 29 bits of bit-map. Why not just keep things separate, and use the enumeration as is for the existing types, and keep the bitmap as a separate "flags" field -- this is much cleaner. - Jack From eli at mellanox.co.il Wed Mar 26 04:48:38 2008 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 26 Mar 2008 13:48:38 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <47EA2D63.7040501@voltaire.com> References: <1205767427.25950.137.camel@mtls03> <47EA2D63.7040501@voltaire.com> Message-ID: <1206532118.25950.408.camel@mtls03> I don't think it is a good idea to mix enumerated types with bitmasks since bitmasks are wasting too much of the size of the enum (which depends on the CPU architecture). I like creation flags more also because it allows more freedom in choosing the semantics of the flag and not worrying if it fits in the category as a kind of "qp type". On Wed, 2008-03-26 at 13:02 +0200, Or Gerlitz wrote: > Roland Dreier wrote: > > > @@ -505,6 +509,7 @@ struct ib_qp_init_attr { > > > enum ib_sig_type sq_sig_type; > > > enum ib_qp_type qp_type; > > > u8 port_num; /* special QP types only */ > > > + enum qp_create_flags create_flags; > > > }; > > > > I'm dubious about this. It seems to me like everything (including the > > mlx4 low-level driver changes for LSO) would be simpler to implement > > and use if we just extend the qp_type to include IB_QPT_UD_LSO. > Roland, All, > > How about making the qp_type field a bit mask, such that in the ipoib > LSO use case it would be (UD | LSO) and in the ipoib ehca case (UD | > LL), etc. The bit mask change would also be propagated up to libibverbs > and be defined in a way such that it preserves backward compatibility re > the qp_type field for users of libibverbs that did not changed their code. > > I don't think it would make the XRC merge harder, and it would be very > helpful in deploying the "block loopback" feature of the connectx, which > in ofed 1.3 was implemented system wide (set or unset, see the patch > below) and now can be set per app per qp, so IPoIB would create its qp > (UD | BL[=block-loopback] | LSO) when running over connectx. > > > mlx4: enable discarding/passing multicast loopback packets by FW/HW. > > > > Multicast (and broadcast) loopback is handled by the network stack > > meaning that > > all MC or BC packets that need to return into receive sockets on the > > machine are > > duplicated and put in the rx path handling by the ip network stack. > > The HCA loops all multicast outgoing packets so that any attached QP > > can get > > these multicast packets as well. > > > > The IPoIB module needs to discard all those self QP loopback packets > > and does > > so by comparing the SLID and QPN. > > > > This patch controls the ConnectX HCA multicast packets block loopback > > (blck_lb) for self QP. > > > > The patch is designed to enable or disable blocking of all multicast > > packets on self QP > > in FW/HW on all QPs created on the ConnectX HCA. > > > > Inter QP multicast packets on the relevant HCA will still be delivered. > > > > The /sys/module/mlx4_core/block_loopback attribute controls the policy > > flag. > > Its default value is blocking-enabled (non-zero). > > The flag can be read and set/unset through sysfs. > > > > Signed-off-by: Alex Rosenbaum > > Signed-off-by: Merav Havuv > > Signed-off-by: Jack Morgenstein > > > > Index: ofed_kernel/drivers/net/mlx4/mcg.c > > =================================================================== > > --- ofed_kernel.orig/drivers/net/mlx4/mcg.c 2007-12-05 > > 10:34:53.519969000 +0200 > > +++ ofed_kernel/drivers/net/mlx4/mcg.c 2008-02-19 > > 08:45:33.257352000 +0200 > > @@ -206,13 +206,14 @@ int mlx4_multicast_attach(struct mlx4_de > > } > > > > for (i = 0; i < members_count; ++i) > > - if (mgm->qp[i] == cpu_to_be32(qp->qpn)) { > > + if ((be32_to_cpu(mgm->qp[i]) & MGM_QPN_MASK) == qp->qpn) { > > mlx4_dbg(dev, "QP %06x already a member of MGM\n", qp->qpn); > > err = 0; > > goto out; > > } > > > > - mgm->qp[members_count++] = cpu_to_be32(qp->qpn); > > + mgm->qp[members_count++] = cpu_to_be32((qp->qpn & MGM_QPN_MASK) | > > + (!!mlx4_blck_lb << MGM_BLCK_LB_BIT)); > > mgm->members_count = cpu_to_be32(members_count); > > > > err = mlx4_WRITE_MCG(dev, index, mailbox); > > @@ -287,7 +288,7 @@ int mlx4_multicast_detach(struct mlx4_de > > > > members_count = be32_to_cpu(mgm->members_count); > > for (loc = -1, i = 0; i < members_count; ++i) > > - if (mgm->qp[i] == cpu_to_be32(qp->qpn)) > > + if ((be32_to_cpu(mgm->qp[i]) & MGM_QPN_MASK) == qp->qpn) > > loc = i; > > > > if (loc == -1) { > > Index: ofed_kernel/drivers/net/mlx4/main.c > > =================================================================== > > --- ofed_kernel.orig/drivers/net/mlx4/main.c 2008-02-19 > > 08:38:33.145870000 +0200 > > +++ ofed_kernel/drivers/net/mlx4/main.c 2008-02-19 > > 08:42:17.836566000 +0200 > > @@ -59,6 +59,10 @@ MODULE_PARM_DESC(debug_level, "Enable de > > > > #endif /* CONFIG_MLX4_DEBUG */ > > > > +int mlx4_blck_lb=1; > > +module_param_named(block_loopback, mlx4_blck_lb, int, 0644); > > +MODULE_PARM_DESC(block_loopback, "Block multicast loopback packets if > > > 0"); > > + > > #ifdef CONFIG_PCI_MSI > > > > static int msi_x = 1; > > Index: ofed_kernel/drivers/net/mlx4/mlx4.h > > =================================================================== > > --- ofed_kernel.orig/drivers/net/mlx4/mlx4.h 2008-02-19 > > 08:38:31.356932000 +0200 > > +++ ofed_kernel/drivers/net/mlx4/mlx4.h 2008-02-19 > > 08:42:17.840568000 +0200 > > @@ -106,6 +106,10 @@ extern int mlx4_debug_level; > > #define mlx4_warn(mdev, format, arg...) \ > > dev_warn(&mdev->pdev->dev, format, ## arg) > > > > +#define MGM_QPN_MASK 0x00FFFFFF > > +#define MGM_BLCK_LB_BIT 30 > > +extern int mlx4_blck_lb; > > + > > struct mlx4_bitmap { > > u32 last; > > u32 top; > > > > > > > From jypjapamalaresortssyv at japamalaresorts.com Wed Mar 26 04:49:42 2008 From: jypjapamalaresortssyv at japamalaresorts.com (Anita English) Date: Wed, 26 Mar 2008 08:49:42 -0300 Subject: [ofa-general] Software Message-ID: <124440925.97864432106162@japamalaresorts.com> In search for the best costs in discount software? Right now you'll find the opportunity to use the softs you dream of from very long ago. And the best thing is, all softwares are exceedingly cheap. Examine by yourself & get the softwares for lowest rates ever seen! http://renelavineoo804.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From HNGUYEN at de.ibm.com Wed Mar 26 05:03:16 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 26 Mar 2008 13:03:16 +0100 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <47EA2D63.7040501@voltaire.com> Message-ID: Hi Or! > Roland Dreier wrote: > > > @@ -505,6 +509,7 @@ struct ib_qp_init_attr { > > > enum ib_sig_type sq_sig_type; > > > enum ib_qp_type qp_type; > > > u8 port_num; /* special QP types only */ > > > + enum qp_create_flags create_flags; > > > }; > > > > I'm dubious about this. It seems to me like everything (including the > > mlx4 low-level driver changes for LSO) would be simpler to implement > > and use if we just extend the qp_type to include IB_QPT_UD_LSO. > Roland, All, > > How about making the qp_type field a bit mask, such that in the ipoib > LSO use case it would be (UD | LSO) and in the ipoib ehca case (UD | > LL), etc. The bit mask change would also be propagated up to libibverbs > and be defined in a way such that it preserves backward compatibility re > the qp_type field for users of libibverbs that did not changed their code. I personally prefer this because we want to providde LL to user space. Using qp_type's bit fields does not require me to change e.g. ib_uverbs_create_qp to reflect create_flags from user space, which will break previous libxyz - currently it's hardcoded to zero in uverbs_cmd.c, see also http://lists.openfabrics.org/pipermail/general/2008-March/047960.html Nam From ogerlitz at voltaire.com Wed Mar 26 05:16:07 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 26 Mar 2008 14:16:07 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <200803261338.01131.jackm@dev.mellanox.co.il> References: <1205767427.25950.137.camel@mtls03> <47EA2D63.7040501@voltaire.com> <200803261338.01131.jackm@dev.mellanox.co.il> Message-ID: <47EA3E87.9090200@voltaire.com> Jack Morgenstein wrote: > You can't have your cake and eat it too. You can't both use a bitmap > and preserve backwards compatibility. > > The current QP types are: > enum ib_qp_type { > /* > * IB_QPT_SMI and IB_QPT_GSI have to be the first two entries > * here (and in that order) since the MAD layer uses them as > * indices into a 2-entry table. > */ > IB_QPT_SMI, > IB_QPT_GSI, > > IB_QPT_RC, > IB_QPT_UC, > IB_QPT_UD, > IB_QPT_XRC, > IB_QPT_RAW_IPV6, > IB_QPT_RAW_ETY > }; > > This means that we are using the 3 LSBs in an enumerated fashion (integers 0..7), so > we can't use a bitmap here. This leads us to a mixed field anyway -- > 3 bits for existing QP types (integers 0..6), and 29 bits of bit-map. > > Why not just keep things separate, and use the enumeration as is for the existing types, > and keep the bitmap as a separate "flags" field -- this is much cleaner. > Basically, my motivation is to find a way to integrate the block-loopback feature without breaking the libibverbs (kernel/user) and (lib/app) ABI. So in that sense I do hope there's a way to eat the cake in a way which is not notable by libibverbs consumers which are compiled and/or linked in any static/dynamic manner against an older version of the library. I assume that space wise (enum == unsigned int == u32) for the compiler. So with that assumption at hand, the above enum has the values 0...7 occupied, what would be the problem to replace in the kernel .h file, the enum qp_type field with u32 qp_type_bits field where the above types getting bits 0...7 respectively and the new "types" LSO, LL, BL getting bits 8,9,10 . Later change the libibverbs .h file to match this?! Or. From sashak at voltaire.com Wed Mar 26 05:38:56 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 12:38:56 +0000 Subject: [ofa-general] Re: [PATCH] infiniband-diags/vendstat.c: Fix port xmit wait handling In-Reply-To: <1206477325.8099.675.camel@hrosenstock-ws.xsigo.com> References: <1206477325.8099.675.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080326123856.GC13708@sashak.voltaire.com> On 13:35 Tue 25 Mar , Hal Rosenstock wrote: > Fix config space accesses for port xmit wait > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From jackm at dev.mellanox.co.il Wed Mar 26 05:21:29 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 14:21:29 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <47EA3E87.9090200@voltaire.com> References: <1205767427.25950.137.camel@mtls03> <200803261338.01131.jackm@dev.mellanox.co.il> <47EA3E87.9090200@voltaire.com> Message-ID: <200803261421.29382.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 14:16, Or Gerlitz wrote: > So with that assumption at hand, the above enum has the values 0...7 > occupied, what would be the problem to replace in the kernel .h file, > the enum qp_type field with u32 qp_type_bits field where the above types > getting bits 0...7 respectively and the new "types" LSO, LL, BL getting > bits 8,9,10 . Later change the libibverbs .h file to match this?! > This *does* break the ABI. An older libibverbs will not interpret the qp_type in the same way as the new kernel will, so you will get a mish-mash. - Jack From ossrosch at linux.vnet.ibm.com Wed Mar 26 05:30:56 2008 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 26 Mar 2008 13:30:56 +0100 Subject: [ofa-general] OFED March 24 meeting summary on OFED 1.4 plans In-Reply-To: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> Message-ID: <200803261330.57342.ossrosch@linux.vnet.ibm.com> On Monday 24 March 2008 21:45:01 Tziporet Koren wrote: > > > * New vendor HW support - non was reported so far (IBM and Chelsio > > - do you have something?) > > * OpenSM: We are not planning to include new features for ehca in OFED-1.4. regards Stefan From HNGUYEN at de.ibm.com Wed Mar 26 05:32:34 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 26 Mar 2008 13:32:34 +0100 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <1206532118.25950.408.camel@mtls03> Message-ID: > I don't think it is a good idea to mix enumerated types with bitmasks > since bitmasks are wasting too much of the size of the enum (which > depends on the CPU architecture). I like creation flags more also > because it allows more freedom in choosing the semantics of the flag and > not worrying if it fits in the category as a kind of "qp type". Eli and Jack, Rethought about this and agree that keeping those enums separately is cleaner. @Roland (and all), depending on this outcome I'll create my patch appropriately. Thx Nam From ogerlitz at voltaire.com Wed Mar 26 05:34:25 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 26 Mar 2008 14:34:25 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <200803261421.29382.jackm@dev.mellanox.co.il> References: <1205767427.25950.137.camel@mtls03> <200803261338.01131.jackm@dev.mellanox.co.il> <47EA3E87.9090200@voltaire.com> <200803261421.29382.jackm@dev.mellanox.co.il> Message-ID: <47EA42D1.4010308@voltaire.com> Jack Morgenstein wrote: > This *does* break the ABI. An older libibverbs will not interpret the qp_type > in the same way as the new kernel will, so you will get a mish-mash. > > This breakage is related to ---forward-- compatibility - that is have an app built against a new version of libibverbs working with --older-- version of libibverbs. I wanted to see that we have backward compatibility. Is libibverbs protected against such breakage all over the place? Or. From clli at blockiron.com Wed Mar 26 07:10:32 2008 From: clli at blockiron.com (Henry Dupree) Date: Wed, 26 Mar 2008 23:10:32 +0900 Subject: [ofa-general] Master in bed games Message-ID: <01c88f96$975a4100$11463775@clli> On top all night Look attached details and find MORE! -------------- next part -------------- A non-text attachment was scrubbed... Name: vo.zip Type: application/zip Size: 636 bytes Desc: not available URL: From jackm at dev.mellanox.co.il Wed Mar 26 07:20:18 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 16:20:18 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <47EA42D1.4010308@voltaire.com> References: <1205767427.25950.137.camel@mtls03> <200803261421.29382.jackm@dev.mellanox.co.il> <47EA42D1.4010308@voltaire.com> Message-ID: <200803261620.19395.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 14:34, Or Gerlitz wrote: > This breakage is related to ---forward-- compatibility - that is have an > app built against a new version of libibverbs working with --older-- > version of libibverbs. I wanted to see that we have backward compatibility. > No, it isn't. If the customer has an old libibverbs running against a new kernel (which uses your bitmap suggestion of bits 0..7), all his apps will fail to run (the qp_type passed by the old libibverbs to the new kernel will be misinterpreted). - Jack From GlennocularGibson at mediamonitors.net Tue Mar 25 16:51:39 2008 From: GlennocularGibson at mediamonitors.net (Bryan Woods) Date: Wed, 26 Mar 2008 07:51:39 +0800 Subject: [ofa-general] Be Debt Free: Fast & Easy Message-ID: <7IX597EJXVWDA106@mediamonitors.net> An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Wed Mar 26 07:53:30 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 26 Mar 2008 16:53:30 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <200803261620.19395.jackm@dev.mellanox.co.il> References: <1205767427.25950.137.camel@mtls03> <200803261421.29382.jackm@dev.mellanox.co.il> <47EA42D1.4010308@voltaire.com> <200803261620.19395.jackm@dev.mellanox.co.il> Message-ID: <47EA636A.8070604@voltaire.com> Jack Morgenstein wrote: > No, it isn't. If the customer has an old libibverbs running against a new > kernel (which uses your bitmap suggestion of bits 0..7), all his apps will fail > to run (the qp_type passed by the old libibverbs to the new kernel will be > misinterpreted). Ok, Jack, I see your point now, sorry for being slow to catch up, so with this in mind: > This means that we are using the 3 LSBs in an enumerated fashion (integers 0..7), so > we can't use a bitmap here. This leads us to a mixed field anyway -- > 3 bits for existing QP types (integers 0..6), and 29 bits of bit-map. Yes, so way we can go the somehow not very elegant way, that is have the uverbs code examine the u32 value and if the only non zero bits are among the 3 LS ones, uses the values from the old enum and if any of the remaining 29 bits are set consider their values under the bit field scheme, etc. > > Why not just keep things separate, and use the enumeration as is for the existing types, > and keep the bitmap as a separate "flags" field -- this is much cleaner. and this means breaking the ABI, correct. If such a breakage is going to happen anyway for the merge of XRC into the mainline kernel and the official libibverbs as maintained by Roland, I guess this is fine. I see now that the device capabilities enum is managed in bit field fashion so at least there the kernel can advertise easily the block-loopback support as you did for XRC (below) why have you left 6 unused bits Or > enum ibv_device_cap_flags { > IBV_DEVICE_RESIZE_MAX_WR = 1, > IBV_DEVICE_BAD_PKEY_CNTR = 1 << 1, > IBV_DEVICE_BAD_QKEY_CNTR = 1 << 2, > IBV_DEVICE_RAW_MULTI = 1 << 3, > IBV_DEVICE_AUTO_PATH_MIG = 1 << 4, > IBV_DEVICE_CHANGE_PHY_PORT = 1 << 5, > IBV_DEVICE_UD_AV_PORT_ENFORCE = 1 << 6, > IBV_DEVICE_CURR_QP_STATE_MOD = 1 << 7, > IBV_DEVICE_SHUTDOWN_PORT = 1 << 8, > IBV_DEVICE_INIT_TYPE = 1 << 9, > IBV_DEVICE_PORT_ACTIVE_EVENT = 1 << 10, > IBV_DEVICE_SYS_IMAGE_GUID = 1 << 11, > IBV_DEVICE_RC_RNR_NAK_GEN = 1 << 12, > IBV_DEVICE_SRQ_RESIZE = 1 << 13, > IBV_DEVICE_N_NOTIFY_CQ = 1 << 14, > IBV_DEVICE_XRC = 1 << 20 > }; > From girdwoynartur at cm-lim.com.pl Wed Mar 26 06:20:48 2008 From: girdwoynartur at cm-lim.com.pl (jimmy kecia) Date: Wed, 26 Mar 2008 13:20:48 +0000 Subject: [ofa-general] Rihanna exposed Message-ID: <000801c88f53$05503f39$384ba79c@vqamq> bl?ttern Download and Watch -------------- next part -------------- An HTML attachment was scrubbed... URL: From HNGUYEN at de.ibm.com Wed Mar 26 08:08:51 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 26 Mar 2008 16:08:51 +0100 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <47EA636A.8070604@voltaire.com> Message-ID: > Jack Morgenstein wrote: > > No, it isn't. If the customer has an old libibverbs running against a new > > kernel (which uses your bitmap suggestion of bits 0..7), all his > apps will fail > > to run (the qp_type passed by the old libibverbs to the new kernel will be > > misinterpreted). > Ok, Jack, I see your point now, sorry for being slow to catch up, so > with this in mind: > > This means that we are using the 3 LSBs in an enumerated fashion > (integers 0..7), so > > we can't use a bitmap here. This leads us to a mixed field anyway -- > > 3 bits for existing QP types (integers 0..6), and 29 bits of bit-map. > Yes, so way we can go the somehow not very elegant way, that is have the > uverbs code examine the u32 value and if the only non zero bits are > among the 3 LS ones, uses the values from the old enum and if any of the > remaining 29 bits are set consider their values under the bit field > scheme, etc. BTW: qp_type in struct ib_uverbs_create_qp is a __u8: struct ib_uverbs_create_qp { __u64 response; __u64 user_handle; __u32 pd_handle; __u32 send_cq_handle; __u32 recv_cq_handle; __u32 srq_handle; __u32 max_send_wr; __u32 max_recv_wr; __u32 max_send_sge; __u32 max_recv_sge; __u32 max_inline_data; __u8 sq_sig_all; __u8 qp_type; __u8 is_srq; __u8 qp_create_flags; __u64 driver_data[0]; }; Nam From ogerlitz at voltaire.com Wed Mar 26 08:15:26 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 26 Mar 2008 17:15:26 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: References: Message-ID: <47EA688E.6070102@voltaire.com> Hoang-Nam Nguyen wrote: > BTW: qp_type in struct ib_uverbs_create_qp is a __u8: mmm, we can still scale up to 5 features on top of the existing types. Or. > struct ib_uverbs_create_qp { > __u64 response; > __u64 user_handle; > __u32 pd_handle; > __u32 send_cq_handle; > __u32 recv_cq_handle; > __u32 srq_handle; > __u32 max_send_wr; > __u32 max_recv_wr; > __u32 max_send_sge; > __u32 max_recv_sge; > __u32 max_inline_data; > __u8 sq_sig_all; > __u8 qp_type; > __u8 is_srq; > __u8 qp_create_flags; > __u64 driver_data[0]; > }; > > > From jackm at dev.mellanox.co.il Wed Mar 26 08:16:25 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 17:16:25 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <47EA636A.8070604@voltaire.com> References: <1205767427.25950.137.camel@mtls03> <200803261620.19395.jackm@dev.mellanox.co.il> <47EA636A.8070604@voltaire.com> Message-ID: <200803261716.26026.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 16:53, Or Gerlitz wrote: > and this means breaking the ABI, correct. If such a breakage is going to > happen anyway for the merge of XRC into the mainline kernel and the > official libibverbs as maintained by Roland, I guess this is fine. I see > Actually, we did not break the ABI for XRC (we had to do cartwheels to avoid this, but what the heck). The create_flags field was added ONLY to struct ib_qp_init_attr (kernel only), and was not exported upwards to the userspace API (so as not to break the ABI). I got around the create_flags problem by adding a new verb to userspace (ibv_create_xrc_rcv_qp() ) with its own ABI to kernel space. Since the kernel-space function (added to uverbs_cmd: ib_uverbs_create_xrc_rcv_qp() ) "knew" that it was creating an XRC_RCV qp, it set the flag in ib_qp_init_attr appropriately. Additionally, Eli did not need user-space involvement for his LSO flag -- so we escaped needing to increment the ABI version number. If we need to use the create_flags field in userspace, we WILL need to break the ABI. You can avoid doing this in one of 2 ways: Either: create a new QP type Or: add new verbs to the core. - Jack From jackm at dev.mellanox.co.il Wed Mar 26 08:17:21 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 17:17:21 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: References: Message-ID: <200803261717.21606.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 17:08, Hoang-Nam Nguyen wrote: > BTW: qp_type in struct ib_uverbs_create_qp is a __u8: > Oops. - Jack From jackm at dev.mellanox.co.il Wed Mar 26 08:27:38 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 26 Mar 2008 17:27:38 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: References: Message-ID: <200803261727.38879.jackm@dev.mellanox.co.il> On Wednesday 26 March 2008 17:08, Hoang-Nam Nguyen wrote: > __u8  qp_create_flags; I see this as a reserved field in OFED 1.3 . Is this something that you added? - Jack From HNGUYEN at de.ibm.com Wed Mar 26 08:46:08 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 26 Mar 2008 16:46:08 +0100 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <200803261727.38879.jackm@dev.mellanox.co.il> Message-ID: > On Wednesday 26 March 2008 17:08, Hoang-Nam Nguyen wrote: > > __u8  qp_create_flags; > > I see this as a reserved field in OFED 1.3 . Is this something thatyou added? Right, omm, my fault. Was testing my private stuff. qp_create_flags should read reserved. Thx for pointing that out. Nam From sashak at voltaire.com Wed Mar 26 10:10:48 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 17:10:48 +0000 Subject: [ofa-general] [PATCH] opensm: switch LFTs incremental update fix Message-ID: <20080326171048.GJ13708@sashak.voltaire.com> When switch is discovered first time its port links could be already initialized by previous SM, so in this case LFTs update must be enforced in order to overwrite an "old junk". This patch enforces sw->need_update flag when switch is found first time. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_port_info_rcv.c | 3 ++- opensm/opensm/osm_switch.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/opensm/opensm/osm_port_info_rcv.c b/opensm/opensm/osm_port_info_rcv.c index c0d9f1f..ff75ef0 100644 --- a/opensm/opensm/osm_port_info_rcv.c +++ b/opensm/opensm/osm_port_info_rcv.c @@ -314,7 +314,8 @@ __osm_pi_rcv_process_switch_port(IN osm_sm_t * sm, } } - if (ib_port_info_get_port_state(p_pi) > IB_LINK_INIT && p_node->sw) + if (ib_port_info_get_port_state(p_pi) > IB_LINK_INIT && p_node->sw && + p_node->sw->need_update == 1) p_node->sw->need_update = 0; if (p_physp->need_update) diff --git a/opensm/opensm/osm_switch.c b/opensm/opensm/osm_switch.c index c962be7..9623db1 100644 --- a/opensm/opensm/osm_switch.c +++ b/opensm/opensm/osm_switch.c @@ -100,7 +100,7 @@ osm_switch_init(IN osm_switch_t * const p_sw, p_sw->p_node = p_node; p_sw->switch_info = *p_si; p_sw->num_ports = num_ports; - p_sw->need_update = 1; + p_sw->need_update = 2; status = osm_fwd_tbl_init(&p_sw->fwd_tbl, p_si); if (status != IB_SUCCESS) -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 10:27:03 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 17:27:03 +0000 Subject: [ofa-general] [PATCH 1/4] opensm: osm_dump_qmap_to_file() function In-Reply-To: <1206552426-17279-1-git-send-email-sashak@voltaire.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> Message-ID: <1206552426-17279-2-git-send-email-sashak@voltaire.com> This makes public osm_dump_qmap_to_file() function to allow module specific file dumpers and updates rest of osm_dump.c code accordingly. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_opensm.h | 4 ++ opensm/opensm/osm_dump.c | 104 ++++++++++++++++++++--------------- 2 files changed, 63 insertions(+), 45 deletions(-) diff --git a/opensm/include/opensm/osm_opensm.h b/opensm/include/opensm/osm_opensm.h index cd1e73b..c8f9c92 100644 --- a/opensm/include/opensm/osm_opensm.h +++ b/opensm/include/opensm/osm_opensm.h @@ -503,6 +503,10 @@ osm_routing_engine_type_t osm_routing_engine_type(IN const char *str); /* dump helpers */ void osm_dump_mcast_routes(osm_opensm_t * osm); void osm_dump_all(osm_opensm_t * osm); +void osm_dump_qmap_to_file(osm_opensm_t * p_osm, const char *file_name, + cl_qmap_t * map, + void (*func) (cl_map_item_t *, FILE *, void *), + void *cxt); /****v* OpenSM/osm_exit_flag */ diff --git a/opensm/opensm/osm_dump.c b/opensm/opensm/osm_dump.c index da4ed49..b350836 100644 --- a/opensm/opensm/osm_dump.c +++ b/opensm/opensm/osm_dump.c @@ -57,12 +57,8 @@ #include #include -struct dump_context { - osm_opensm_t *p_osm; - FILE *file; -}; - -static void dump_ucast_path_distribution(cl_map_item_t * p_map_item, void *cxt) +static void dump_ucast_path_distribution(cl_map_item_t * p_map_item, + FILE *file, void *cxt) { osm_node_t *p_node; osm_node_t *p_remote_node; @@ -71,7 +67,7 @@ static void dump_ucast_path_distribution(cl_map_item_t * p_map_item, void *cxt) uint32_t num_paths; ib_net64_t remote_guid_ho; osm_switch_t *p_sw = (osm_switch_t *) p_map_item; - osm_opensm_t *p_osm = ((struct dump_context *)cxt)->p_osm; + osm_opensm_t *p_osm = cxt; p_node = p_sw->p_node; num_ports = p_sw->num_ports; @@ -125,7 +121,7 @@ static void dump_ucast_path_distribution(cl_map_item_t * p_map_item, void *cxt) osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, "\n"); } -static void dump_ucast_routes(cl_map_item_t * p_map_item, void *cxt) +static void dump_ucast_routes(cl_map_item_t *p_map_item, FILE *file, void *cxt) { const osm_node_t *p_node; osm_port_t *p_port; @@ -138,8 +134,7 @@ static void dump_ucast_routes(cl_map_item_t * p_map_item, void *cxt) boolean_t direct_route_exists = FALSE; boolean_t dor; osm_switch_t *p_sw = (osm_switch_t *) p_map_item; - osm_opensm_t *p_osm = ((struct dump_context *)cxt)->p_osm; - FILE *file = ((struct dump_context *)cxt)->file; + osm_opensm_t *p_osm = cxt; p_node = p_sw->p_node; @@ -245,10 +240,9 @@ static void dump_ucast_routes(cl_map_item_t * p_map_item, void *cxt) } } -static void dump_mcast_routes(cl_map_item_t * p_map_item, void *cxt) +static void dump_mcast_routes(cl_map_item_t *p_map_item, FILE *file, void *cxt) { osm_switch_t *p_sw = (osm_switch_t *) p_map_item; - FILE *file = ((struct dump_context *)cxt)->file; osm_mcast_tbl_t *p_tbl; int16_t mlid_ho = 0; int16_t mlid_start_ho; @@ -311,11 +305,10 @@ static void dump_mcast_routes(cl_map_item_t * p_map_item, void *cxt) } } -static void dump_lid_matrix(cl_map_item_t * p_map_item, void *cxt) +static void dump_lid_matrix(cl_map_item_t *p_map_item, FILE *file, void *cxt) { osm_switch_t *p_sw = (osm_switch_t *) p_map_item; - osm_opensm_t *p_osm = ((struct dump_context *)cxt)->p_osm; - FILE *file = ((struct dump_context *)cxt)->file; + osm_opensm_t *p_osm = cxt; osm_node_t *p_node = p_sw->p_node; unsigned max_lid = p_sw->max_lid_ho; unsigned max_port = p_sw->num_ports; @@ -340,11 +333,10 @@ static void dump_lid_matrix(cl_map_item_t * p_map_item, void *cxt) } } -static void dump_ucast_lfts(cl_map_item_t * p_map_item, void *cxt) +static void dump_ucast_lfts(cl_map_item_t *p_map_item, FILE *file, void *cxt) { osm_switch_t *p_sw = (osm_switch_t *) p_map_item; - osm_opensm_t *p_osm = ((struct dump_context *)cxt)->p_osm; - FILE *file = ((struct dump_context *)cxt)->file; + osm_opensm_t *p_osm = cxt; osm_node_t *p_node = p_sw->p_node; unsigned max_lid = p_sw->max_lid_ho; unsigned max_port = p_sw->num_ports; @@ -378,10 +370,9 @@ static void dump_ucast_lfts(cl_map_item_t * p_map_item, void *cxt) fprintf(file, "%u lids dumped\n", max_lid); } -static void dump_topology_node(cl_map_item_t * p_map_item, void *cxt) +static void dump_topology_node(cl_map_item_t *p_map_item, FILE *file, void *cxt) { osm_node_t *p_node = (osm_node_t *) p_map_item; - FILE *file = ((struct dump_context *)cxt)->file; uint32_t cPort; osm_node_t *p_nbnode; osm_physp_t *p_physp, *p_default_physp, *p_rphysp; @@ -481,10 +472,10 @@ static void dump_topology_node(cl_map_item_t * p_map_item, void *cxt) } } -static void print_node_report(cl_map_item_t * p_map_item, void *cxt) +static void print_node_report(cl_map_item_t * p_map_item, FILE *file, void *cxt) { osm_node_t *p_node = (osm_node_t *) p_map_item; - osm_opensm_t *osm = ((struct dump_context *)cxt)->p_osm; + osm_opensm_t *osm = cxt; osm_log_t *log = &osm->log; const osm_physp_t *p_physp, *p_remote_physp; const ib_port_info_t *p_pi; @@ -586,20 +577,37 @@ static void print_node_report(cl_map_item_t * p_map_item, void *cxt) /********************************************************************** **********************************************************************/ -static void dump_qmap(osm_opensm_t * p_osm, FILE * file, - cl_qmap_t * map, void (*func) (cl_map_item_t *, void *)) +struct dump_context { + osm_opensm_t *p_osm; + FILE *file; + void (*func) (cl_map_item_t *, FILE *, void *); + void *cxt; +}; + +static void dump_item(cl_map_item_t *item, void *cxt) +{ + ((struct dump_context *)cxt)->func(item, + ((struct dump_context *)cxt)->file, + ((struct dump_context *)cxt)->cxt); +} + +static void dump_qmap(FILE * file, cl_qmap_t * map, + void (*func) (cl_map_item_t *, FILE *, void *), + void *cxt) { struct dump_context dump_context; - dump_context.p_osm = p_osm; dump_context.file = file; + dump_context.func = func; + dump_context.cxt = cxt; - cl_qmap_apply_func(map, func, &dump_context); + cl_qmap_apply_func(map, dump_item, &dump_context); } -static void dump_qmap_to_file(osm_opensm_t * p_osm, const char *file_name, - cl_qmap_t * map, - void (*func) (cl_map_item_t *, void *)) +void osm_dump_qmap_to_file(osm_opensm_t * p_osm, const char *file_name, + cl_qmap_t * map, + void (*func) (cl_map_item_t *, FILE *, void *), + void *cxt) { char path[1024]; FILE *file; @@ -615,7 +623,7 @@ static void dump_qmap_to_file(osm_opensm_t * p_osm, const char *file_name, return; } - dump_qmap(p_osm, file, map, func); + dump_qmap(file, map, func, cxt); fclose(file); } @@ -632,15 +640,16 @@ static void print_report(osm_opensm_t * osm) ": # : Sta : LID : LMC : MTU : LWA : LSA : Port GUID " " : Neighbor Port (Port #)\n"); - dump_qmap(osm, NULL, &osm->subn.node_guid_tbl, print_node_report); + dump_qmap(stdout, &osm->subn.node_guid_tbl, print_node_report, osm); } void osm_dump_mcast_routes(osm_opensm_t * osm) { if (osm_log_is_active(&osm->log, OSM_LOG_ROUTING)) { /* multicast routes */ - dump_qmap_to_file(osm, "opensm.mcfdbs", - &osm->subn.sw_guid_tbl, dump_mcast_routes); + osm_dump_qmap_to_file(osm, "opensm.mcfdbs", + &osm->subn.sw_guid_tbl, + dump_mcast_routes, osm); } } @@ -648,21 +657,26 @@ void osm_dump_all(osm_opensm_t * osm) { if (osm_log_is_active(&osm->log, OSM_LOG_ROUTING)) { /* unicast routes */ - dump_qmap_to_file(osm, "opensm-lid-matrix.dump", - &osm->subn.sw_guid_tbl, dump_lid_matrix); - dump_qmap_to_file(osm, "opensm-lfts.dump", - &osm->subn.sw_guid_tbl, dump_ucast_lfts); + osm_dump_qmap_to_file(osm, "opensm-lid-matrix.dump", + &osm->subn.sw_guid_tbl, dump_lid_matrix, + osm); + osm_dump_qmap_to_file(osm, "opensm-lfts.dump", + &osm->subn.sw_guid_tbl, dump_ucast_lfts, + osm); if (osm_log_is_active(&osm->log, OSM_LOG_DEBUG)) - dump_qmap(osm, NULL, &osm->subn.sw_guid_tbl, - dump_ucast_path_distribution); - dump_qmap_to_file(osm, "opensm.fdbs", - &osm->subn.sw_guid_tbl, dump_ucast_routes); + dump_qmap(stdout, &osm->subn.sw_guid_tbl, + dump_ucast_path_distribution, osm); + osm_dump_qmap_to_file(osm, "opensm.fdbs", + &osm->subn.sw_guid_tbl, + dump_ucast_routes, osm); /* multicast routes */ - dump_qmap_to_file(osm, "opensm.mcfdbs", - &osm->subn.sw_guid_tbl, dump_mcast_routes); + osm_dump_qmap_to_file(osm, "opensm.mcfdbs", + &osm->subn.sw_guid_tbl, + dump_mcast_routes, osm); } - dump_qmap_to_file(osm, "opensm-subnet.lst", &osm->subn.node_guid_tbl, - dump_topology_node); + osm_dump_qmap_to_file(osm, "opensm-subnet.lst", + &osm->subn.node_guid_tbl, dump_topology_node, + osm); if (osm_log_is_active(&osm->log, OSM_LOG_VERBOSE)) print_report(osm); } -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 10:27:02 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 17:27:02 +0000 Subject: [ofa-general] [PATCH 0/4] opensm: dump updn root nodes guids and dumper improvements Message-ID: <1206552426-17279-1-git-send-email-sashak@voltaire.com> Hi, This patch series add ability to use qmap dumpers in OpenSM "opaque" style modules and use this with new Up/Down root nodes guids dumper. Also some improvements and cleanups in dumper interface and functions were done here. Sasha From sashak at voltaire.com Wed Mar 26 10:27:04 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 17:27:04 +0000 Subject: [ofa-general] [PATCH 2/4] opensm/updn: dump used root nodes guid In-Reply-To: <1206552426-17279-1-git-send-email-sashak@voltaire.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> Message-ID: <1206552426-17279-3-git-send-email-sashak@voltaire.com> This dump file has similar to root_node_guids format. Useful for debugging and root nodes list generation. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_updn.c | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_ucast_updn.c b/opensm/opensm/osm_ucast_updn.c index 0dc9f1f..93e6c7c 100644 --- a/opensm/opensm/osm_ucast_updn.c +++ b/opensm/opensm/osm_ucast_updn.c @@ -544,6 +544,14 @@ static void delete_updn_node(struct updn_node *u) /********************************************************************** **********************************************************************/ +static void dump_roots(cl_map_item_t *item, FILE *file, void *cxt) +{ + osm_switch_t *sw = (osm_switch_t *)item; + if (!((struct updn_node *)sw->priv)->rank) + fprintf(file, "0x%" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(sw->p_node))); +} + /* UPDN callback function */ static int __osm_updn_call(void *ctx) { @@ -590,6 +598,11 @@ static int __osm_updn_call(void *ctx) ret = 1; } + if (osm_log_is_active(&p_updn->p_osm->log, OSM_LOG_ROUTING)) + osm_dump_qmap_to_file(p_updn->p_osm, "opensm-updn-roots.dump", + &p_updn->p_osm->subn.sw_guid_tbl, + dump_roots, NULL); + p_item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); while (p_item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl)) { p_sw = (osm_switch_t *) p_item; -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 10:27:05 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 17:27:05 +0000 Subject: [ofa-general] [PATCH 3/4] opensm: unify dumpers, use fprintf() every there In-Reply-To: <1206552426-17279-1-git-send-email-sashak@voltaire.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> Message-ID: <1206552426-17279-4-git-send-email-sashak@voltaire.com> Unify dumper functions, use fprintf() (instead of osm_log_printf()) with stdout as parameter when output is a terminal. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_dump.c | 157 +++++++++++++++++---------------------------- 1 files changed, 59 insertions(+), 98 deletions(-) diff --git a/opensm/opensm/osm_dump.c b/opensm/opensm/osm_dump.c index b350836..33a3ce0 100644 --- a/opensm/opensm/osm_dump.c +++ b/opensm/opensm/osm_dump.c @@ -67,24 +67,19 @@ static void dump_ucast_path_distribution(cl_map_item_t * p_map_item, uint32_t num_paths; ib_net64_t remote_guid_ho; osm_switch_t *p_sw = (osm_switch_t *) p_map_item; - osm_opensm_t *p_osm = cxt; p_node = p_sw->p_node; num_ports = p_sw->num_ports; - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, - "dump_ucast_path_distribution: " - "Switch 0x%" PRIx64 "\n" - "Port : Path Count Through Port", - cl_ntoh64(osm_node_get_node_guid(p_node))); + fprintf(file, "dump_ucast_path_distribution: Switch 0x%" PRIx64 "\n" + "Port : Path Count Through Port", + cl_ntoh64(osm_node_get_node_guid(p_node))); for (i = 0; i < num_ports; i++) { num_paths = osm_switch_path_count_get(p_sw, i); - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, "\n %03u : %u", i, - num_paths); + fprintf(file, "\n %03u : %u", i, num_paths); if (i == 0) { - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, - " (switch management port)"); + fprintf(file, " (switch management port)"); continue; } @@ -97,28 +92,23 @@ static void dump_ucast_path_distribution(cl_map_item_t * p_map_item, switch (osm_node_get_type(p_remote_node)) { case IB_NODE_TYPE_SWITCH: - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, - " (link to switch"); + fprintf(file, " (link to switch"); break; case IB_NODE_TYPE_ROUTER: - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, - " (link to router"); + fprintf(file, " (link to router"); break; case IB_NODE_TYPE_CA: - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, - " (link to CA"); + fprintf(file, " (link to CA"); break; default: - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, - " (link to unknown node type"); + fprintf(file, " (link to unknown node type"); break; } - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, " 0x%" PRIx64 ")", - remote_guid_ho); + fprintf(file, " 0x%" PRIx64 ")", remote_guid_ho); } - osm_log_printf(&p_osm->log, OSM_LOG_DEBUG, "\n"); + fprintf(file, "\n"); } static void dump_ucast_routes(cl_map_item_t *p_map_item, FILE *file, void *cxt) @@ -141,8 +131,7 @@ static void dump_ucast_routes(cl_map_item_t *p_map_item, FILE *file, void *cxt) max_lid_ho = p_sw->max_lid_ho; fprintf(file, "__osm_ucast_mgr_dump_ucast_routes: " - "Switch 0x%016" PRIx64 "\n" - "LID : Port : Hops : Optimal\n", + "Switch 0x%016" PRIx64 "\nLID : Port : Hops : Optimal\n", cl_ntoh64(osm_node_get_node_guid(p_node))); dor = (p_osm->routing_engine_used == OSM_ROUTING_ENGINE_TYPE_DOR); @@ -230,8 +219,7 @@ static void dump_ucast_routes(cl_map_item_t *p_map_item, FILE *file, void *cxt) /* No LMC Optimization */ best_port = osm_switch_recommend_path(p_sw, p_port, lid_ho, TRUE, dor, - NULL, NULL, - NULL); + NULL, NULL, NULL); fprintf(file, "No %u hop path possible via port %u!", best_hops, best_port); } @@ -260,8 +248,7 @@ static void dump_mcast_routes(cl_map_item_t *p_map_item, FILE *file, void *cxt) p_tbl = osm_switch_get_mcast_tbl_ptr(p_sw); - sprintf(sw_hdr, "\nSwitch 0x%016" PRIx64 "\n" - "LID : Out Port(s)\n", + sprintf(sw_hdr, "\nSwitch 0x%016" PRIx64 "\nLID : Out Port(s)\n", cl_ntoh64(osm_node_get_node_guid(p_node))); first_mlid = TRUE; while (block_num <= p_tbl->max_block_in_use) { @@ -399,10 +386,8 @@ static void dump_topology_node(cl_map_item_t *p_map_item, FILE *file, void *cxt) else p_default_physp = p_physp; - fprintf(file, "{ %s%s Ports:%02X" - " SystemGUID:%016" PRIx64 - " NodeGUID:%016" PRIx64 - " PortGUID:%016" PRIx64 + fprintf(file, "{ %s%s Ports:%02X SystemGUID:%016" PRIx64 + " NodeGUID:%016" PRIx64 " PortGUID:%016" PRIx64 " VenID:%06X DevID:%04X Rev:%08X {%s} LID:%04X PN:%02X } ", p_node->node_info.node_type == IB_NODE_TYPE_SWITCH ? "SW" : p_node->node_info.node_type == @@ -429,10 +414,8 @@ static void dump_topology_node(cl_map_item_t *p_map_item, FILE *file, void *cxt) else p_default_physp = p_rphysp; - fprintf(file, "{ %s%s Ports:%02X" - " SystemGUID:%016" PRIx64 - " NodeGUID:%016" PRIx64 - " PortGUID:%016" PRIx64 + fprintf(file, "{ %s%s Ports:%02X SystemGUID:%016" PRIx64 + " NodeGUID:%016" PRIx64 " PortGUID:%016" PRIx64 " VenID:%08X DevID:%04X Rev:%08X {%s} LID:%04X PN:%02X } ", p_nbnode->node_info.node_type == IB_NODE_TYPE_SWITCH ? "SW" : p_nbnode->node_info.node_type == @@ -472,22 +455,16 @@ static void dump_topology_node(cl_map_item_t *p_map_item, FILE *file, void *cxt) } } -static void print_node_report(cl_map_item_t * p_map_item, FILE *file, void *cxt) +static void print_node_report(cl_map_item_t *p_map_item, FILE *file, void *cxt) { osm_node_t *p_node = (osm_node_t *) p_map_item; osm_opensm_t *osm = cxt; - osm_log_t *log = &osm->log; const osm_physp_t *p_physp, *p_remote_physp; const ib_port_info_t *p_pi; uint8_t port_num; uint32_t num_ports; uint8_t node_type; - if (osm_log_is_active(log, OSM_LOG_DEBUG)) - OSM_LOG(log, OSM_LOG_DEBUG, - "Processing node 0x%016" PRIx64 "\n", - cl_ntoh64(osm_node_get_node_guid(p_node))); - node_type = osm_node_get_type(p_node); num_ports = osm_node_get_num_physp(p_node); @@ -497,12 +474,11 @@ static void print_node_report(cl_map_item_t * p_map_item, FILE *file, void *cxt) if (!p_physp) continue; - osm_log_printf(log, OSM_LOG_VERBOSE, "%-11s : %s : %02X :", - osm_get_manufacturer_str(cl_ntoh64 - (osm_node_get_node_guid - (p_node))), - osm_get_node_type_str_fixed_width - (node_type), port_num); + fprintf(file, "%-11s : %s : %02X :", + osm_get_manufacturer_str(cl_ntoh64 + (osm_node_get_node_guid + (p_node))), + osm_get_node_type_str_fixed_width(node_type), port_num); p_pi = &p_physp->port_info; @@ -510,11 +486,11 @@ static void print_node_report(cl_map_item_t * p_map_item, FILE *file, void *cxt) * Port state is not defined for switch port 0 */ if (port_num == 0) - osm_log_printf(log, OSM_LOG_VERBOSE, " :"); + fprintf(file, " :"); else - osm_log_printf(log, OSM_LOG_VERBOSE, " %s :", - osm_get_port_state_str_fixed_width - (ib_port_info_get_port_state(p_pi))); + fprintf(file, " %s :", + osm_get_port_state_str_fixed_width + (ib_port_info_get_port_state(p_pi))); /* * LID values are only meaningful in select cases. @@ -522,57 +498,46 @@ static void print_node_report(cl_map_item_t * p_map_item, FILE *file, void *cxt) if (ib_port_info_get_port_state(p_pi) != IB_LINK_DOWN && ((node_type == IB_NODE_TYPE_SWITCH && port_num == 0) || node_type != IB_NODE_TYPE_SWITCH)) - osm_log_printf(log, OSM_LOG_VERBOSE, " %04X : %01X :", - cl_ntoh16(p_pi->base_lid), - ib_port_info_get_lmc(p_pi)); + fprintf(file, " %04X : %01X :", + cl_ntoh16(p_pi->base_lid), + ib_port_info_get_lmc(p_pi)); else - osm_log_printf(log, OSM_LOG_VERBOSE, " : :"); + fprintf(file, " : :"); if (port_num != 0) - osm_log_printf(log, OSM_LOG_VERBOSE, " %s : %s : %s ", - osm_get_mtu_str - (ib_port_info_get_neighbor_mtu(p_pi)), - osm_get_lwa_str(p_pi->link_width_active), - osm_get_lsa_str - (ib_port_info_get_link_speed_active - (p_pi))); + fprintf(file, " %s : %s : %s ", + osm_get_mtu_str + (ib_port_info_get_neighbor_mtu(p_pi)), + osm_get_lwa_str(p_pi->link_width_active), + osm_get_lsa_str + (ib_port_info_get_link_speed_active(p_pi))); else - osm_log_printf(log, OSM_LOG_VERBOSE, - " : : "); + fprintf(file, " : : "); if (osm_physp_get_port_guid(p_physp) == osm->subn.sm_port_guid) - osm_log_printf(log, OSM_LOG_VERBOSE, - "* %016" PRIx64 " *", - cl_ntoh64(osm_physp_get_port_guid - (p_physp))); + fprintf(file, "* %016" PRIx64 " *", + cl_ntoh64(osm_physp_get_port_guid(p_physp))); else - osm_log_printf(log, OSM_LOG_VERBOSE, - ": %016" PRIx64 " :", - cl_ntoh64(osm_physp_get_port_guid - (p_physp))); + fprintf(file, ": %016" PRIx64 " :", + cl_ntoh64(osm_physp_get_port_guid(p_physp))); if (port_num && (ib_port_info_get_port_state(p_pi) != IB_LINK_DOWN)) { p_remote_physp = osm_physp_get_remote(p_physp); if (p_remote_physp) - osm_log_printf(log, OSM_LOG_VERBOSE, - " %016" PRIx64 " (%02X)", - cl_ntoh64 - (osm_physp_get_port_guid - (p_remote_physp)), - osm_physp_get_port_num - (p_remote_physp)); + fprintf(file, " %016" PRIx64 " (%02X)", + cl_ntoh64(osm_physp_get_port_guid + (p_remote_physp)), + osm_physp_get_port_num(p_remote_physp)); else - osm_log_printf(log, OSM_LOG_VERBOSE, - " UNKNOWN"); + fprintf(file, " UNKNOWN"); } - osm_log_printf(log, OSM_LOG_VERBOSE, "\n"); + fprintf(file, "\n"); } - osm_log_printf(log, OSM_LOG_VERBOSE, - "------------------------------------------------------" - "------------------------------------------------\n"); + fprintf(file, "------------------------------------------------------" + "------------------------------------------------\n"); } /********************************************************************** @@ -591,9 +556,8 @@ static void dump_item(cl_map_item_t *item, void *cxt) ((struct dump_context *)cxt)->cxt); } -static void dump_qmap(FILE * file, cl_qmap_t * map, - void (*func) (cl_map_item_t *, FILE *, void *), - void *cxt) +static void dump_qmap(FILE *file, cl_qmap_t *map, + void (*func)(cl_map_item_t *, FILE *, void *), void *cxt) { struct dump_context dump_context; @@ -631,15 +595,12 @@ void osm_dump_qmap_to_file(osm_opensm_t * p_osm, const char *file_name, /********************************************************************** **********************************************************************/ -static void print_report(osm_opensm_t * osm) +static void print_report(osm_opensm_t *osm, FILE *file) { - osm_log_printf(&osm->log, OSM_LOG_VERBOSE, - "\n===================================================" - "====================================================" - "\nVendor : Ty " - ": # : Sta : LID : LMC : MTU : LWA : LSA : Port GUID " - " : Neighbor Port (Port #)\n"); - + fprintf(file, "\n===================================================" + "====================================================\n" + "Vendor : Ty : # : Sta : LID : LMC : MTU : LWA :" + " LSA : Port GUID : Neighbor Port (Port #)\n"); dump_qmap(stdout, &osm->subn.node_guid_tbl, print_node_report, osm); } @@ -678,5 +639,5 @@ void osm_dump_all(osm_opensm_t * osm) &osm->subn.node_guid_tbl, dump_topology_node, osm); if (osm_log_is_active(&osm->log, OSM_LOG_VERBOSE)) - print_report(osm); + print_report(osm, stdout); } -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 10:27:06 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 17:27:06 +0000 Subject: [ofa-general] [PATCH 4/4] opensm: remove not used osm_log_printf() function In-Reply-To: <1206552426-17279-1-git-send-email-sashak@voltaire.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> Message-ID: <1206552426-17279-5-git-send-email-sashak@voltaire.com> Remove not used anymore osm_log_printf() function from osm_log. Update libopensm library version. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_log.h | 2 -- opensm/opensm/libopensm.map | 1 - opensm/opensm/libopensm.ver | 2 +- opensm/opensm/osm_log.c | 19 ------------------- 4 files changed, 1 insertions(+), 23 deletions(-) diff --git a/opensm/include/opensm/osm_log.h b/opensm/include/opensm/osm_log.h index dd63dc4..48fb15e 100644 --- a/opensm/include/opensm/osm_log.h +++ b/opensm/include/opensm/osm_log.h @@ -389,8 +389,6 @@ osm_log_is_active(IN const osm_log_t * const p_log, * osm_log_destroy *********/ -extern int osm_log_printf(osm_log_t * p_log, osm_log_level_t level, - const char *fmt, ...) STRICT_OSM_LOG_FORMAT; extern void osm_log_msg_box(osm_log_t *log, osm_log_level_t level, const char *func_name, const char *msg); extern void osm_log_raw(IN osm_log_t * const p_log, diff --git a/opensm/opensm/libopensm.map b/opensm/opensm/libopensm.map index 1d574bc..759a4e8 100644 --- a/opensm/opensm/libopensm.map +++ b/opensm/opensm/libopensm.map @@ -1,7 +1,6 @@ OPENSM_1.5 { global: osm_log; - osm_log_printf; osm_log_msg_box; osm_is_debug; osm_log_init; diff --git a/opensm/opensm/libopensm.ver b/opensm/opensm/libopensm.ver index c00a3a9..3324b1a 100644 --- a/opensm/opensm/libopensm.ver +++ b/opensm/opensm/libopensm.ver @@ -6,4 +6,4 @@ # API_REV - advance on any added API # RUNNING_REV - advance any change to the vendor files # AGE - number of backward versions the API still supports -LIBVERSION=2:0:1 +LIBVERSION=2:1:0 diff --git a/opensm/opensm/osm_log.c b/opensm/opensm/osm_log.c index b5b4bd9..727aabc 100644 --- a/opensm/opensm/osm_log.c +++ b/opensm/opensm/osm_log.c @@ -106,25 +106,6 @@ static void truncate_log_file(osm_log_t * const p_log) } #endif /* ndef WIN32 */ -int osm_log_printf(osm_log_t * p_log, osm_log_level_t level, - const char *fmt, ...) -{ - va_list args; - int ret; - - if (!(p_log->level & level)) - return 0; - - va_start(args, fmt); - ret = vfprintf(stdout, fmt, args); - va_end(args); - - if (p_log->flush || level & OSM_LOG_ERROR) - fflush(stdout); - - return ret; -} - void osm_log(IN osm_log_t * const p_log, IN const osm_log_level_t verbosity, IN const char *p_str, ...) -- 1.5.4.1.122.gaa8d From HNGUYEN at de.ibm.com Wed Mar 26 10:16:57 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Wed, 26 Mar 2008 18:16:57 +0100 Subject: [ofa-general] How ot support LL QP for ehca Message-ID: Hello Roland, Or and Jack! Creating a new thread because we need besides the QP_CREATE_LL flag say two other flags like below: QP_CREATE_LL_SND_COMP means you want to get send completions QP_CREATE_LL_RCV_COMP means you want to get recv completions Basically LL QP works in the way that you put the payload directly into the queue instead specifying it via sges. While you always get recv completions for normal QP, you can suppress that for LL QP and check a certain pattern in the queue mem by yourself. That's the reason why we've those two flags when creating a LL QP. Above example is one option to add those flags in ib_verbs.h Another one is to enhance enum ib_sig_type with eg. IB_SIGNAL_SND_WR, IB_SIGNAL_RCV_WR, IB_SIGNAL_NONE /* last one could be encoded like IB_SIGNAL_REQ_WR */ Look at struct ib_init_qp_attr I see enum ib_sig_type sq_sig_type; That means it is thought for send only. Using sq_sig_type for LL QP without renaming the field name is misleading to me. The 3rd option is to introduce a new field like rq_sig_type. Thus, the first option would not break older libibverbs. The last two will. What is your preferred way? Other comments? Appreciate your help! Nam From a-annmr at adest.com Wed Mar 26 10:35:21 2008 From: a-annmr at adest.com (Santos Mccall) Date: Wed, 26 Mar 2008 18:35:21 +0100 Subject: [ofa-general] i am looking for you Message-ID: <01c88f70$2599ba80$2b67c34d@a-annmr> Hello! I am tired this afternoon. I am nice girl that would like to chat with you. Email me at Jenny at BestGolova.com only, because I am using my friend's email to write this. Hope you like my pictures. From hrosenstock at xsigo.com Wed Mar 26 11:07:27 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 26 Mar 2008 11:07:27 -0700 Subject: [ofa-general] Re: [PATCH 4/4] opensm: remove not used osm_log_printf() function In-Reply-To: <1206552426-17279-5-git-send-email-sashak@voltaire.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> <1206552426-17279-5-git-send-email-sashak@voltaire.com> Message-ID: <1206554847.8099.747.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-26 at 17:27 +0000, Sasha Khapyorsky wrote: > Remove not used anymore osm_log_printf() function from osm_log. Update > libopensm library version. Shouldn't this be deprecated for one library version prior to removing ? > Signed-off-by: Sasha Khapyorsky > --- > opensm/include/opensm/osm_log.h | 2 -- > opensm/opensm/libopensm.map | 1 - > opensm/opensm/libopensm.ver | 2 +- > opensm/opensm/osm_log.c | 19 ------------------- > 4 files changed, 1 insertions(+), 23 deletions(-) > > diff --git a/opensm/include/opensm/osm_log.h b/opensm/include/opensm/osm_log.h > index dd63dc4..48fb15e 100644 > --- a/opensm/include/opensm/osm_log.h > +++ b/opensm/include/opensm/osm_log.h > @@ -389,8 +389,6 @@ osm_log_is_active(IN const osm_log_t * const p_log, > * osm_log_destroy > *********/ > > -extern int osm_log_printf(osm_log_t * p_log, osm_log_level_t level, > - const char *fmt, ...) STRICT_OSM_LOG_FORMAT; > extern void osm_log_msg_box(osm_log_t *log, osm_log_level_t level, > const char *func_name, const char *msg); > extern void osm_log_raw(IN osm_log_t * const p_log, > diff --git a/opensm/opensm/libopensm.map b/opensm/opensm/libopensm.map > index 1d574bc..759a4e8 100644 > --- a/opensm/opensm/libopensm.map > +++ b/opensm/opensm/libopensm.map > @@ -1,7 +1,6 @@ > OPENSM_1.5 { > global: > osm_log; > - osm_log_printf; > osm_log_msg_box; > osm_is_debug; > osm_log_init; > diff --git a/opensm/opensm/libopensm.ver b/opensm/opensm/libopensm.ver > index c00a3a9..3324b1a 100644 > --- a/opensm/opensm/libopensm.ver > +++ b/opensm/opensm/libopensm.ver > @@ -6,4 +6,4 @@ > # API_REV - advance on any added API > # RUNNING_REV - advance any change to the vendor files > # AGE - number of backward versions the API still supports > -LIBVERSION=2:0:1 > +LIBVERSION=2:1:0 > diff --git a/opensm/opensm/osm_log.c b/opensm/opensm/osm_log.c > index b5b4bd9..727aabc 100644 > --- a/opensm/opensm/osm_log.c > +++ b/opensm/opensm/osm_log.c > @@ -106,25 +106,6 @@ static void truncate_log_file(osm_log_t * const p_log) > } > #endif /* ndef WIN32 */ > > -int osm_log_printf(osm_log_t * p_log, osm_log_level_t level, > - const char *fmt, ...) > -{ > - va_list args; > - int ret; > - > - if (!(p_log->level & level)) > - return 0; > - > - va_start(args, fmt); > - ret = vfprintf(stdout, fmt, args); > - va_end(args); > - > - if (p_log->flush || level & OSM_LOG_ERROR) > - fflush(stdout); > - > - return ret; > -} > - > void > osm_log(IN osm_log_t * const p_log, > IN const osm_log_level_t verbosity, IN const char *p_str, ...) From sashak at voltaire.com Wed Mar 26 11:37:42 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 26 Mar 2008 18:37:42 +0000 Subject: [ofa-general] Re: [PATCH 4/4] opensm: remove not used osm_log_printf() function In-Reply-To: <1206554847.8099.747.camel@hrosenstock-ws.xsigo.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> <1206552426-17279-5-git-send-email-sashak@voltaire.com> <1206554847.8099.747.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080326183742.GK13708@sashak.voltaire.com> On 11:07 Wed 26 Mar , Hal Rosenstock wrote: > On Wed, 2008-03-26 at 17:27 +0000, Sasha Khapyorsky wrote: > > Remove not used anymore osm_log_printf() function from osm_log. Update > > libopensm library version. > > Shouldn't this be deprecated for one library version prior to removing ? If nothing uses this then why? Sasha From 8irul at atec.co.th Wed Mar 26 09:37:35 2008 From: 8irul at atec.co.th (coleman clive) Date: Wed, 26 Mar 2008 16:37:35 +0000 Subject: [ofa-general] 08 Collection of Prada Gucci Dior Chanel & More Top Designer Shoes Message-ID: <000801c88f6e$064a2707$ceec3f90@dbglvl> Hey have you heard? Finally, the 2008 Collections are in, enjoy 70% OFF Brand Name Shoes & Boots for Men & Women from TOP Fashion Designers. Choose from a variety of the season's hottest models from Gucci, Prada, Chanel, Dior, Ugg Boots, Burberry, D&G, Dsquared & much more. Enter and Save TODAY! Free International Shipping on ALL ORDERS! Click here! Make your way here & Save Today! NoW That's an AMAZING Offer! -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrosenstock at xsigo.com Wed Mar 26 11:28:22 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Wed, 26 Mar 2008 11:28:22 -0700 Subject: [ofa-general] Re: [PATCH 4/4] opensm: remove not used osm_log_printf() function In-Reply-To: <20080326183742.GK13708@sashak.voltaire.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> <1206552426-17279-5-git-send-email-sashak@voltaire.com> <1206554847.8099.747.camel@hrosenstock-ws.xsigo.com> <20080326183742.GK13708@sashak.voltaire.com> Message-ID: <1206556102.8099.754.camel@hrosenstock-ws.xsigo.com> On Wed, 2008-03-26 at 18:37 +0000, Sasha Khapyorsky wrote: > On 11:07 Wed 26 Mar , Hal Rosenstock wrote: > > On Wed, 2008-03-26 at 17:27 +0000, Sasha Khapyorsky wrote: > > > Remove not used anymore osm_log_printf() function from osm_log. Update > > > libopensm library version. > > > > Shouldn't this be deprecated for one library version prior to removing ? > > If nothing uses this then why? Because it was published as a global API and something outside might be using it. It's not necessarily just for internal consumers. -- Hal From leo at ultra.su Wed Mar 26 10:40:53 2008 From: leo at ultra.su (kimmo hong) Date: Wed, 26 Mar 2008 17:40:53 +0000 Subject: [ofa-general] Perfectly crafted luxury timepieces Message-ID: <000a01c88f77$05df0c2c$cda0ddba@wnava> Prestige replicas Most popular watches Rolex Datejusts•Cartier•Hublot•Panerai•Rolex Sports Models•Chanel•Jacob & Co•Patek•Philippe•Alain Silberstein•Chopard•Jaeger•LeCoutrePorsche•Design Most popular TIFFANY & CO. JEWERLY Tiffany & CO Bracelets•Tiffany & CO Necklace•Tiffany & CO Earings Most popular PENS Mont Blanc Ballpoint•Louis Vuitton Rollerball•St Dupont Fountain•Mont Blanc Rollerball•Gucci Roller•St Dupont Ballpoint•Louis Vuitton Ballpoint•St Dupont Roller Click here -------------- next part -------------- An HTML attachment was scrubbed... URL: From suri at baymicrosystems.com Wed Mar 26 14:39:21 2008 From: suri at baymicrosystems.com (Suresh Shelvapille) Date: Wed, 26 Mar 2008 17:39:21 -0400 Subject: [ofa-general] new NFS/RDMA instructions for 2.6.25-rc1 In-Reply-To: References: Message-ID: <057f01c88f89$ddf28850$3414a8c0@md.baymicrosystems.com> Is it possible to get Server/Client running on RHEL5? > -----Original Message----- > From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf > Of James Lentini > Sent: Monday, February 11, 2008 12:25 PM > To: general at lists.openfabrics.org; linux-nfs at vger.kernel.org > Subject: [ofa-general] new NFS/RDMA instructions for 2.6.25-rc1 > > > Linux 2.6.25 will be the first official kernel release to contain the > NFS/RDMA server. With the client and server now both available in > 2.6.25-rc1, we've simplified our NFS/RDMA installation instructions. > The new instructions are available here: > > http://nfs-rdma.sourceforge.net/Documents/README > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From changquing.tang at hp.com Wed Mar 26 14:45:38 2008 From: changquing.tang at hp.com (Tang, Changqing) Date: Wed, 26 Mar 2008 21:45:38 +0000 Subject: [ofa-general] QP connection healthy detection problem with fork/exec Message-ID: Hi: I have a connection healthy detection problem, here is what I do. Rank 0 and Rank 1 setup a QP connection. Rank 0 is waiting a message from rank 1, during this time, Rank 0 periodically sends a heart-beat message back to Rank 1 to detect if the connection is OK, or if rank 1 has died. The heart-beat is a zero-byte RDMA message: sr.next = NULL; sr.wr_id = (uint64_t)(AULONG)rdmahdr; sr.sg_list = &ssg; sr.num_sge = 0; sr.opcode = IBV_WR_RDMA_WRITE; sr.send_flags = IBV_SEND_INLINE|IBV_SEND_SIGNALED; If this heart-beat message completes with success, I think, the connection is OK, and peer process is alive. However in Rank 1, fork() is called, and parent exit(), the child call sleep for 5 minutes. But in rank 0, The hear-beat message is always success untill I kill rank 2's child. Further, rank 1 calls fork() and exits, the child calls execl("/bin/sleep", "sleep", "300", (char *)0); In rank 0, the heart-beat is still success untill I kill the 'sleep' process. It is easy to understand that if only fork() is called, the child will hold QP resources from parent, rank 0 can NOT detect anything wrong. But if child calls exec, everything in rank 1 has been destroyed, why can't rank 0 detect the connection is broken ? Thanks for any help. --CQ Tang, HP-MPI From jjolly at novell.com Wed Mar 26 16:12:19 2008 From: jjolly at novell.com (John Jolly) Date: Wed, 26 Mar 2008 17:12:19 -0600 Subject: [ofa-general] Possibly proprietary license in dapl2 getopt.c and getopt.h Message-ID: <47EAD853.8090808@novell.com> All, Within the dapl2 package, there are two source files: dapl-2.0.7/test/dtest/GETOPT.C dapl-2.0.7/test/dtest/GETOPT.H were derived by Intel Corporation from public domain code originally owned by AT&T. Intel modified the code and licensed it under the license shown below - effectively a proprietary license which states that it is license "free of charge" but does _not_ give sufficient rights to actually copy/modify/distribute the code. However, we (SuSE/Novell) tried building the package without the files (i.e. added a rm in the spec file) and the package built successfully. Since they are not used at all, we (SuSE/Novell) have consider removing them completely from _source_ and _binary_ (in the same manner that we remove e.g. mp3 decoders from the tarball and the binary) as it represents an unnecessary business risk for which there is no perceivable business advantage to be gained. If this is a mistake in the copyright header - i.e. if you think Intel might have forgotten to update their copyrights, shouldn't these headers be removed? /*------------------------------------------------------------------------ ** ** Copyright (c) 1996, Intel Corp. All rights reserved. ** ** THIS IS UNPUBLISHED PROPRIETARY SOURCE CODE OF ** Intel Corp. ** The copyright notice above does not evidence any ** actual or intended publication of such source code. ** ** ** Module: getopt.c ** ** Source: PAL, Intel Corp. Mike Ripley ** ** Abstract: ** This is source for the standard getopt() library. ** Original source was obtained from the public domain, ** and modified to better suit DOS and Win32 environments. ** ** Environment: ** Microsoft Win32 ** Microsoft Visual C++ 4.0 --------------------------------------------------------------------- LIMITATION OF LIABILITY AND DISCLAIMER OF WARRANTY FOR SOFTWARE THE SOFTWARE THAT YOU ARE RECEIVING IS LICENSED FREE OF CHARGE. THE SOFTWARE IS PROVIDED "AS IS." INTEL MAKES NO WARRANTY OF ANY KIND REGARDING THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE STATE LAW. INTEL WILL NOT PROVIDE ANY INSTALLATION OR TECHNICAL SUPPORT, OR ANY OTHER KIND OF ASSISTANCE TO YOU REGARDING YOUR USE OF THE SOFTWARE. IF THE SOFTWARE PROVES DEFECTIVE ANYWAY, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. INTEL WILL NOT PROVIDE ANY UPDATES, ENHANCEMENTS OR EXTENSIONS TO THE SOFTWARE. ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ARE EXCLUDED, AND DO NOT APPLY TO THE SOFTWARE YOU ARE RECEIVING. IN ADDITION TO ANY OTHER DISCLAIMER SET FORTH HEREIN, INTEL WILL NOT HAVE ANY LIABILITY TO YOU FOR: a) any defects in the software you are receiving; or b) any inability by you to operate the software you are receiving to any performance specification; or c) any claim by you or any third party with respect to, or arising out of, the use of the software. INTEL DOES NOT MAKE ANY WARRANTIES OF ANY KIND THAT THE SOFTWARE DOES NOT OR WILL NOT INFRINGE ANY COPYRIGHT, PATENT, TRADE SECRET OR ANY OTHER INTELLECTUAL PROPERTY RIGHT OF ANY THIRD PARTY IN ANY COUNTRY. IN NO EVENT, SHALL INTEL BE LIABLE TO YOU OR ANY THIRD PARTY FOR ANY INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING BUT NOT LIMITED TO LOSS OF PROFITS, LOSS OF USE OF DATA, OR INTERRUPTION OF BUSINESS, EVEN IF ADVISED OF THE POSSIBILITIES OF SUCH DAMAGES. -- "If we knew what it was we were doing, it would not be called research, would it?" - Albert Einstein John L. Jolly SUSE Server Arch T:+1 801 861 1148 Mailstop PRV-H633 1800 South Novell Place Provo Utah 84606-6101 GPG 1024D/6CFB3E25 1935 9C20 B43B 43F3 26F3 CD80 74B5 A52F 6CFB 3E25 From Brian.Murrell at Sun.COM Wed Mar 26 16:38:18 2008 From: Brian.Murrell at Sun.COM (Brian J. Murrell) Date: Wed, 26 Mar 2008 19:38:18 -0400 Subject: [ofa-general] determine release version of available source Message-ID: <1206574698.10075.290.camel@pc.ilinx> Hello, How can I determine the release version of a given set of OFED sources? For example, looking in the SLES 10 2.6.16.54-0.2.3 source tree I see a drivers/infiniband directory as well as an include/rdma/ the contents of both being: drivers/infiniband/Makefile drivers/infiniband/Kconfig drivers/infiniband/core/uverbs_cmd.c drivers/infiniband/core/Makefile drivers/infiniband/core/agent.c drivers/infiniband/core/agent.h drivers/infiniband/core/cache.c drivers/infiniband/core/cm.c drivers/infiniband/core/cm_msgs.h drivers/infiniband/core/core_priv.h drivers/infiniband/core/device.c drivers/infiniband/core/fmr_pool.c drivers/infiniband/core/mad.c drivers/infiniband/core/mad_priv.h drivers/infiniband/core/mad_rmpp.c drivers/infiniband/core/mad_rmpp.h drivers/infiniband/core/packer.c drivers/infiniband/core/sa_query.c drivers/infiniband/core/smi.c drivers/infiniband/core/smi.h drivers/infiniband/core/sysfs.c drivers/infiniband/core/ucm.c drivers/infiniband/core/ud_header.c drivers/infiniband/core/user_mad.c drivers/infiniband/core/uverbs.h drivers/infiniband/core/uverbs_main.c drivers/infiniband/core/uverbs_mem.c drivers/infiniband/core/verbs.c drivers/infiniband/hw/mthca/Makefile drivers/infiniband/hw/mthca/Kconfig drivers/infiniband/hw/mthca/mthca_allocator.c drivers/infiniband/hw/mthca/mthca_av.c drivers/infiniband/hw/mthca/mthca_catas.c drivers/infiniband/hw/mthca/mthca_cmd.c drivers/infiniband/hw/mthca/mthca_cmd.h drivers/infiniband/hw/mthca/mthca_config_reg.h drivers/infiniband/hw/mthca/mthca_cq.c drivers/infiniband/hw/mthca/mthca_dev.h drivers/infiniband/hw/mthca/mthca_doorbell.h drivers/infiniband/hw/mthca/mthca_eq.c drivers/infiniband/hw/mthca/mthca_mad.c drivers/infiniband/hw/mthca/mthca_main.c drivers/infiniband/hw/mthca/mthca_mcg.c drivers/infiniband/hw/mthca/mthca_memfree.c drivers/infiniband/hw/mthca/mthca_memfree.h drivers/infiniband/hw/mthca/mthca_mr.c drivers/infiniband/hw/mthca/mthca_pd.c drivers/infiniband/hw/mthca/mthca_profile.c drivers/infiniband/hw/mthca/mthca_profile.h drivers/infiniband/hw/mthca/mthca_provider.c drivers/infiniband/hw/mthca/mthca_provider.h drivers/infiniband/hw/mthca/mthca_qp.c drivers/infiniband/hw/mthca/mthca_reset.c drivers/infiniband/hw/mthca/mthca_srq.c drivers/infiniband/hw/mthca/mthca_uar.c drivers/infiniband/hw/mthca/mthca_user.h drivers/infiniband/hw/mthca/mthca_wqe.h drivers/infiniband/ulp/ipoib/Makefile drivers/infiniband/ulp/ipoib/Kconfig drivers/infiniband/ulp/ipoib/ipoib_fs.c drivers/infiniband/ulp/ipoib/ipoib.h drivers/infiniband/ulp/ipoib/ipoib_main.c drivers/infiniband/ulp/ipoib/ipoib_ib.c drivers/infiniband/ulp/ipoib/ipoib_multicast.c drivers/infiniband/ulp/ipoib/ipoib_verbs.c drivers/infiniband/ulp/ipoib/ipoib_vlan.c drivers/infiniband/ulp/srp/ib_srp.c drivers/infiniband/ulp/srp/Kbuild drivers/infiniband/ulp/srp/Kconfig drivers/infiniband/ulp/srp/ib_srp.h include/rdma/ib_fmr_pool.h include/rdma/ib_cache.h include/rdma/ib_cm.h include/rdma/ib_user_cm.h include/rdma/ib_mad.h include/rdma/ib_pack.h include/rdma/ib_sa.h include/rdma/ib_smi.h include/rdma/ib_user_mad.h include/rdma/ib_user_verbs.h include/rdma/ib_verbs.h Is there anything or anyway in those files (or possibly others) to determine if this is OFED 1.0, 1.1, 1.2.x, or something else entirely? It's entirely acceptable to build a small test program with them even to assist. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From hhq at boutiquebeverages.com.au Wed Mar 26 17:05:51 2008 From: hhq at boutiquebeverages.com.au (Georgia Cervantes) Date: Thu, 27 Mar 2008 08:05:51 +0800 Subject: [ofa-general] Smart in bed games Message-ID: <01c88fe1$5f56d980$dc4a1779@hhq> Take her to love heaven Please look attached file and know MORE ABOUT THIS! -------------- next part -------------- A non-text attachment was scrubbed... Name: cvv2.zip Type: application/zip Size: 636 bytes Desc: not available URL: From sashak at voltaire.com Wed Mar 26 17:27:11 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 00:27:11 +0000 Subject: [ofa-general] [PATCH 0/6] opensm: node guids to ids map or Up/Down Message-ID: <1206577637-18226-1-git-send-email-sashak@voltaire.com> Hi, This adds possibility to redefine IDs which will be used by Up/Down algorithm to determine up or down path direction when from root node distance is equal. I found it very useful for routing optimizations with some topologies (specifically chassis to chassis connected clusters). Also it consolidates a few file parsers and adds another improvements. Sasha From sashak at voltaire.com Wed Mar 26 17:27:13 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 00:27:13 +0000 Subject: [ofa-general] [PATCH 2/6] opensm/updn: use parse_node_map() for root node guids file processing In-Reply-To: <1206577637-18226-1-git-send-email-sashak@voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> Message-ID: <1206577637-18226-3-git-send-email-sashak@voltaire.com> Also this converts cl_list to cl_qlist as root_guids storage. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_updn.c | 118 +++++++++++++++++++--------------------- 1 files changed, 55 insertions(+), 63 deletions(-) diff --git a/opensm/opensm/osm_ucast_updn.c b/opensm/opensm/osm_ucast_updn.c index cd89879..8a72a15 100644 --- a/opensm/opensm/osm_ucast_updn.c +++ b/opensm/opensm/osm_ucast_updn.c @@ -75,7 +75,7 @@ typedef struct _updn_input { typedef struct _updn { boolean_t auto_detect_root_nodes; updn_input_t updn_ucast_reg_inputs; - cl_list_t *p_root_nodes; + cl_qlist_t root_nodes_list; osm_opensm_t *p_osm; } updn_t; @@ -230,21 +230,14 @@ __updn_bfs_by_node(IN osm_log_t * p_log, **********************************************************************/ static void updn_destroy(IN updn_t * const p_updn) { - uint64_t *p_guid_list_item; - /* free the array of guids */ if (p_updn->updn_ucast_reg_inputs.guid_list) free(p_updn->updn_ucast_reg_inputs.guid_list); /* destroy the list of root nodes */ - if (p_updn->p_root_nodes) { - while ((p_guid_list_item = - cl_list_remove_head(p_updn->p_root_nodes))) - free(p_guid_list_item); - cl_list_remove_all(p_updn->p_root_nodes); - cl_list_destroy(p_updn->p_root_nodes); - free(p_updn->p_root_nodes); - } + while (!cl_is_qlist_empty(&p_updn->root_nodes_list)) + free(cl_qlist_remove_head(&p_updn->root_nodes_list)); + free(p_updn); } @@ -266,24 +259,38 @@ static updn_t *updn_construct(osm_log_t * p_log) /********************************************************************** **********************************************************************/ +struct guid_list_item { + cl_list_item_t list; + uint64_t guid; +}; + +static int add_guid_item_to_list(void *cxt, uint64_t guid, char *p) +{ + updn_t *updn = cxt; + struct guid_list_item *item; + + item = malloc(sizeof(*item)); + if (!item) + return -1; + + item->guid = guid; + cl_qlist_insert_tail(&updn->root_nodes_list, &item->list); + + OSM_LOG(&updn->p_osm->log, OSM_LOG_DEBUG, + "Inserting GUID 0x%" PRIx64 " as root node\n", guid); + + return 0; +} + static cl_status_t updn_init(IN updn_t * const p_updn, IN osm_opensm_t * p_osm) { - cl_list_t *p_list; - cl_list_iterator_t guid_iterator; ib_api_status_t status = IB_SUCCESS; OSM_LOG_ENTER(&p_osm->log); p_updn->p_osm = p_osm; - p_list = (cl_list_t *) malloc(sizeof(cl_list_t)); - if (!p_list) { - status = IB_ERROR; - goto Exit; - } - cl_list_construct(p_list); - cl_list_init(p_list, 10); - p_updn->p_root_nodes = p_list; + cl_qlist_init(&p_updn->root_nodes_list); p_updn->updn_ucast_reg_inputs.num_guids = 0; p_updn->updn_ucast_reg_inputs.guid_list = NULL; p_updn->auto_detect_root_nodes = FALSE; @@ -293,27 +300,20 @@ static cl_status_t updn_init(IN updn_t * const p_updn, IN osm_opensm_t * p_osm) wait for a callback to activate auto detection */ if (p_osm->subn.opt.root_guid_file) { - status = osm_ucast_mgr_read_guid_file(&p_osm->sm.ucast_mgr, - p_osm->subn.opt. - root_guid_file, - p_updn->p_root_nodes); - if (status != IB_SUCCESS) - goto Exit; - /* For Debug Purposes ... */ OSM_LOG(&p_osm->log, OSM_LOG_DEBUG, "UPDN - Fetching root nodes from file %s\n", p_osm->subn.opt.root_guid_file); - guid_iterator = cl_list_head(p_updn->p_root_nodes); - while (guid_iterator != cl_list_end(p_updn->p_root_nodes)) { - OSM_LOG(&p_osm->log, OSM_LOG_DEBUG, - "Inserting GUID 0x%" PRIx64 " as root node\n", - *((uint64_t *) cl_list_obj(guid_iterator))); - guid_iterator = cl_list_next(guid_iterator); + + if (parse_node_map(p_osm->subn.opt.root_guid_file, + add_guid_item_to_list, p_updn)) { + OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR : " + "cannot parse root guids file \'%s\'\n", + p_osm->subn.opt.root_guid_file); + goto Exit; } - } else { + } else p_updn->auto_detect_root_nodes = TRUE; - } /* If auto mode detection required - will be executed in main b4 the assignment of UI Ucast */ Exit: @@ -619,13 +619,12 @@ static int __osm_updn_call(void *ctx) /* UPDN convert cl_list to guid array in updn struct */ static void __osm_updn_convert_list2array(IN updn_t * p_updn) { - uint32_t i = 0, max_num = 0; - uint64_t *p_guid; + unsigned i; OSM_LOG_ENTER(&p_updn->p_osm->log); p_updn->updn_ucast_reg_inputs.num_guids = - cl_list_count(p_updn->p_root_nodes); + cl_qlist_count(&p_updn->root_nodes_list); if (p_updn->updn_ucast_reg_inputs.guid_list) free(p_updn->updn_ucast_reg_inputs.guid_list); p_updn->updn_ucast_reg_inputs.guid_list = @@ -635,19 +634,14 @@ static void __osm_updn_convert_list2array(IN updn_t * p_updn) memset(p_updn->updn_ucast_reg_inputs.guid_list, 0, p_updn->updn_ucast_reg_inputs.num_guids * sizeof(uint64_t)); - if (!cl_is_list_empty(p_updn->p_root_nodes)) { - while ((p_guid = - (uint64_t *) cl_list_remove_head(p_updn-> - p_root_nodes))) { - p_updn->updn_ucast_reg_inputs.guid_list[i] = *p_guid; - free(p_guid); - i++; - } - max_num = i; - for (i = 0; i < max_num; i++) - OSM_LOG(&p_updn->p_osm->log, OSM_LOG_DEBUG, - "Map GUID 0x%" PRIx64 " into UPDN array\n", - p_updn->updn_ucast_reg_inputs.guid_list[i]); + for (i = 0; !cl_is_qlist_empty(&p_updn->root_nodes_list); i++) { + struct guid_list_item *item; + item = (void *)cl_qlist_remove_head(&p_updn->root_nodes_list); + p_updn->updn_ucast_reg_inputs.guid_list[i] = item->guid; + OSM_LOG(&p_updn->p_osm->log, OSM_LOG_DEBUG, + "Map GUID 0x%" PRIx64 " into UPDN array\n", + p_updn->updn_ucast_reg_inputs.guid_list[i]); + free(item); } /* Since we need the template list for other sweeps, we wont destroy & free it */ OSM_LOG_EXIT(&p_updn->p_osm->log); @@ -662,8 +656,6 @@ static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn) osm_switch_t *p_next_sw, *p_sw; osm_port_t *p_next_port, *p_port; osm_physp_t *p_physp; - uint64_t *p_guid; - cl_list_t *p_root_nodes_list = p_updn->p_root_nodes; double thd1, thd2; unsigned i, cas_num = 0; unsigned *cas_per_sw; @@ -757,19 +749,19 @@ static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn) /* If thd conditions are valid insert the root node to the list */ if ((numHopBarsOverThd1 == 1) && (numHopBarsOverThd2 == 1)) { - p_guid = malloc(sizeof(uint64_t)); - if (p_guid) { - *p_guid = - cl_ntoh64(osm_node_get_node_guid - (p_sw->p_node)); + struct guid_list_item *item; + item = malloc(sizeof(*item)); + if (item) { + item->guid = cl_ntoh64(osm_node_get_node_guid + (p_sw->p_node)); OSM_LOG(&p_osm->log, OSM_LOG_DEBUG, "Inserting GUID 0x%" PRIx64 - " as root node\n", *p_guid); - cl_list_insert_tail(p_root_nodes_list, p_guid); - } else { + " as root node\n", item->guid); + cl_qlist_insert_tail(&p_updn->root_nodes_list, + &item->list); + } else OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR AA13: " "No memory for p_guid\n"); - } } } -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 17:27:12 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 00:27:12 +0000 Subject: [ofa-general] [PATCH 1/6] complib/nodenamemap: add generic parse_node_map() function In-Reply-To: <1206577637-18226-1-git-send-email-sashak@voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> Message-ID: <1206577637-18226-2-git-send-email-sashak@voltaire.com> This parser is able to parse text file with format similar to nodenamemap file: guid # comment .... It will be used later for unification parsing of root nodes and compute nodes guid list files. Signed-off-by: Sasha Khapyorsky --- opensm/complib/cl_nodenamemap.c | 45 +++++++++++++++++++++++++++++++ opensm/complib/libosmcomp.map | 1 + opensm/include/complib/cl_nodenamemap.h | 3 ++ 3 files changed, 49 insertions(+), 0 deletions(-) diff --git a/opensm/complib/cl_nodenamemap.c b/opensm/complib/cl_nodenamemap.c index af5da39..696d455 100644 --- a/opensm/complib/cl_nodenamemap.c +++ b/opensm/complib/cl_nodenamemap.c @@ -160,3 +160,48 @@ clean_nodedesc(char *nodedesc) return (nodedesc); } + +int parse_node_map(const char *file_name, + int (*create)(void *, uint64_t, char *), void *cxt) +{ + char line[256]; + FILE *f; + + if (!(f = fopen(file_name, "r"))) + return -1; + + while (fgets(line, sizeof(line),f)) { + uint64_t guid; + char *p, *e; + + p = line; + while (isspace(*p)) + p++; + if (*p == '\0' || *p == '\n' || *p == '#') + continue; + + guid = strtoull(p, &e, 0); + if (e == p || !isspace(*e)) { + fclose(f); + return -1; + } + + p = e; + if (*e) + e++; + while (isspace(*p)) + p++; + + e = strpbrk(p, "# \t\n"); + if (e) + *e = '\0'; + + if (create(cxt, guid, p)) { + fclose(f); + return -1; + } + } + + fclose(f); + return 0; +} diff --git a/opensm/complib/libosmcomp.map b/opensm/complib/libosmcomp.map index d0e107e..788eb2a 100644 --- a/opensm/complib/libosmcomp.map +++ b/opensm/complib/libosmcomp.map @@ -147,6 +147,7 @@ OSMCOMP_2.3 { ib_wc_status_str; open_node_name_map; close_node_name_map; + parse_node_map; remap_node_name; clean_nodedesc; local: *; diff --git a/opensm/include/complib/cl_nodenamemap.h b/opensm/include/complib/cl_nodenamemap.h index b20ca4e..5f4a026 100644 --- a/opensm/include/complib/cl_nodenamemap.h +++ b/opensm/include/complib/cl_nodenamemap.h @@ -1,4 +1,5 @@ /* + * Copyright (c) 2008 Voltaire, Inc. All rights reserved. * Copyright (c) 2007 Lawrence Livermore National Lab * * This software is available to you under a choice of one of two @@ -60,5 +61,7 @@ void close_node_name_map(nn_map_t *map); char *remap_node_name(nn_map_t *map, uint64_t target_guid, char *nodedesc); /* NOTE: parameter "nodedesc" may be modified here. */ +int parse_node_map(const char *file_name, + int (*create)(void *, uint64_t, char *), void *cxt); #endif /* _CL_NODE_NAME_MAP_H_ */ -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 17:27:14 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 00:27:14 +0000 Subject: [ofa-general] [PATCH 3/6] opensm/updn: update root nodes at each run In-Reply-To: <1206577637-18226-1-git-send-email-sashak@voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> Message-ID: <1206577637-18226-4-git-send-email-sashak@voltaire.com> Update root nodes at each run, so restarting OpenSM is not needed when Up/Down powered fabric topology is altered and some new nodes are addded. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_updn.c | 331 ++++++++++++---------------------------- 1 files changed, 95 insertions(+), 236 deletions(-) diff --git a/opensm/opensm/osm_ucast_updn.c b/opensm/opensm/osm_ucast_updn.c index 8a72a15..3623a69 100644 --- a/opensm/opensm/osm_ucast_updn.c +++ b/opensm/opensm/osm_ucast_updn.c @@ -65,17 +65,9 @@ typedef enum _updn_switch_dir { DOWN } updn_switch_dir_t; -/* guids list */ -typedef struct _updn_input { - uint32_t num_guids; - uint64_t *guid_list; -} updn_input_t; - /* updn structure */ typedef struct _updn { - boolean_t auto_detect_root_nodes; - updn_input_t updn_ucast_reg_inputs; - cl_qlist_t root_nodes_list; + unsigned num_roots; osm_opensm_t *p_osm; } updn_t; @@ -228,139 +220,30 @@ __updn_bfs_by_node(IN osm_log_t * p_log, /********************************************************************** **********************************************************************/ -static void updn_destroy(IN updn_t * const p_updn) -{ - /* free the array of guids */ - if (p_updn->updn_ucast_reg_inputs.guid_list) - free(p_updn->updn_ucast_reg_inputs.guid_list); - - /* destroy the list of root nodes */ - while (!cl_is_qlist_empty(&p_updn->root_nodes_list)) - free(cl_qlist_remove_head(&p_updn->root_nodes_list)); - - free(p_updn); -} - -/********************************************************************** - **********************************************************************/ -static updn_t *updn_construct(osm_log_t * p_log) -{ - updn_t *p_updn; - - OSM_LOG_ENTER(p_log); - - p_updn = malloc(sizeof(updn_t)); - if (p_updn) - memset(p_updn, 0, sizeof(updn_t)); - - OSM_LOG_EXIT(p_log); - return (p_updn); -} - -/********************************************************************** - **********************************************************************/ -struct guid_list_item { - cl_list_item_t list; - uint64_t guid; -}; - -static int add_guid_item_to_list(void *cxt, uint64_t guid, char *p) -{ - updn_t *updn = cxt; - struct guid_list_item *item; - - item = malloc(sizeof(*item)); - if (!item) - return -1; - - item->guid = guid; - cl_qlist_insert_tail(&updn->root_nodes_list, &item->list); - - OSM_LOG(&updn->p_osm->log, OSM_LOG_DEBUG, - "Inserting GUID 0x%" PRIx64 " as root node\n", guid); - - return 0; -} - -static cl_status_t updn_init(IN updn_t * const p_updn, IN osm_opensm_t * p_osm) -{ - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER(&p_osm->log); - - p_updn->p_osm = p_osm; - - cl_qlist_init(&p_updn->root_nodes_list); - p_updn->updn_ucast_reg_inputs.num_guids = 0; - p_updn->updn_ucast_reg_inputs.guid_list = NULL; - p_updn->auto_detect_root_nodes = FALSE; - - /* - Check the source for root node list, if file parse it, otherwise - wait for a callback to activate auto detection - */ - if (p_osm->subn.opt.root_guid_file) { - /* For Debug Purposes ... */ - OSM_LOG(&p_osm->log, OSM_LOG_DEBUG, - "UPDN - Fetching root nodes from file %s\n", - p_osm->subn.opt.root_guid_file); - - if (parse_node_map(p_osm->subn.opt.root_guid_file, - add_guid_item_to_list, p_updn)) { - OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR : " - "cannot parse root guids file \'%s\'\n", - p_osm->subn.opt.root_guid_file); - goto Exit; - } - } else - p_updn->auto_detect_root_nodes = TRUE; - /* If auto mode detection required - will be executed in main b4 the assignment of UI Ucast */ - -Exit: - OSM_LOG_EXIT(&p_osm->log); - return (status); -} - -/********************************************************************** - **********************************************************************/ /* NOTE : PLS check if we need to decide that the first */ /* rank is a SWITCH for BFS purpose */ -static int -updn_subn_rank(IN unsigned num_guids, - IN uint64_t * guid_list, IN updn_t * p_updn) +static int updn_subn_rank(IN updn_t * p_updn) { osm_switch_t *p_sw; osm_physp_t *p_physp, *p_remote_physp; cl_qlist_t list; + cl_map_item_t *item; struct updn_node *u, *remote_u; uint8_t num_ports, port_num; osm_log_t *p_log = &p_updn->p_osm->log; - unsigned idx = 0; unsigned max_rank = 0; OSM_LOG_ENTER(p_log); cl_qlist_init(&list); - /* Rank all the roots and add them to list */ - - for (idx = 0; idx < num_guids; idx++) { - /* Apply the ranking for each guid given by user - bypass illegal ones */ - p_sw = - osm_get_switch_by_guid(&p_updn->p_osm->subn, - cl_hton64(guid_list[idx])); - if (!p_sw) { - OSM_LOG(p_log, OSM_LOG_ERROR, "ERR AA05: " - "Root switch GUID 0x%" PRIx64 " not found\n", - guid_list[idx]); - continue; - } - + /* add all roots to the list */ + for (item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); + item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl); + item = cl_qmap_next(item)) { + p_sw = (osm_switch_t *)item; u = p_sw->priv; - OSM_LOG(p_log, OSM_LOG_DEBUG, - "Ranking root port GUID 0x%" PRIx64 "\n", - guid_list[idx]); - u->rank = 0; - cl_qlist_insert_tail(&list, &u->list); + if (!u->rank) + cl_qlist_insert_tail(&list, &u->list); } /* BFS the list till it's empty */ @@ -440,7 +323,8 @@ static int __osm_subn_set_up_down_min_hop_table(IN updn_t * p_updn) { osm_subn_t *p_subn = &p_updn->p_osm->subn; osm_log_t *p_log = &p_updn->p_osm->log; - osm_switch_t *p_next_sw, *p_sw; + osm_switch_t *p_sw; + cl_map_item_t *item; OSM_LOG_ENTER(p_log); @@ -449,10 +333,10 @@ static int __osm_subn_set_up_down_min_hop_table(IN updn_t * p_updn) OSM_LOG(p_log, OSM_LOG_VERBOSE, "Init Min Hop Table of all switches [\n"); - p_next_sw = (osm_switch_t *) cl_qmap_head(&p_subn->sw_guid_tbl); - while (p_next_sw != (osm_switch_t *) cl_qmap_end(&p_subn->sw_guid_tbl)) { - p_sw = p_next_sw; - p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + for (item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); + item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl); + item = cl_qmap_next(item)) { + p_sw = (osm_switch_t *)item; /* Clear Min Hop Table */ if (p_subn->opt.connect_roots) updn_clear_root_hops(p_updn, p_sw); @@ -467,10 +351,10 @@ static int __osm_subn_set_up_down_min_hop_table(IN updn_t * p_updn) OSM_LOG(p_log, OSM_LOG_VERBOSE, "BFS through all port guids in the subnet [\n"); - p_next_sw = (osm_switch_t *) cl_qmap_head(&p_subn->sw_guid_tbl); - while (p_next_sw != (osm_switch_t *) cl_qmap_end(&p_subn->sw_guid_tbl)) { - p_sw = p_next_sw; - p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + for (item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); + item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl); + item = cl_qmap_next(item)) { + p_sw = (osm_switch_t *)item; __updn_bfs_by_node(p_log, p_subn, p_sw); } @@ -483,9 +367,7 @@ static int __osm_subn_set_up_down_min_hop_table(IN updn_t * p_updn) /********************************************************************** **********************************************************************/ -static int -updn_build_lid_matrices(IN uint32_t num_guids, - IN uint64_t * guid_list, IN updn_t * p_updn) +static int updn_build_lid_matrices(IN updn_t * p_updn) { int status; @@ -493,7 +375,7 @@ updn_build_lid_matrices(IN uint32_t num_guids, OSM_LOG(&p_updn->p_osm->log, OSM_LOG_VERBOSE, "Ranking all port guids in the list\n"); - if (num_guids == 0) { + if (!p_updn->num_roots) { OSM_LOG(&p_updn->p_osm->log, OSM_LOG_ERROR, "ERR AA0A: " "No guids were provided or number of guids is 0\n"); status = -1; @@ -509,7 +391,7 @@ updn_build_lid_matrices(IN uint32_t num_guids, } /* Rank the subnet switches */ - updn_subn_rank(num_guids, guid_list, p_updn); + updn_subn_rank(p_updn); /* After multiple ranking need to set Min Hop Table by UpDn algorithm */ OSM_LOG(&p_updn->p_osm->log, OSM_LOG_VERBOSE, @@ -552,20 +434,41 @@ static void dump_roots(cl_map_item_t *item, FILE *file, void *cxt) cl_ntoh64(osm_node_get_node_guid(sw->p_node))); } +static int rank_root_node(void *cxt, uint64_t guid, char *p) +{ + updn_t *updn = cxt; + osm_switch_t *sw; + + sw = osm_get_switch_by_guid(&updn->p_osm->subn, cl_hton64(guid)); + if (!sw) { + OSM_LOG(&updn->p_osm->log, OSM_LOG_VERBOSE, + "switch with guid 0x%" PRIx64 " is not found\n", guid); + return 0; + } + + OSM_LOG(&updn->p_osm->log, OSM_LOG_DEBUG, + "Ranking root port GUID 0x%" PRIx64 "\n", guid); + + ((struct updn_node *)sw->priv)->rank = 0; + updn->num_roots++; + + return 0; +} + /* UPDN callback function */ static int __osm_updn_call(void *ctx) { updn_t *p_updn = ctx; - cl_map_item_t *p_item; + cl_map_item_t *item; osm_switch_t *p_sw; int ret = 0; OSM_LOG_ENTER(&p_updn->p_osm->log); - p_item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); - while (p_item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl)) { - p_sw = (osm_switch_t *) p_item; - p_item = cl_qmap_next(p_item); + for (item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); + item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl); + item = cl_qmap_next(item)) { + p_sw = (osm_switch_t *)item; p_sw->priv = create_updn_node(p_sw); if (!p_sw->priv) { OSM_LOG(&(p_updn->p_osm->log), OSM_LOG_ERROR, "ERR AA0C: " @@ -575,23 +478,33 @@ static int __osm_updn_call(void *ctx) } } - /* First auto detect root nodes - if required */ - if (p_updn->auto_detect_root_nodes) { + /* First setup root nodes */ + p_updn->num_roots = 0; + + if (p_updn->p_osm->subn.opt.root_guid_file) { + OSM_LOG(&p_updn->p_osm->log, OSM_LOG_DEBUG, + "UPDN - Fetching root nodes from file %s\n", + p_updn->p_osm->subn.opt.root_guid_file); + + ret = parse_node_map(p_updn->p_osm->subn.opt.root_guid_file, + rank_root_node, p_updn); + if (ret) + OSM_LOG(&p_updn->p_osm->log, OSM_LOG_ERROR, "ERR : " + "cannot parse root guids file \'%s\'\n", + p_updn->p_osm->subn.opt.root_guid_file); + if (p_updn->p_osm->subn.opt.connect_roots && + p_updn->num_roots > 1) + osm_ucast_mgr_build_lid_matrices(&p_updn->p_osm->sm.ucast_mgr); + } else { osm_ucast_mgr_build_lid_matrices(&p_updn->p_osm->sm.ucast_mgr); __osm_updn_find_root_nodes_by_min_hop(p_updn); - } else if (p_updn->p_osm->subn.opt.connect_roots && - p_updn->updn_ucast_reg_inputs.num_guids > 1) - osm_ucast_mgr_build_lid_matrices(&p_updn->p_osm->sm.ucast_mgr); + } - /* printf ("-V- after osm_updn_find_root_nodes_by_min_hop\n"); */ /* Only if there are assigned root nodes do the algorithm, otherwise perform do nothing */ - if (p_updn->updn_ucast_reg_inputs.num_guids > 0) { + if (p_updn->num_roots) { OSM_LOG(&p_updn->p_osm->log, OSM_LOG_DEBUG, "activating UPDN algorithm\n"); - ret = updn_build_lid_matrices(p_updn->updn_ucast_reg_inputs. - num_guids, - p_updn->updn_ucast_reg_inputs. - guid_list, p_updn); + ret = updn_build_lid_matrices(p_updn); } else { OSM_LOG(&p_updn->p_osm->log, OSM_LOG_INFO, "disabling UPDN algorithm, no root nodes were found\n"); @@ -603,10 +516,10 @@ static int __osm_updn_call(void *ctx) &p_updn->p_osm->subn.sw_guid_tbl, dump_roots, NULL); - p_item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); - while (p_item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl)) { - p_sw = (osm_switch_t *) p_item; - p_item = cl_qmap_next(p_item); + for (item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); + item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl); + item = cl_qmap_next(item)) { + p_sw = (osm_switch_t *) item; delete_updn_node(p_sw->priv); } @@ -616,46 +529,14 @@ static int __osm_updn_call(void *ctx) /********************************************************************** **********************************************************************/ -/* UPDN convert cl_list to guid array in updn struct */ -static void __osm_updn_convert_list2array(IN updn_t * p_updn) -{ - unsigned i; - - OSM_LOG_ENTER(&p_updn->p_osm->log); - - p_updn->updn_ucast_reg_inputs.num_guids = - cl_qlist_count(&p_updn->root_nodes_list); - if (p_updn->updn_ucast_reg_inputs.guid_list) - free(p_updn->updn_ucast_reg_inputs.guid_list); - p_updn->updn_ucast_reg_inputs.guid_list = - (uint64_t *) malloc(p_updn->updn_ucast_reg_inputs.num_guids * - sizeof(uint64_t)); - if (p_updn->updn_ucast_reg_inputs.guid_list) - memset(p_updn->updn_ucast_reg_inputs.guid_list, 0, - p_updn->updn_ucast_reg_inputs.num_guids * - sizeof(uint64_t)); - for (i = 0; !cl_is_qlist_empty(&p_updn->root_nodes_list); i++) { - struct guid_list_item *item; - item = (void *)cl_qlist_remove_head(&p_updn->root_nodes_list); - p_updn->updn_ucast_reg_inputs.guid_list[i] = item->guid; - OSM_LOG(&p_updn->p_osm->log, OSM_LOG_DEBUG, - "Map GUID 0x%" PRIx64 " into UPDN array\n", - p_updn->updn_ucast_reg_inputs.guid_list[i]); - free(item); - } - /* Since we need the template list for other sweeps, we wont destroy & free it */ - OSM_LOG_EXIT(&p_updn->p_osm->log); -} - -/********************************************************************** - **********************************************************************/ /* Find Root nodes automatically by Min Hop Table info */ static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn) { osm_opensm_t *p_osm = p_updn->p_osm; - osm_switch_t *p_next_sw, *p_sw; - osm_port_t *p_next_port, *p_port; + osm_switch_t *p_sw; + osm_port_t *p_port; osm_physp_t *p_physp; + cl_map_item_t *item; double thd1, thd2; unsigned i, cas_num = 0; unsigned *cas_per_sw; @@ -678,12 +559,10 @@ static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn) /* Find the Maximum number of CAs (and routers) for histogram normalization */ OSM_LOG(&p_osm->log, OSM_LOG_VERBOSE, "Finding the number of CAs and storing them in cl_map\n"); - p_next_port = (osm_port_t *) cl_qmap_head(&p_osm->subn.port_guid_tbl); - while (p_next_port != - (osm_port_t *) cl_qmap_end(&p_osm->subn.port_guid_tbl)) { - p_port = p_next_port; - p_next_port = - (osm_port_t *) cl_qmap_next(&p_next_port->map_item); + for (item = cl_qmap_head(&p_updn->p_osm->subn.port_guid_tbl); + item != cl_qmap_end(&p_updn->p_osm->subn.port_guid_tbl); + item = cl_qmap_next(item)) { + p_port = (osm_port_t *)item; if (!p_port->p_node->sw) { p_physp = p_port->p_physp->p_remote_physp; if (!p_physp || !p_physp->p_node->sw) @@ -706,20 +585,18 @@ static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn) "Thresholds are thd1 = %f && thd2 = %f\n", cas_num, cl_qmap_count(&p_osm->subn.sw_guid_tbl), thd1, thd2); - p_next_sw = (osm_switch_t *) cl_qmap_head(&p_osm->subn.sw_guid_tbl); OSM_LOG(&p_osm->log, OSM_LOG_VERBOSE, "Passing through all switches to collect Min Hop info\n"); - while (p_next_sw != - (osm_switch_t *) cl_qmap_end(&p_osm->subn.sw_guid_tbl)) { + for (item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); + item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl); + item = cl_qmap_next(item)) { unsigned hop_hist[IB_SUBNET_PATH_HOPS_MAX]; uint16_t max_lid_ho; uint8_t hop_val; uint16_t numHopBarsOverThd1 = 0; uint16_t numHopBarsOverThd2 = 0; - p_sw = p_next_sw; - /* Roll to the next switch */ - p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + p_sw = (osm_switch_t *) item; memset(hop_hist, 0, sizeof(hop_hist)); @@ -747,28 +624,17 @@ static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn) numHopBarsOverThd2++; } - /* If thd conditions are valid insert the root node to the list */ + /* If thd conditions are valid - rank the root node */ if ((numHopBarsOverThd1 == 1) && (numHopBarsOverThd2 == 1)) { - struct guid_list_item *item; - item = malloc(sizeof(*item)); - if (item) { - item->guid = cl_ntoh64(osm_node_get_node_guid - (p_sw->p_node)); - OSM_LOG(&p_osm->log, OSM_LOG_DEBUG, - "Inserting GUID 0x%" PRIx64 - " as root node\n", item->guid); - cl_qlist_insert_tail(&p_updn->root_nodes_list, - &item->list); - } else - OSM_LOG(&p_osm->log, OSM_LOG_ERROR, "ERR AA13: " - "No memory for p_guid\n"); + OSM_LOG(&p_osm->log, OSM_LOG_DEBUG, + "Ranking GUID 0x%" PRIx64 " as root node\n", + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node))); + ((struct updn_node *)p_sw->priv)->rank = 0; + p_updn->num_roots++; } } free(cas_per_sw); - /* Now convert the cl_list to array */ - __osm_updn_convert_list2array(p_updn); - _exit: OSM_LOG_EXIT(&p_osm->log); return; @@ -778,30 +644,23 @@ _exit: **********************************************************************/ static void __osm_updn_delete(void *context) { - updn_t *p_updn = context; - - updn_destroy(p_updn); + free(context); } int osm_ucast_updn_setup(osm_opensm_t * p_osm) { updn_t *p_updn; - p_updn = updn_construct(&p_osm->log); + p_updn = malloc(sizeof(updn_t)); if (!p_updn) return -1; + memset(p_updn, 0, sizeof(updn_t)); - if (updn_init(p_updn, p_osm) != IB_SUCCESS) { - updn_destroy(p_updn); - return -1; - } + p_updn->p_osm = p_osm; p_osm->routing_engine.context = p_updn; p_osm->routing_engine.delete = __osm_updn_delete; p_osm->routing_engine.build_lid_matrices = __osm_updn_call; - if (!p_updn->auto_detect_root_nodes) - __osm_updn_convert_list2array(p_updn); - return 0; } -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 17:27:15 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 00:27:15 +0000 Subject: [ofa-general] [PATCH 4/6] opensm/ftree: use parse_node_map() for guids file processing In-Reply-To: <1206577637-18226-1-git-send-email-sashak@voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> Message-ID: <1206577637-18226-5-git-send-email-sashak@voltaire.com> Also it converts cl_list to cl_qlist as root_guids storage, reuses node_map_t and removes some intermediate stuff. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_ucast_ftree.c | 173 ++++++++++++++------------------------- 1 files changed, 61 insertions(+), 112 deletions(-) diff --git a/opensm/opensm/osm_ucast_ftree.c b/opensm/opensm/osm_ucast_ftree.c index c3759ca..36d0e0e 100644 --- a/opensm/opensm/osm_ucast_ftree.c +++ b/opensm/opensm/osm_ucast_ftree.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004-2007 Voltaire, Inc. All rights reserved. + * Copyright (c) 2004-2008 Voltaire, Inc. All rights reserved. * Copyright (c) 2002-2007 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * @@ -105,6 +105,11 @@ struct ftree_fabric_t_; typedef uint8_t ftree_tuple_t[FTREE_TUPLE_LEN]; typedef uint64_t ftree_tuple_key_t; +struct guid_list_item { + cl_list_item_t list; + uint64_t guid; +}; + /*************************************************** ** ** ftree_sw_table_element_t definition @@ -118,17 +123,6 @@ typedef struct { /*************************************************** ** - ** ftree_guid_tbl_element_t definition - ** - ***************************************************/ - -typedef struct { - cl_map_item_t map_item; - uint64_t guid_ho; -} ftree_guid_tbl_element_t; - -/*************************************************** - ** ** ftree_fwd_tbl_t definition ** ***************************************************/ @@ -224,7 +218,7 @@ typedef struct ftree_fabric_t_ { cl_qmap_t hca_tbl; cl_qmap_t sw_tbl; cl_qmap_t sw_by_tuple_tbl; - cl_list_t root_guid_list; + cl_qlist_t root_guid_list; cl_qmap_t cn_guid_tbl; unsigned cn_num; uint8_t leaf_switch_rank; @@ -372,35 +366,6 @@ static void __osm_ftree_sw_tbl_element_destroy(IN ftree_sw_tbl_element_t * /*************************************************** ** - ** ftree_guid_tbl_element_t functions - ** - ***************************************************/ - -static ftree_guid_tbl_element_t *__osm_ftree_guid_tbl_element_create(IN uint64_t - guid) -{ - ftree_guid_tbl_element_t *p_element = (ftree_guid_tbl_element_t *) - malloc(sizeof(ftree_guid_tbl_element_t)); - if (!p_element) - return NULL; - memset(p_element, 0, sizeof(ftree_guid_tbl_element_t)); - - memcpy(&p_element->guid_ho, &guid, sizeof(uint64_t)); - return p_element; -} - -/***************************************************/ - -static void __osm_ftree_guid_tbl_element_destroy(IN ftree_guid_tbl_element_t * - p_element) -{ - if (!p_element) - return; - free(p_element); -} - -/*************************************************** - ** ** ftree_port_t functions ** ***************************************************/ @@ -962,8 +927,7 @@ static ftree_fabric_t *__osm_ftree_fabric_create() cl_qmap_init(&p_ftree->sw_by_tuple_tbl); cl_qmap_init(&p_ftree->cn_guid_tbl); - cl_list_construct(&p_ftree->root_guid_list); - cl_list_init(&p_ftree->root_guid_list, 10); + cl_qlist_init(&p_ftree->root_guid_list); status = cl_pool_init(&p_ftree->sw_fwd_tbl_pool, 8, /* min pool size */ 0, /* max pool size - unlimited */ @@ -988,9 +952,7 @@ static void __osm_ftree_fabric_clear(ftree_fabric_t * p_ftree) ftree_sw_t *p_next_sw; ftree_sw_tbl_element_t *p_element; ftree_sw_tbl_element_t *p_next_element; - ftree_guid_tbl_element_t *p_guid_element; - ftree_guid_tbl_element_t *p_next_guid_element; - uint64_t *p_guid; + name_map_item_t *p_guid_element, *p_next_guid_element; if (!p_ftree) return; @@ -1031,25 +993,20 @@ static void __osm_ftree_fabric_clear(ftree_fabric_t * p_ftree) cl_qmap_remove_all(&p_ftree->sw_by_tuple_tbl); /* remove all the elements of cn_guid_tbl */ - p_next_guid_element = - (ftree_guid_tbl_element_t *) cl_qmap_head(&p_ftree->cn_guid_tbl); + (name_map_item_t *) cl_qmap_head(&p_ftree->cn_guid_tbl); while (p_next_guid_element != - (ftree_guid_tbl_element_t *) cl_qmap_end(&p_ftree-> - cn_guid_tbl)) { + (name_map_item_t *) cl_qmap_end(&p_ftree->cn_guid_tbl)) { p_guid_element = p_next_guid_element; p_next_guid_element = - (ftree_guid_tbl_element_t *) cl_qmap_next(&p_guid_element-> - map_item); - __osm_ftree_guid_tbl_element_destroy(p_guid_element); + (name_map_item_t *) cl_qmap_next(&p_guid_element->item); + free(p_guid_element); } cl_qmap_remove_all(&p_ftree->cn_guid_tbl); /* remove all the elements of root_guid_list */ - - while ((p_guid = - (uint64_t *) cl_list_remove_head(&p_ftree->root_guid_list))) - free(p_guid); + while (!cl_is_qlist_empty(&p_ftree->root_guid_list)) + free(cl_qlist_remove_head(&p_ftree->root_guid_list)); /* free the leaf switches array */ if ((p_ftree->leaf_switches_num > 0) && (p_ftree->leaf_switches)) @@ -1073,7 +1030,6 @@ static void __osm_ftree_fabric_destroy(ftree_fabric_t * p_ftree) if (!p_ftree) return; __osm_ftree_fabric_clear(p_ftree); - cl_list_destroy(&p_ftree->root_guid_list); cl_pool_destroy(&p_ftree->sw_fwd_tbl_pool); free(p_ftree); } @@ -2992,14 +2948,14 @@ __osm_ftree_fabric_construct_hca_ports(IN ftree_fabric_t * p_ftree, if (!__osm_ftree_fabric_cns_provided(p_ftree)) { is_cn = TRUE; } else { - ftree_guid_tbl_element_t *p_elem = - (ftree_guid_tbl_element_t *) cl_qmap_get(&p_ftree-> - cn_guid_tbl, - osm_physp_get_port_guid - (p_osm_port)); + name_map_item_t *p_elem = + (name_map_item_t *) cl_qmap_get(&p_ftree-> + cn_guid_tbl, + osm_physp_get_port_guid + (p_osm_port)); if (p_elem != - (ftree_guid_tbl_element_t *) cl_qmap_end(&p_ftree-> - cn_guid_tbl)) + (name_map_item_t *) cl_qmap_end(&p_ftree-> + cn_guid_tbl)) is_cn = TRUE; } @@ -3180,37 +3136,33 @@ static int __osm_ftree_fabric_rank_from_roots(IN ftree_fabric_t * p_ftree) ftree_sw_t *p_sw; ftree_sw_t *p_remote_sw; cl_list_t ranking_bfs_list; - uint64_t *p_guid; + struct guid_list_item *item; int res = 0; unsigned num_roots; unsigned max_rank = 0; unsigned i; - cl_list_iterator_t guid_iterator; OSM_LOG_ENTER(&p_ftree->p_osm->log); cl_list_init(&ranking_bfs_list, 10); /* Rank all the roots and add them to list */ - - guid_iterator = cl_list_head(&p_ftree->root_guid_list); - while (guid_iterator != cl_list_end(&p_ftree->root_guid_list)) { - p_guid = (uint64_t *) cl_list_obj(guid_iterator); - guid_iterator = cl_list_next(guid_iterator); - + for (item = (void *)cl_qlist_head(&p_ftree->root_guid_list); + item != (void *)cl_qlist_end(&p_ftree->root_guid_list); + item = (void *)cl_qlist_next(&item->list)) { p_sw = __osm_ftree_fabric_get_sw_by_guid(p_ftree, - cl_hton64(*p_guid)); + cl_hton64(item->guid)); if (!p_sw) { /* the specified root guid wasn't found in the fabric */ OSM_LOG(&p_ftree->p_osm->log, OSM_LOG_ERROR, "ERR AB24: " "Root switch GUID 0x%" PRIx64 " not found\n", - *p_guid); + item->guid); continue; } OSM_LOG(&p_ftree->p_osm->log, OSM_LOG_DEBUG, "Ranking root switch with GUID 0x%" PRIx64 "\n", - *p_guid); + item->guid); p_sw->rank = 0; cl_list_insert_tail(&ranking_bfs_list, p_sw); } @@ -3444,28 +3396,35 @@ Exit: /*************************************************** ***************************************************/ +static int add_guid_item_to_list(void *cxt, uint64_t guid, char *p) +{ + cl_qlist_t *list = cxt; + struct guid_list_item *item; + + item = malloc(sizeof(*item)); + if (!item) + return -1; + + item->guid = guid; + cl_qlist_insert_tail(list, &item->list); -static void __osm_ftree_convert_list2qmap(cl_list_t * p_guid_list, - cl_qmap_t * p_map) + return 0; +} + +static int add_guid_item_to_map(void *cxt, uint64_t guid, char *p) { - uint64_t *p_guid; - CL_ASSERT(p_map); - if (!p_guid_list || !cl_list_count(p_guid_list)) - return; + cl_qmap_t *map = cxt; + name_map_item_t *item; - while ((p_guid = (uint64_t *) cl_list_remove_head(p_guid_list))) { - /* object key is guid in network order */ - cl_qmap_insert(p_map, cl_hton64(*p_guid), - &((__osm_ftree_guid_tbl_element_create - (*p_guid))->map_item)); - free(p_guid); - } - CL_ASSERT(cl_is_list_empty(p_guid_list)); + item = malloc(sizeof(*item)); + if (!item) + return -1; -} /* __osm_ftree_convert_list2qmap() */ + item->guid = guid; + cl_qmap_insert(map, guid, &item->item); -/*************************************************** - ***************************************************/ + return 0; +} static int __osm_ftree_fabric_read_guid_files(IN ftree_fabric_t * p_ftree) { @@ -3478,15 +3437,14 @@ static int __osm_ftree_fabric_read_guid_files(IN ftree_fabric_t * p_ftree) "Fetching root nodes from file %s\n", p_ftree->p_osm->subn.opt.root_guid_file); - if (osm_ucast_mgr_read_guid_file(&p_ftree->p_osm->sm.ucast_mgr, - p_ftree->p_osm->subn.opt. - root_guid_file, - &p_ftree->root_guid_list)) { + if (parse_node_map(p_ftree->p_osm->subn.opt.root_guid_file, + add_guid_item_to_list, + &p_ftree->root_guid_list)) { status = -1; goto Exit; } - if (!cl_list_count(&p_ftree->root_guid_list)) { + if (!cl_qlist_count(&p_ftree->root_guid_list)) { OSM_LOG(&p_ftree->p_osm->log, OSM_LOG_ERROR, "ERR AB22: " "Root guids file has no valid guids\n"); status = -1; @@ -3495,32 +3453,23 @@ static int __osm_ftree_fabric_read_guid_files(IN ftree_fabric_t * p_ftree) } if (__osm_ftree_fabric_cns_provided(p_ftree)) { - cl_list_t cn_guid_list; - cl_list_construct(&cn_guid_list); - cl_list_init(&cn_guid_list, 10); - OSM_LOG(&p_ftree->p_osm->log, OSM_LOG_DEBUG, "Fetching compute nodes from file %s\n", p_ftree->p_osm->subn.opt.cn_guid_file); - if (osm_ucast_mgr_read_guid_file(&p_ftree->p_osm->sm.ucast_mgr, - p_ftree->p_osm->subn.opt. - cn_guid_file, &cn_guid_list)) { + if (parse_node_map(p_ftree->p_osm->subn.opt.cn_guid_file, + add_guid_item_to_map, + &p_ftree->cn_guid_tbl)) { status = -1; goto Exit; } - if (!cl_list_count(&cn_guid_list)) { + if (!cl_qmap_count(&p_ftree->cn_guid_tbl)) { OSM_LOG(&p_ftree->p_osm->log, OSM_LOG_ERROR, "ERR AB23: " "Compute node guids file has no valid guids\n"); status = -1; goto Exit; } - - __osm_ftree_convert_list2qmap(&cn_guid_list, - &p_ftree->cn_guid_tbl); - cl_list_destroy(&cn_guid_list); - CL_ASSERT(cl_qmap_count(&p_ftree->cn_guid_tbl)); } Exit: -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 17:27:16 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 00:27:16 +0000 Subject: [ofa-general] [PATCH 5/6] opensm: remove unused osm_ucast_mgr_read_guid_file() In-Reply-To: <1206577637-18226-1-git-send-email-sashak@voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> Message-ID: <1206577637-18226-6-git-send-email-sashak@voltaire.com> Finally remove osm_ucast_mgr_read_guid_file() function which is unused anymore. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_ucast_mgr.h | 37 +----------------- opensm/opensm/osm_ucast_mgr.c | 68 +-------------------------------- 2 files changed, 2 insertions(+), 103 deletions(-) diff --git a/opensm/include/opensm/osm_ucast_mgr.h b/opensm/include/opensm/osm_ucast_mgr.h index 2acab49..0317c93 100644 --- a/opensm/include/opensm/osm_ucast_mgr.h +++ b/opensm/include/opensm/osm_ucast_mgr.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004, 2005 Voltaire, Inc. All rights reserved. + * Copyright (c) 2004-2008 Voltaire, Inc. All rights reserved. * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * @@ -278,41 +278,6 @@ void osm_ucast_mgr_build_lid_matrices(IN osm_ucast_mgr_t * const p_mgr); * Unicast Manager *********/ -/****f* OpenSM: Unicast Manager/osm_ucast_mgr_read_guid_file -* NAME -* osm_ucast_mgr_read_guid_file -* -* DESCRIPTION -* Read guid list from file. -* -* SYNOPSIS -*/ -cl_status_t -osm_ucast_mgr_read_guid_file(IN osm_ucast_mgr_t * const p_mgr, - IN const char *guid_file_name, - IN cl_list_t * p_list); -/* -* PARAMETERS -* p_mgr -* [in] Pointer to an osm_ucast_mgr_t object. -* -* guid_file_name -* [in] Name of the file to read. -* -* p_list -* [in] Pointer to the list that will be filled with guids. -* -* RETURN VALUES -* IB_SUCCESS if the file was read successfully. -* -* NOTES -* This function reads guids from a file and inserts them -* into a list. -* -* SEE ALSO -* Unicast Manager -*********/ - /****f* OpenSM: Unicast Manager/osm_ucast_mgr_process * NAME * osm_ucast_mgr_process diff --git a/opensm/opensm/osm_ucast_mgr.c b/opensm/opensm/osm_ucast_mgr.c index 46a4f09..938db84 100644 --- a/opensm/opensm/osm_ucast_mgr.c +++ b/opensm/opensm/osm_ucast_mgr.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2004-2007 Voltaire, Inc. All rights reserved. + * Copyright (c) 2004-2008 Voltaire, Inc. All rights reserved. * Copyright (c) 2002-2006 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * @@ -659,72 +659,6 @@ static int ucast_mgr_setup_all_switches(osm_subn_t * p_subn) /********************************************************************** **********************************************************************/ -cl_status_t -osm_ucast_mgr_read_guid_file(IN osm_ucast_mgr_t * const p_mgr, - IN const char *guid_file_name, - IN cl_list_t * p_list) -{ - cl_status_t status = IB_SUCCESS; - FILE *guid_file; - char line[MAX_GUID_FILE_LINE_LENGTH]; - char *endptr; - uint64_t *p_guid; - - OSM_LOG_ENTER(p_mgr->p_log); - - guid_file = fopen(guid_file_name, "r"); - if (guid_file == NULL) { - OSM_LOG(p_mgr->p_log, OSM_LOG_ERROR, "ERR 3A13: " - "Failed to open guid list file (%s)\n", guid_file_name); - status = IB_NOT_FOUND; - goto Exit; - } - - while (fgets(line, sizeof(line), guid_file)) { - if (strcspn(line, " ,;.") != strlen(line)) { - OSM_LOG(p_mgr->p_log, OSM_LOG_ERROR, "ERR 3A14: " - "Poorly formatted guid in file (%s): %s\n", - guid_file_name, line); - status = IB_NOT_FOUND; - break; - } - - /* Skip empty lines anywhere in the file - only one - char means the null termination */ - if (strlen(line) <= 1) - continue; - - p_guid = malloc(sizeof(uint64_t)); - if (!p_guid) { - status = IB_ERROR; - goto Exit; - } - - *p_guid = strtoull(line, &endptr, 16); - - /* check that the string is a number */ - if (!(*p_guid) && (*endptr != '\0')) { - OSM_LOG(p_mgr->p_log, OSM_LOG_ERROR, "ERR 3A15: " - "Poorly formatted guid in file (%s): %s\n", - guid_file_name, line); - status = IB_NOT_FOUND; - free(p_guid); - break; - } - - /* store the parsed guid */ - cl_list_insert_tail(p_list, p_guid); - } - -Exit: - if (guid_file) - fclose(guid_file); - OSM_LOG_EXIT(p_mgr->p_log); - return (status); -} - -/********************************************************************** - **********************************************************************/ osm_signal_t osm_ucast_mgr_process(IN osm_ucast_mgr_t * const p_mgr) { osm_opensm_t *p_osm; -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 17:27:17 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 00:27:17 +0000 Subject: [ofa-general] [PATCH 6/6] opensm/updn: --ids_guid_file - node guids to ids map In-Reply-To: <1206577637-18226-1-git-send-email-sashak@voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> Message-ID: <1206577637-18226-7-git-send-email-sashak@voltaire.com> With Up/Down if switches have equal rank (distance from root node), up or down direction is determined by additional switch unique ID comparison. Currently in OpenSM node GUIDs is used as such IDs. Doing some experiments with cluster simulation I found that with some topologies using IDs other than random node GUID numbers can optimize Up/Down routing and cluster performance dramatically. It was resulted in this patch where such IDs can be defined in text file which has simple " " format and can be passed as command line or configuration option to OpenSM. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_subnet.h | 5 +++ opensm/opensm/main.c | 14 +++++++- opensm/opensm/osm_subnet.c | 10 ++++++ opensm/opensm/osm_ucast_updn.c | 63 +++++++++++++++++++++++++++++------- 4 files changed, 79 insertions(+), 13 deletions(-) diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index 1c8b6d6..1b24381 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -260,6 +260,7 @@ typedef struct _osm_subn_opt { char *ucast_dump_file; char *root_guid_file; char *cn_guid_file; + char *ids_guid_file; char *sa_db_file; boolean_t exit_on_fatal; boolean_t honor_guid2lid_file; @@ -456,6 +457,10 @@ typedef struct _osm_subn_opt { * Name of the file that contains list of compute node guids that * will be used by fat-tree routing (provided by User) * +* ids_guid_file +* Name of the file that contains list of ids which should be +* used by Up/Down algorithm instead of node GUIDs +* * sa_db_file * Name of the SA database file. * diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index 5be0271..fb41d50 100644 --- a/opensm/opensm/main.c +++ b/opensm/opensm/main.c @@ -206,6 +206,11 @@ static void show_usage(void) " Set the compute nodes for the Fat-Tree routing algorithm\n" " to the guids provided in the given file (one to a line)\n" "\n"); + printf("-m\n" + "--ids_guid_file \n" + " Name of the map file with set of the IDs which will be used\n" + " by Up/Down routing algorithm instead of node GUIDs\n" + " (format: per line)\n"); printf("-o\n" "--once\n" " This option causes OpenSM to configure the subnet\n" @@ -594,7 +599,7 @@ int main(int argc, char *argv[]) char *ignore_guids_file_name = NULL; uint32_t val; const char *const short_option = - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; + "i:f:ed:g:l:L:s:t:a:u:m:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; /* In the array below, the 2nd parameter specifies the number @@ -634,6 +639,7 @@ int main(int argc, char *argv[]) {"sadb_file", 1, NULL, 'S'}, {"root_guid_file", 1, NULL, 'a'}, {"cn_guid_file", 1, NULL, 'u'}, + {"ids_guid_file", 1, NULL, 'm'}, {"cache-options", 0, NULL, 'c'}, {"stay_on_fatal", 0, NULL, 'y'}, {"honor_guid2lid", 0, NULL, 'x'}, @@ -917,6 +923,12 @@ int main(int argc, char *argv[]) opt.cn_guid_file); break; + case 'm': + /* Specifies ids guid file */ + opt.ids_guid_file = optarg; + printf(" IDs Guid File: %s\n", opt.ids_guid_file); + break; + case 'c': cache_options = TRUE; printf(" Caching command line options\n"); diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index b65cd74..47d735f 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -465,6 +465,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) p_opt->ucast_dump_file = NULL; p_opt->root_guid_file = NULL; p_opt->cn_guid_file = NULL; + p_opt->ids_guid_file = NULL; p_opt->sa_db_file = NULL; p_opt->exit_on_fatal = TRUE; p_opt->enable_quirks = FALSE; @@ -1324,6 +1325,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) opts_unpack_charp("cn_guid_file", p_key, p_val, &p_opts->cn_guid_file); + opts_unpack_charp("ids_guid_file", + p_key, p_val, &p_opts->ids_guid_file); + opts_unpack_charp("sa_db_file", p_key, p_val, &p_opts->sa_db_file); @@ -1558,6 +1562,12 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) "# The file holding the fat-tree compute node guids\n" "# One guid in each line\n" "cn_guid_file %s\n\n", p_opts->cn_guid_file); + if (p_opts->ids_guid_file) + fprintf(opts_file, + "# The file holding the node ids which will be used by" + " Up/Down algorithm instead\n# of GUIDs (one guid and" + " id in each line)\n" + "ids_guid_file %s\n\n", p_opts->ids_guid_file); if (p_opts->sa_db_file) fprintf(opts_file, "# SA database file name\n" diff --git a/opensm/opensm/osm_ucast_updn.c b/opensm/opensm/osm_ucast_updn.c index 3623a69..5e778c2 100644 --- a/opensm/opensm/osm_ucast_updn.c +++ b/opensm/opensm/osm_ucast_updn.c @@ -74,6 +74,7 @@ typedef struct _updn { struct updn_node { cl_list_item_t list; osm_switch_t *sw; + uint64_t id; updn_switch_dir_t dir; unsigned rank; unsigned visited; @@ -90,7 +91,7 @@ static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn); remote ports */ static updn_switch_dir_t __updn_get_dir(IN unsigned cur_rank, - IN unsigned rem_rank, IN uint64_t cur_guid, IN uint64_t rem_guid) + IN unsigned rem_rank, IN uint64_t cur_id, IN uint64_t rem_id) { /* HACK: comes to solve root nodes connection, in a classic subnet root nodes do not connect directly, but in case they are we assign to root node an UP direction to allow UPDN to discover @@ -104,8 +105,8 @@ __updn_get_dir(IN unsigned cur_rank, else if (cur_rank > rem_rank) return UP; else { - /* Equal rank, decide by guid number, bigger == UP direction */ - if (cur_guid > rem_guid) + /* Equal rank, decide by id number, bigger == UP direction */ + if (cur_id > rem_id) return UP; else return DOWN; @@ -145,12 +146,9 @@ __updn_bfs_by_node(IN osm_log_t * p_log, /* BFS the list till no next element */ while (!cl_is_qlist_empty(&list)) { - ib_net64_t remote_guid, current_guid; - u = (struct updn_node *)cl_qlist_remove_head(&list); u->visited = 0; /* cleanup */ current_dir = u->dir; - current_guid = osm_node_get_node_guid(u->sw->p_node); /* Go over all ports of the switch and find unvisited remote nodes */ for (pn = 1; pn < u->sw->num_ports; pn++) { osm_node_t *p_remote_node; @@ -167,13 +165,11 @@ __updn_bfs_by_node(IN osm_log_t * p_log, if (!p_remote_node || !p_remote_node->sw) continue; /* Fetch remote guid only after validation of remote node */ - remote_guid = osm_node_get_node_guid(p_remote_node); p_remote_sw = p_remote_node->sw; rem_u = p_remote_sw->priv; /* Decide which direction to mark it (UP/DOWN) */ next_dir = __updn_get_dir(u->rank, rem_u->rank, - cl_ntoh64(current_guid), - cl_ntoh64(remote_guid)); + u->id, rem_u->id); /* Check if this is a legal step : the only illegal step is going from DOWN to UP */ @@ -181,8 +177,8 @@ __updn_bfs_by_node(IN osm_log_t * p_log, OSM_LOG(p_log, OSM_LOG_DEBUG, "Avoiding move from 0x%016" PRIx64 " to 0x%016" PRIx64 "\n", - cl_ntoh64(current_guid), - cl_ntoh64(remote_guid)); + cl_ntoh64(osm_node_get_node_guid(u->sw->p_node)), + cl_ntoh64(osm_node_get_node_guid(p_remote_node))); /* Illegal step */ continue; } @@ -414,6 +410,7 @@ static struct updn_node *create_updn_node(osm_switch_t * sw) return NULL; memset(u, 0, sizeof(*u)); u->sw = sw; + u->id = cl_ntoh64(osm_node_get_node_guid(sw->p_node)); u->rank = 0xffffffff; return u; } @@ -434,6 +431,35 @@ static void dump_roots(cl_map_item_t *item, FILE *file, void *cxt) cl_ntoh64(osm_node_get_node_guid(sw->p_node))); } +static int update_id(void *cxt, uint64_t guid, char *p) +{ + osm_opensm_t *osm = cxt; + osm_switch_t *sw; + uint64_t id; + char *e; + + sw = osm_get_switch_by_guid(&osm->subn, cl_hton64(guid)); + if (!sw) { + OSM_LOG(&osm->log, OSM_LOG_VERBOSE, + "switch with guid 0x%" PRIx64 " is not found\n", guid); + return 0; + } + + id = strtoull(p, &e, 0); + if (*e) { + OSM_LOG(&osm->log, OSM_LOG_ERROR, + "ERR: cannot parse node id \'%s\'", p); + return -1; + } + + OSM_LOG(&osm->log, OSM_LOG_DEBUG, + "update node 0x%" PRIx64 " id to 0x%" PRIx64 "\n", guid, id); + + ((struct updn_node *)sw->priv)->id = id; + + return 0; +} + static int rank_root_node(void *cxt, uint64_t guid, char *p) { updn_t *updn = cxt; @@ -483,7 +509,7 @@ static int __osm_updn_call(void *ctx) if (p_updn->p_osm->subn.opt.root_guid_file) { OSM_LOG(&p_updn->p_osm->log, OSM_LOG_DEBUG, - "UPDN - Fetching root nodes from file %s\n", + "UPDN - Fetching root nodes from file \'%s\'\n", p_updn->p_osm->subn.opt.root_guid_file); ret = parse_node_map(p_updn->p_osm->subn.opt.root_guid_file, @@ -500,6 +526,19 @@ static int __osm_updn_call(void *ctx) __osm_updn_find_root_nodes_by_min_hop(p_updn); } + if (p_updn->p_osm->subn.opt.ids_guid_file) { + OSM_LOG(&p_updn->p_osm->log, OSM_LOG_DEBUG, + "UPDN - update node ids from file \'%s\'\n", + p_updn->p_osm->subn.opt.ids_guid_file); + + ret = parse_node_map(p_updn->p_osm->subn.opt.ids_guid_file, + update_id, p_updn->p_osm); + if (ret) + OSM_LOG(&p_updn->p_osm->log, OSM_LOG_ERROR, "ERR : " + "cannot parse node ids file \'%s\'\n", + p_updn->p_osm->subn.opt.ids_guid_file); + } + /* Only if there are assigned root nodes do the algorithm, otherwise perform do nothing */ if (p_updn->num_roots) { OSM_LOG(&p_updn->p_osm->log, OSM_LOG_DEBUG, -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Wed Mar 26 17:41:48 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 00:41:48 +0000 Subject: [ofa-general] Re: [PATCH 4/4] opensm: remove not used osm_log_printf() function In-Reply-To: <1206556102.8099.754.camel@hrosenstock-ws.xsigo.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> <1206552426-17279-5-git-send-email-sashak@voltaire.com> <1206554847.8099.747.camel@hrosenstock-ws.xsigo.com> <20080326183742.GK13708@sashak.voltaire.com> <1206556102.8099.754.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080327004148.GQ13708@sashak.voltaire.com> On 11:28 Wed 26 Mar , Hal Rosenstock wrote: > > Because it was published as a global API and something outside might be > using it. It's not necessarily just for internal consumers. It was recent enough and almost useless function so I'm very skeptical it was used anywhere. And of course I checked publicly available places (like ibutils). If you like I can depreciate this for some period of time, but personally don't think it is needed for anybody. Sasha From weiny2 at llnl.gov Wed Mar 26 17:52:28 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 26 Mar 2008 17:52:28 -0700 Subject: [ofa-general] Re: [PATCH 1/6] complib/nodenamemap: add generic parse_node_map() function In-Reply-To: <1206577637-18226-2-git-send-email-sashak@voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> <1206577637-18226-2-git-send-email-sashak@voltaire.com> Message-ID: <20080326175228.0cee3e5a.weiny2@llnl.gov> Hey Sasha, On Thu, 27 Mar 2008 00:27:12 +0000 Sasha Khapyorsky wrote: > This parser is able to parse text file with format similar to nodenamemap > file: > > guid # comment > .... I like the idea of using this same format. > > It will be used later for unification parsing of root nodes and compute > nodes guid list files. > > Signed-off-by: Sasha Khapyorsky > --- > opensm/complib/cl_nodenamemap.c | 45 +++++++++++++++++++++++++++++++ > opensm/complib/libosmcomp.map | 1 + > opensm/include/complib/cl_nodenamemap.h | 3 ++ > 3 files changed, 49 insertions(+), 0 deletions(-) > > diff --git a/opensm/complib/cl_nodenamemap.c b/opensm/complib/cl_nodenamemap.c > index af5da39..696d455 100644 > --- a/opensm/complib/cl_nodenamemap.c > +++ b/opensm/complib/cl_nodenamemap.c > @@ -160,3 +160,48 @@ clean_nodedesc(char *nodedesc) > > return (nodedesc); > } > + > +int parse_node_map(const char *file_name, > + int (*create)(void *, uint64_t, char *), void *cxt) Why do you need this new way of parsing the file format? I think passing this callback makes the code more complicated that it needs to be. Is there a reason not to use the nn_map_t as the type for root_nodes_list in the updn_t struct? We could rename nn_map_t to be more generic if necessary. I worry about maintaining 2 functions which parse the same file format in the same c file. Ira > +{ > + char line[256]; > + FILE *f; > + > + if (!(f = fopen(file_name, "r"))) > + return -1; > + > + while (fgets(line, sizeof(line),f)) { > + uint64_t guid; > + char *p, *e; > + > + p = line; > + while (isspace(*p)) > + p++; > + if (*p == '\0' || *p == '\n' || *p == '#') > + continue; > + > + guid = strtoull(p, &e, 0); > + if (e == p || !isspace(*e)) { > + fclose(f); > + return -1; > + } > + > + p = e; > + if (*e) > + e++; > + while (isspace(*p)) > + p++; > + > + e = strpbrk(p, "# \t\n"); > + if (e) > + *e = '\0'; > + > + if (create(cxt, guid, p)) { > + fclose(f); > + return -1; > + } > + } > + > + fclose(f); > + return 0; > +} > diff --git a/opensm/complib/libosmcomp.map b/opensm/complib/libosmcomp.map > index d0e107e..788eb2a 100644 > --- a/opensm/complib/libosmcomp.map > +++ b/opensm/complib/libosmcomp.map > @@ -147,6 +147,7 @@ OSMCOMP_2.3 { > ib_wc_status_str; > open_node_name_map; > close_node_name_map; > + parse_node_map; > remap_node_name; > clean_nodedesc; > local: *; > diff --git a/opensm/include/complib/cl_nodenamemap.h b/opensm/include/complib/cl_nodenamemap.h > index b20ca4e..5f4a026 100644 > --- a/opensm/include/complib/cl_nodenamemap.h > +++ b/opensm/include/complib/cl_nodenamemap.h > @@ -1,4 +1,5 @@ > /* > + * Copyright (c) 2008 Voltaire, Inc. All rights reserved. > * Copyright (c) 2007 Lawrence Livermore National Lab > * > * This software is available to you under a choice of one of two > @@ -60,5 +61,7 @@ void close_node_name_map(nn_map_t *map); > char *remap_node_name(nn_map_t *map, uint64_t target_guid, > char *nodedesc); > /* NOTE: parameter "nodedesc" may be modified here. */ > +int parse_node_map(const char *file_name, > + int (*create)(void *, uint64_t, char *), void *cxt); > > #endif /* _CL_NODE_NAME_MAP_H_ */ > -- > 1.5.4.1.122.gaa8d From ardavis at ichips.intel.com Wed Mar 26 17:54:49 2008 From: ardavis at ichips.intel.com (Arlin Davis) Date: Wed, 26 Mar 2008 17:54:49 -0700 Subject: [ofa-general] Possibly proprietary license in dapl2 getopt.c and getopt.h In-Reply-To: <47EAD853.8090808@novell.com> References: <47EAD853.8090808@novell.com> Message-ID: <47EAF059.2070505@ichips.intel.com> John Jolly wrote: > All, > > Within the dapl2 package, there are two source files: > > dapl-2.0.7/test/dtest/GETOPT.C > dapl-2.0.7/test/dtest/GETOPT.H > > were derived by Intel Corporation from public domain code originally > owned by AT&T. Intel modified the code and licensed it under the license > shown below - effectively a proprietary license which states that it is > license "free of charge" but does _not_ give sufficient rights to > actually copy/modify/distribute the code. > > However, we (SuSE/Novell) tried building the package without the files > (i.e. added a rm in the spec file) and the package built successfully. > Since they are not used at all, we (SuSE/Novell) have consider removing > them completely from _source_ and _binary_ (in the same manner that we > remove e.g. mp3 decoders from the tarball and the binary) as it > represents an unnecessary business risk for which there is no > perceivable business advantage to be gained. These files came in with some WinOF updates. They should not be included with the Linux distribution package. Hmmm, do you picked up the dapl package from git tree instead of released packages from OFA downloads? Either way, you are correct, these files should be removed. -arlin From LyndainflameDunham at associated.org Wed Mar 26 18:00:36 2008 From: LyndainflameDunham at associated.org (Angie Swartz) Date: Thu, 27 Mar 2008 01:00:36 +0000 Subject: [ofa-general] Visa Line of Credit Message-ID: <7IX568EJXVWDA941@associated.org> An HTML attachment was scrubbed... URL: From nheujyhcnvr at blendpak.com Wed Mar 26 20:08:08 2008 From: nheujyhcnvr at blendpak.com (Morgan Crump) Date: Thu, 27 Mar 2008 12:08:08 +0900 Subject: [ofa-general] No weight - no problems Message-ID: <01c89003$382f8880$8ec079db@nheujyhcnvr> Never late to try. Watch details attached and know more. -------------- next part -------------- A non-text attachment was scrubbed... Name: mp5.zip Type: application/zip Size: 579 bytes Desc: not available URL: From xpouu at bluent.com Wed Mar 26 20:46:39 2008 From: xpouu at bluent.com (Newton Silva) Date: Thu, 27 Mar 2008 12:46:39 +0900 Subject: [ofa-general] Be damn goood in it Message-ID: <01c89008$99f09c80$622a64de@xpouu> Take her to love heaven Look attached details and find MORE! -------------- next part -------------- A non-text attachment was scrubbed... Name: vo.zip Type: application/zip Size: 636 bytes Desc: not available URL: From sodjkonlinestudiorox at jkonlinestudio.com Wed Mar 26 23:14:49 2008 From: sodjkonlinestudiorox at jkonlinestudio.com (Ken Chu) Date: Thu, 27 Mar 2008 00:14:49 -0600 Subject: [ofa-general] Software Message-ID: <011522334.32085761277970@jkonlinestudio.com> Are you searhing for the better value in software abatements? This time you will catch the chance to use the software equipment you wanted for a long time. And the best matter for you is, all softs are of terribly low cost. Check it out by yourself & have the softwares for cheap rates! http://sherilankru966.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From devesh28 at gmail.com Wed Mar 26 23:24:11 2008 From: devesh28 at gmail.com (Devesh Sharma) Date: Thu, 27 Mar 2008 11:54:11 +0530 Subject: [ofa-general] IPoIB : sendmsg: No buffer space available Message-ID: <309a667c0803262324n11838fbbrb5f010c2ac930273@mail.gmail.com> Hello all, I am receiving IPoIB : sendmsg: No buffer space available message after some time when I ping between two IB nodes. what is the probable reason? -Devesh From malstafc at kci1.com Wed Mar 26 23:39:21 2008 From: malstafc at kci1.com (EuroSoftware) Date: Thu, 27 Mar 2008 14:39:21 +0800 Subject: [ofa-general] Vergleichen Sie Preise und kaufen Sie hier Message-ID: <401697092.70302998239170@kci1.com> There is no cheaper source of original and perfectly working software.Bekommen Sie Ihre Software unverzueglich. Einfach zahlen und sofort runterladen. Hier sind Programme in allen europaeischen Sprachen verfuegbar, programmiert fuer Windows und Macintosh. Alle Softwaren sind sehr guenstig, es handelt sich dabei garantiert um originale, komplette und voellig funktionale Versionen.http://sottesof.com* Office Enterprise 2007: $79.95 * Windows XP Professional With SP2 Full Version: $59.95 * AutoCAD 2008: $129.95 * Adobe Creative Suite 3 Design Premium: $229.95 http://sottesof.comProfessionelle und persoenliche Beratung von unserem Kundencenter wird Ihnen sicherlich bei der Softwareinstallation helfen. Schnelle Antworten garantiert. Geld-Zurueck-Garantie ist verfuegbar! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Wed Mar 26 23:49:18 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 27 Mar 2008 08:49:18 +0200 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <200803261716.26026.jackm@dev.mellanox.co.il> References: <1205767427.25950.137.camel@mtls03> <200803261620.19395.jackm@dev.mellanox.co.il> <47EA636A.8070604@voltaire.com> <200803261716.26026.jackm@dev.mellanox.co.il> Message-ID: <47EB436E.50908@voltaire.com> Jack Morgenstein wrote: > Actually, we did not break the ABI for XRC (we had to do cartwheels to avoid > this, but what the heck). > .... > You can avoid doing this in one of 2 ways: > Either: create a new QP type > Or: add new verbs to the core. Jack - OK, got it, and thanks a lot! for the coaching. Roland - in the case of the connectx block loopback feature, it seems that the simplest way to go would for user space is to create a new QP type IB_QPT_UD_BL[block-loopback] which is translated by the uverbs layer to what ever scheme we decide on for the kernel (eg create_flags, qp_types, bit fields, etc). As for IPoIB, it would create a UD/LSO/BL QP, thoughts? Or. From an.immerfall at ph-gmuend.de Thu Mar 27 00:21:00 2008 From: an.immerfall at ph-gmuend.de (Jenna Sanders) Date: Thu, 27 Mar 2008 16:21:00 +0900 Subject: [ofa-general] Viia..aa^_^aaagra... - 1.79$ - Preise ohne Konk. urrenz Message-ID: <01c89026$8bd3a700$350458dc@an.immerfall> Haben Sie endlich wieder Spass am Leben! Pr. .. Eise die keine Konk... ..Urrenz kennen - keine versteckte Kos// Ten - Bequem und dis kret 0... . N-line! be... .Stellen. - Disk rete Verpackung und Zahlung - Kein peinlicher Arz t besuch erforderlich - Kos... Tenlose, arztliche Telefon-Beratung - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen Originalme/ dikamente Ciia..aa^_^aaalis...... 10 Pack. 21,00 Euro Viia..aa^_^aaagra... 10 Pack. 11,00 Euro Jetzt bestellen - und vier Pil. .. len umsonst erhalten (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jrdonfxk at brandspringsolutions.com Thu Mar 27 00:44:16 2008 From: jrdonfxk at brandspringsolutions.com (Henrietta Read) Date: Thu, 27 Mar 2008 08:44:16 +0100 Subject: [ofa-general] On top all night Message-ID: <01c88fe6$bd7e4c00$86010453@jrdonfxk> Take her to seven heaven Look attached details and find MORE! -------------- next part -------------- A non-text attachment was scrubbed... Name: vo.zip Type: application/zip Size: 636 bytes Desc: not available URL: From eli at dev.mellanox.co.il Thu Mar 27 01:18:48 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 27 Mar 2008 10:18:48 +0200 Subject: [ofa-general] IPoIB : sendmsg: No buffer space available In-Reply-To: <309a667c0803262324n11838fbbrb5f010c2ac930273@mail.gmail.com> References: <309a667c0803262324n11838fbbrb5f010c2ac930273@mail.gmail.com> Message-ID: <1206605928.9065.17.camel@mtls03> On Thu, 2008-03-27 at 11:54 +0530, Devesh Sharma wrote: > I am receiving IPoIB : sendmsg: No buffer space available message > after some time > when I ping between two IB nodes. > what is the probable reason? > It means the socket buffer ping is using is exhausted. Did you have connectivity in the first place or you just try to ping forever with no success? From sashak at voltaire.com Thu Mar 27 01:54:29 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 08:54:29 +0000 Subject: [ofa-general] Re: [PATCH 1/6] complib/nodenamemap: add generic parse_node_map() function In-Reply-To: <20080326175228.0cee3e5a.weiny2@llnl.gov> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> <1206577637-18226-2-git-send-email-sashak@voltaire.com> <20080326175228.0cee3e5a.weiny2@llnl.gov> Message-ID: <20080327085429.GA23849@sashak.voltaire.com> Hi Ira, On 17:52 Wed 26 Mar , Ira Weiny wrote: > > diff --git a/opensm/complib/cl_nodenamemap.c b/opensm/complib/cl_nodenamemap.c > > index af5da39..696d455 100644 > > --- a/opensm/complib/cl_nodenamemap.c > > +++ b/opensm/complib/cl_nodenamemap.c > > @@ -160,3 +160,48 @@ clean_nodedesc(char *nodedesc) > > > > return (nodedesc); > > } > > + > > +int parse_node_map(const char *file_name, > > + int (*create)(void *, uint64_t, char *), void *cxt) > > Why do you need this new way of parsing the file format? I think passing this > callback makes the code more complicated that it needs to be. I don't think that it is much more complicated. OTOH it provides a great flexibility (it is used in later patches). > Is there a reason not to use the nn_map_t as the type for root_nodes_list in > the updn_t struct? If you will look at later updn patches you will find that there is no any root nodes storage at all. The same is true for later up/down IDs patch. Likely similar (intermediate storage free) improvement is possible in fat-tree as well. > We could rename nn_map_t to be more generic if necessary. > > I worry about maintaining 2 functions which parse the same file format in the > same c file. I agree with you, just didn't time to merge it. Sasha From getoearn at runbox.com Thu Mar 27 02:11:02 2008 From: getoearn at runbox.com (getoearn at runbox.com) Date: Thu, 27 Mar 2008 10:11:02 +0100 Subject: [ofa-general] A trend suggestion from :) Shee Message-ID: Uncover The New Trend Today With The Secrets Through The "VAULT" :) http://earnsmartcashwithlcm.firez.org/ ------------------------------------------------------------------------------ Greetings! Welcome to the most EXCITING, SIMPLE, and FAST way to make money that we have ever witnessed! The New Trend Today; Smart Cash System is so EASY and can make you SO MUCH MONEY that those who use it soon begin to weigh the option of going to work... or staying home and playing... yes playing... the Smart Cash System! What's even more amazing is once you buy the Smart Cash System today, you are entitled to an $80.00 commission when everyone who you personally refer to Lawn Chair Millionaire buys the system too! And guaranteed... a massive percentage of those who so much as glance at the Smart Cash System snatch it up without thinking twice because it's so simple. 11% gains in one day, 54% gains in 3 days... $300 into $3,000 in less than 30 days... $80.00 commission per sale! You wouldn't believe the number of success stories we get bombarded with on a daily basis. The Smart Cash System is so consistent and so easy for everyone that most people admit feeling as if they are costing themselves money by going to work rather than staying home and having fun with the Smart Cash System. There are a lot of simple ways to make money in the world but the reason the Smart Cash System was introduced to The Vault is because it is hands down the most consistent money-maker the Group has ever seen. Month after month we have been making MASSIVE financial gains. Wondering how much? It's OK... most people do! We had never used the Smart Cash System before so this truly was a test. We read the simple instruction for about an hour, followed the outline for opening an account with a bookie, and within two hours we went into action. Get to Earn In Your Spare Time Right NOW! Click Here ;) http://earnsmartcashwithlcm.firez.org/ ;) http://earnsmartcashwithlcm.firez.org/ ;) http://earnsmartcashwithlcm.firez.org/ Your Associates, Sheena Jane Contact me to YM: ren0_x Contact me here: xxheena at gmail.com http://profitizers.info/ KM. 6 Orchid St. Phil. ------------------------------------------------------------------------------- If you no longer wish to receive communication from us: Opt-out here :( http://unsubscribehere.zeroweb.org/ This message was sent to you by www.trendwatching.com, on behalf of :) Shee (getoearn at runbox.com) From duggarjd at hawes.navy.mil Thu Mar 27 01:03:29 2008 From: duggarjd at hawes.navy.mil (irv kinson) Date: Thu, 27 Mar 2008 08:03:29 +0000 Subject: [ofa-general] Shocking mpeg4 Kate Moss Message-ID: <000a01c88ff0$03a2389f$fb205cbd@xjnrg> #ZZRbGkMeg Ryan Stunning porno dvd. #FWBJrEThe pornos is Kick-up! #ZRbGkXOnly 1 day trial - get this Full mpeg4 now! #FWBJrE Download it now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tziporet at dev.mellanox.co.il Thu Mar 27 03:28:48 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 27 Mar 2008 12:28:48 +0200 Subject: [ofa-general] determine release version of available source In-Reply-To: <1206574698.10075.290.camel@pc.ilinx> References: <1206574698.10075.290.camel@pc.ilinx> Message-ID: <47EB76E0.6060005@mellanox.co.il> Brian J. Murrell wrote: > Is there anything or anyway in those files (or possibly others) to > determine if this is OFED 1.0, 1.1, 1.2.x, or something else entirely? > It's entirely acceptable to build a small test program with them even to > assist. > If OFED is installed on you system you can sue ofed_info script to get the git trees of the sources. Tziporet From csd at grantscanada.info Thu Mar 27 20:29:55 2008 From: csd at grantscanada.info (csd at grantscanada.info) Date: Fri, 28 Mar 2008 09:29:55 +0600 Subject: [ofa-general] *Government financing available* Message-ID: Press release. / Communiqu� (Message en francais ci-dessous) Canadian Subsidy directory (2008 EDITION) Member of Quebec Better Business Bureau Legal Deposit-National Library of Canada The new Subsidy Directory 2008 is now available, newly revised it is the most complete and affordable reference for anyone looking for financing. It is the perfect tool for new and existing businesses, individuals, foundations and associations. This Publication contains more than 3200 direct and indirect financial subsidies, grants and loans offered by government departments and agencies, foundations, associations and organizations. In this edition all programs are well described. Canadian Subsidy Directory (All Canada, federal + provincial + foundations) CD-Rom (Pdf file).............................$ 69.95 Printed (430 pages)..........................$149.95 Also available for each province on CD-Rom only...........$ 49.95 Alberta British Columbia New Brunswick Newfoundland & Labrador Northwest Territories / Nunavut / Yukon Manitoba Nova Scotia Ontario Prince Edward Island Quebec .............................$ 69.95 Saskatchewan To obtain a copy please call toll free 1-866-322-3376 or local 819-322-5756 Delete your email address at delete2008 at csd2008.net ********************************* Canadian Subsidy Directory 14-A Des Seigneurs St-Sauveur Qc J0R 1R0 ********************************* FRANCAIS Subventions Qu�bec (Annuaire des Subventions au Qu�bec 2008) Vous vous demandez s�il existe un programme d�aide financi�re pour vos besoins personnels ou pour assister votre entreprise dans son �tat actuel? Subventions Qu�bec vous offre un acc�s unique � tous les programmes d�assistance financi�re sous forme de subventions provenant des diff�rents gouvernements et organismes du Canada. Subventions Qu�bec vous offre la liste de toutes les institutions gouvernementales qui participent aux nombreuses subventions destin�es aux particuliers et aux entreprises du Canada. Subventions Qu�bec identifie, pour vous, sous son � Canadian Subsidy Directory � quelques trois mille (3,000) programmes d�aide disponible aux entreprises et aux particuliers �tablis au Canada. Ces programmes vous sont offerts sous forme de bourses, de pr�ts et de subventions de toutes sortes Subventions Qu�bec identifie, en plus des trois mille (3,000) programmes identifi�s dans le � Canadian Subsidy Directory �, quelques mille huit cent (1,800) programmes d�aide suppl�mentaires sous le couvert de � l�Annuaire des Subventions au Qu�bec �. Ce guide, destin� aux particuliers ainsi qu�aux entreprises � la recherche de subventions et implant�es au Qu�bec, d�crit les programmes d�aide et subventions disponibles par le biais des diff�rents paliers gouvernementaux et organismes. � L�Annuaire des Subventions au Qu�bec � est con�u pour assister les particuliers et les entreprises dans leur recherche de financement, que ce soit sous forme de pr�ts, de bourses ou sous forme de subventions. Subventions Qu�bec offre de l�information destin�e aux particuliers pour leurs besoins personnels mais aussi aux entreprises de toutes tailles, de la soci�t� � propri�taire unique, la PME, l�organisme � but non lucratif jusqu�� l�entreprise multinationale. Subventions Qu�bec vous invite � consulter ces deux (2) ouvrages pour d�couvrir plusieurs outils de financement disponibles sous formes diverses et acc�der aux subventions, aux bourses et aux pr�ts consentis par les autorit�s comp�tentes en mesurant les conditions de votre situation personnelle ou celle de votre entreprise aux crit�res d�admissibilit� en vigueur. ASQ-2008 Cd-rom(format .pdf).............$ 69.95 ASQ-2008 Imprim�(Cd-inclus) .............$ 149.95 Informations.....................ligne sans frais 1-877-227-8551 ********************************* Subventions Qu�bec 14-A Des Seigneurs St-Sauveur Qc J0R 1R0 ********************************* From devesh28 at gmail.com Thu Mar 27 03:53:37 2008 From: devesh28 at gmail.com (Devesh Sharma) Date: Thu, 27 Mar 2008 16:23:37 +0530 Subject: Fwd: [ofa-general] IPoIB : sendmsg: No buffer space available In-Reply-To: <309a667c0803270331t230407ecifdcbd9dd40f70c6e@mail.gmail.com> References: <309a667c0803262324n11838fbbrb5f010c2ac930273@mail.gmail.com> <1206605928.9065.17.camel@mtls03> <309a667c0803270331t230407ecifdcbd9dd40f70c6e@mail.gmail.com> Message-ID: <309a667c0803270353q1769c604rae3308bcfcfe7b2c@mail.gmail.com> ---------- Forwarded message ---------- From: Devesh Sharma Date: Thu, Mar 27, 2008 at 4:01 PM Subject: Re: [ofa-general] IPoIB : sendmsg: No buffer space available To: eli at dev.mellanox.co.il Hello Eli, Thanks for replying, yes initially if I am ping'ing with 64 Bytes it pings successfully for some 500 odd times after that this message starts flashing on my screen. If I increase the data size number of itrations comes down. On Thu, Mar 27, 2008 at 1:48 PM, Eli Cohen wrote: > On Thu, 2008-03-27 at 11:54 +0530, Devesh Sharma wrote: > > > I am receiving IPoIB : sendmsg: No buffer space available message > > after some time > > when I ping between two IB nodes. > > what is the probable reason? > > > It means the socket buffer ping is using is exhausted. Did you have > connectivity in the first place or you just try to ping forever with no > success? > > From eli at dev.mellanox.co.il Thu Mar 27 04:22:35 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 27 Mar 2008 13:22:35 +0200 Subject: Fwd: [ofa-general] IPoIB : sendmsg: No buffer space available In-Reply-To: <309a667c0803270353q1769c604rae3308bcfcfe7b2c@mail.gmail.com> References: <309a667c0803262324n11838fbbrb5f010c2ac930273@mail.gmail.com> <1206605928.9065.17.camel@mtls03> <309a667c0803270331t230407ecifdcbd9dd40f70c6e@mail.gmail.com> <309a667c0803270353q1769c604rae3308bcfcfe7b2c@mail.gmail.com> Message-ID: <1206616955.9065.36.camel@mtls03> On Thu, 2008-03-27 at 16:23 +0530, Devesh Sharma wrote: > Hello Eli, > Thanks for replying, > yes initially if I am ping'ing with 64 Bytes it pings successfully for > some 500 odd times after that this message starts flashing on my > screen. If I increase the data size number of itrations comes down. Which version are you using, is it ofed? Which OS? Please send any useful information and I'll try to reproduce. > > On Thu, Mar 27, 2008 at 1:48 PM, Eli Cohen wrote: > > On Thu, 2008-03-27 at 11:54 +0530, Devesh Sharma wrote: > > > > > I am receiving IPoIB : sendmsg: No buffer space available message > > > after some time > > > when I ping between two IB nodes. > > > what is the probable reason? > > > > > It means the socket buffer ping is using is exhausted. Did you have > > connectivity in the first place or you just try to ping forever with no > > success? > > > > From Brian.Murrell at Sun.COM Thu Mar 27 05:21:56 2008 From: Brian.Murrell at Sun.COM (Brian J. Murrell) Date: Thu, 27 Mar 2008 08:21:56 -0400 Subject: [ofa-general] determine release version of available source In-Reply-To: <47EB76E0.6060005@mellanox.co.il> References: <1206574698.10075.290.camel@pc.ilinx> <47EB76E0.6060005@mellanox.co.il> Message-ID: <1206620516.10075.299.camel@pc.ilinx> On Thu, 2008-03-27 at 12:28 +0200, Tziporet Koren wrote: > > If OFED is installed on you system you can sue ofed_info script to get > the git trees of the sources. I don't have an installed system, just kernel sources and I don't really want to "get the git trees". I just want to know what version/release of the driver I already have given the sources I have on hand. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From devesh28 at gmail.com Thu Mar 27 05:37:45 2008 From: devesh28 at gmail.com (Devesh Sharma) Date: Thu, 27 Mar 2008 18:07:45 +0530 Subject: Fwd: [ofa-general] IPoIB : sendmsg: No buffer space available In-Reply-To: <1206616955.9065.36.camel@mtls03> References: <309a667c0803262324n11838fbbrb5f010c2ac930273@mail.gmail.com> <1206605928.9065.17.camel@mtls03> <309a667c0803270331t230407ecifdcbd9dd40f70c6e@mail.gmail.com> <309a667c0803270353q1769c604rae3308bcfcfe7b2c@mail.gmail.com> <1206616955.9065.36.camel@mtls03> Message-ID: <309a667c0803270537h234a0065ib774b0d5392f9e63@mail.gmail.com> Its OFED-1.1 RHEL4-U4-x86_64, How the send buffers are freed in IPoIB module? Is there any configurable limit of socket buffers usage? or what is the default number ? any information or link will be helpful. On Thu, Mar 27, 2008 at 4:52 PM, Eli Cohen wrote: > On Thu, 2008-03-27 at 16:23 +0530, Devesh Sharma wrote: > > Hello Eli, > > Thanks for replying, > > yes initially if I am ping'ing with 64 Bytes it pings successfully for > > some 500 odd times after that this message starts flashing on my > > screen. If I increase the data size number of itrations comes down. > Which version are you using, is it ofed? Which OS? Please send any > useful information and I'll try to reproduce. > > > > > > > On Thu, Mar 27, 2008 at 1:48 PM, Eli Cohen wrote: > > > On Thu, 2008-03-27 at 11:54 +0530, Devesh Sharma wrote: > > > > > > > I am receiving IPoIB : sendmsg: No buffer space available message > > > > after some time > > > > when I ping between two IB nodes. > > > > what is the probable reason? > > > > > > > It means the socket buffer ping is using is exhausted. Did you have > > > connectivity in the first place or you just try to ping forever with no > > > success? > > > > > > > > > From hrosenstock at xsigo.com Thu Mar 27 06:11:02 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Thu, 27 Mar 2008 06:11:02 -0700 Subject: [ofa-general] Re: [PATCH 4/4] opensm: remove not used osm_log_printf() function In-Reply-To: <20080327004148.GQ13708@sashak.voltaire.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> <1206552426-17279-5-git-send-email-sashak@voltaire.com> <1206554847.8099.747.camel@hrosenstock-ws.xsigo.com> <20080326183742.GK13708@sashak.voltaire.com> <1206556102.8099.754.camel@hrosenstock-ws.xsigo.com> <20080327004148.GQ13708@sashak.voltaire.com> Message-ID: <1206623462.15600.39.camel@hrosenstock-ws.xsigo.com> On Thu, 2008-03-27 at 00:41 +0000, Sasha Khapyorsky wrote: > On 11:28 Wed 26 Mar , Hal Rosenstock wrote: > > > > Because it was published as a global API and something outside might be > > using it. It's not necessarily just for internal consumers. > > It was recent enough It's been around since 11/06 (svn days) so that's several releases: commit 77d05cd79a1cad790aa7766331dcf4f2d8bddc4e Author: Sasha Khapyorsky Date: Fri Nov 3 11:55:58 2006 +0000 r10053: OpenSM: Remove obsolete p_report_buf This removes obsolete now shared sm->p_report_buf buffer and cleans up related code. And also introduces new log function osm_log_printf() which currently trivially sends formatted output to stdout. Signed-off-by: Sasha Khapyorsky > and almost useless function so I'm very skeptical it was used anywhere. Yes, it's trivial. > And of course I checked publicly available places (like ibutils). > > If you like I can depreciate this for some period of time, but personally > don't think it is needed for anybody. This issue was initially the procedural issue of removing a public API without deprecation warning for a release which is the norm. Personally I don't care about this specific API. -- Hal > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From jlentini at netapp.com Thu Mar 27 06:55:09 2008 From: jlentini at netapp.com (James Lentini) Date: Thu, 27 Mar 2008 09:55:09 -0400 (EDT) Subject: [ofa-general] new NFS/RDMA instructions for 2.6.25-rc1 In-Reply-To: <057f01c88f89$ddf28850$3414a8c0@md.baymicrosystems.com> References: <057f01c88f89$ddf28850$3414a8c0@md.baymicrosystems.com> Message-ID: If you are able to install the software described in the instructions below (new kernel sources, nfs-utils, ...), then you should be able to use NFS/RDMA on RHEL5. If you are looking for a prebuilt package with a backport of NFS/RDMA to the RHEL5 kernel (2.6.18?), we don't have that yet. james On Wed, 26 Mar 2008, Suresh Shelvapille wrote: > Is it possible to get Server/Client running on RHEL5? > > > > -----Original Message----- > > From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf > > Of James Lentini > > Sent: Monday, February 11, 2008 12:25 PM > > To: general at lists.openfabrics.org; linux-nfs at vger.kernel.org > > Subject: [ofa-general] new NFS/RDMA instructions for 2.6.25-rc1 > > > > > > Linux 2.6.25 will be the first official kernel release to contain the > > NFS/RDMA server. With the client and server now both available in > > 2.6.25-rc1, we've simplified our NFS/RDMA installation instructions. > > The new instructions are available here: > > > > http://nfs-rdma.sourceforge.net/Documents/README > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From yukimura at samfur.com Thu Mar 27 07:06:19 2008 From: yukimura at samfur.com (Thaila Indrawati) Date: Thu, 27 Mar 2008 16:06:19 +0200 Subject: [ofa-general] Probieren Sie es - Mann Lebt nur einmal Message-ID: <01c89024$7e2a8f80$8060614e@yukimura> Online anonym bestellen - original Qualitat - 100% wirksam Fakten von unseren Kunden: - Sex ist befriedigender denn je. Stress und Leistungsdruck verschwinden. Sie ist nie wieder frustriert, ich habe keine Angst mehr zu versagen. Es ist ein wundervolles korperliches Erlebnis, dem ein genauso tiefes Gefuhl folgt. - Die Nebenwirkungen sind minimal: manchmal eine verstopfte Nase, kurzzeitig ein roter Kopf - kein Kopfschmerz, sondern das Gefuhl, als wurde man eine Flasche eiskalte Cola in einem Zug trinken. - Interessanterweise macht eine Vi. allein noch keinen Stander. Man(n) muss wenigstens ein bisschen Lust auf Sex mit der Frau haben. Gegen eine Eiserne Jungfrau im Bett hilft auch die grosste Dosis nichts. Wer aber das erste Kribbeln in den Lenden spurt, wird einen stahlharten Stander haben, und das fur wenigstens vier Stunden. - Eine volle 100-mg-Dosis macht den Schwanz zum Schwert. Wer es ubertreibt, ist Schuld, wenn die Herzallerliebste am Ende einen Y-formigen Sarg braucht. Fur die meisten von uns sind 50 mg mehr als genug, wenn man das gute Stuck zwischen den Hohepunkten auch mal hangen lassen will ... zur Not hilft es da vielleicht, sich ein nacktes Grossmutterchen vorzustellen. - Wer noch Zeit und Lust fur eine schnelle Nummer am nachsten Morgen hat, sollte dafur sorgen, dann noch genug Vi. im Blut zu haben - damit es noch fur ein oder zwei "Stehaufmannchen" reicht. - Das Beste an Vi. ist die Sicherheit, dass man "mit Autopilot fliegt", dass man entspannt und ohne Sorgen zur Sache kommen kann, dass der Stander auch halt, auch wenn man unterbrochen wird (die Kinder klopfen an die Schlafzimmertur, der Hund bellt, das Kondom sitzt schlecht). Wenn man Vi. bewusst anwendet, kann es auch der Partnerin gegenuber ein grosses Geschenk sein. Nur ein Rat: Sagen Sie ihr nicht, dass Sie es verwenden, das weibliche Selbstwertgefuhl ist genauso verletzlich wie das unsere. Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro - kostenlose, arztliche Telefon-Beratung - kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - keine versteckte Kosten - Visa verifizierter Onlineshop - diskrete Verpackung - diskrete Zahlung - kein peinlicher Arztbesuch erforderlich - bequem und diskret online bestellen. Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen http://hurryfeel.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Thu Mar 27 07:38:53 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 27 Mar 2008 14:38:53 +0000 Subject: [ofa-general] Re: [PATCH 4/4] opensm: remove not used osm_log_printf() function In-Reply-To: <1206623462.15600.39.camel@hrosenstock-ws.xsigo.com> References: <1206552426-17279-1-git-send-email-sashak@voltaire.com> <1206552426-17279-5-git-send-email-sashak@voltaire.com> <1206554847.8099.747.camel@hrosenstock-ws.xsigo.com> <20080326183742.GK13708@sashak.voltaire.com> <1206556102.8099.754.camel@hrosenstock-ws.xsigo.com> <20080327004148.GQ13708@sashak.voltaire.com> <1206623462.15600.39.camel@hrosenstock-ws.xsigo.com> Message-ID: <20080327143853.GD23849@sashak.voltaire.com> On 06:11 Thu 27 Mar , Hal Rosenstock wrote: > > It's been around since 11/06 (svn days) so that's several releases: So yes, it is only 1+ year. :) > This issue was initially the procedural issue of removing a public API > without deprecation warning for a release which is the norm. I'm pretty sure that in this case we can do exception due to obvious useless state of this function. If you disagree let me know - it is your call. Sasha From dotanb at dev.mellanox.co.il Thu Mar 27 07:36:00 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 27 Mar 2008 16:36:00 +0200 Subject: [ofa-general] [PATCH] mthca: update QP state after query QP Message-ID: <200803271636.00414.dotanb@dev.mellanox.co.il> Update the QP state after a successful query QP command was executed. If the QP was moved to other state (such as SQE) by the CA, the user won't have to set the IBV_QP_CUR_STATE mask in order to execute the modify QP in order to recover from this state. Signed-off-by: Dotan Barak --- diff --git a/drivers/infiniband/hw/mthca/mthca_qp.c b/drivers/infiniband/hw/mthca/mthca_qp.c index db5595b..c5ba9cd 100644 --- a/drivers/infiniband/hw/mthca/mthca_qp.c +++ b/drivers/infiniband/hw/mthca/mthca_qp.c @@ -437,29 +437,34 @@ int mthca_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_m int mthca_state; u8 status; + mutex_lock(&qp->mutex); + if (qp->state == IB_QPS_RESET) { qp_attr->qp_state = IB_QPS_RESET; goto done; } mailbox = mthca_alloc_mailbox(dev, GFP_KERNEL); - if (IS_ERR(mailbox)) - return PTR_ERR(mailbox); + if (IS_ERR(mailbox)) { + err = PTR_ERR(mailbox); + goto out; + } err = mthca_QUERY_QP(dev, qp->qpn, 0, mailbox, &status); if (err) - goto out; + goto out_mailbox; if (status) { mthca_warn(dev, "QUERY_QP returned status %02x\n", status); err = -EINVAL; - goto out; + goto out_mailbox; } qp_param = mailbox->buf; context = &qp_param->context; mthca_state = be32_to_cpu(context->flags) >> 28; - qp_attr->qp_state = to_ib_qp_state(mthca_state); + qp->state = to_ib_qp_state(mthca_state); + qp_attr->qp_state = qp->state; qp_attr->path_mtu = context->mtu_msgmax >> 5; qp_attr->path_mig_state = to_ib_mig_state((be32_to_cpu(context->flags) >> 11) & 0x3); @@ -506,8 +511,10 @@ done: qp_init_attr->cap = qp_attr->cap; -out: +out_mailbox: mthca_free_mailbox(dev, mailbox); +out: + mutex_unlock(&qp->mutex); return err; } From eli at dev.mellanox.co.il Thu Mar 27 07:45:53 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Thu, 27 Mar 2008 16:45:53 +0200 Subject: Fwd: [ofa-general] IPoIB : sendmsg: No buffer space available In-Reply-To: <309a667c0803270537h234a0065ib774b0d5392f9e63@mail.gmail.com> References: <309a667c0803262324n11838fbbrb5f010c2ac930273@mail.gmail.com> <1206605928.9065.17.camel@mtls03> <309a667c0803270331t230407ecifdcbd9dd40f70c6e@mail.gmail.com> <309a667c0803270353q1769c604rae3308bcfcfe7b2c@mail.gmail.com> <1206616955.9065.36.camel@mtls03> <309a667c0803270537h234a0065ib774b0d5392f9e63@mail.gmail.com> Message-ID: <1206629153.9065.41.camel@mtls03> OFED 1.3 is out already so please try it and see if you still have a problem. On Thu, 2008-03-27 at 18:07 +0530, Devesh Sharma wrote: > Its OFED-1.1 RHEL4-U4-x86_64, > How the send buffers are freed in IPoIB module? > Is there any configurable limit of socket buffers usage? or what is > the default number ? > any information or link will be helpful. > > > On Thu, Mar 27, 2008 at 4:52 PM, Eli Cohen wrote: > > On Thu, 2008-03-27 at 16:23 +0530, Devesh Sharma wrote: > > > Hello Eli, > > > Thanks for replying, > > > yes initially if I am ping'ing with 64 Bytes it pings successfully for > > > some 500 odd times after that this message starts flashing on my > > > screen. If I increase the data size number of itrations comes down. > > Which version are you using, is it ofed? Which OS? Please send any > > useful information and I'll try to reproduce. > > > > > > > > > > > > On Thu, Mar 27, 2008 at 1:48 PM, Eli Cohen wrote: > > > > On Thu, 2008-03-27 at 11:54 +0530, Devesh Sharma wrote: > > > > > > > > > I am receiving IPoIB : sendmsg: No buffer space available message > > > > > after some time > > > > > when I ping between two IB nodes. > > > > > what is the probable reason? > > > > > > > > > It means the socket buffer ping is using is exhausted. Did you have > > > > connectivity in the first place or you just try to ping forever with no > > > > success? > > > > > > > > > > > > > > From Thomas.Talpey at netapp.com Thu Mar 27 07:46:55 2008 From: Thomas.Talpey at netapp.com (Talpey, Thomas) Date: Thu, 27 Mar 2008 10:46:55 -0400 Subject: [ofa-general] Spam on mailing list general@openib.org Message-ID: I'm getting more than a little sick of the spam that's coming in through the old mail alias on openib.org. It has increased greatly in the past week. Can we either tighten up the protections there, or cut it off? In fact, does it make sense to move the entire list to vger, as is the norm with the other mainline kernel projects? Tom. From dpn at isomerica.net Thu Mar 27 07:56:32 2008 From: dpn at isomerica.net (Dan Noe) Date: Thu, 27 Mar 2008 10:56:32 -0400 Subject: [ofa-general] Spam on mailing list general@openib.org In-Reply-To: References: Message-ID: <47EBB5A0.6030000@isomerica.net> On 3/27/2008 10:46, Talpey, Thomas wrote: > I'm getting more than a little sick of the spam that's coming in through > the old mail alias on openib.org. It has increased greatly in the past week. > > Can we either tighten up the protections there, or cut it off? In fact, does > it make sense to move the entire list to vger, as is the norm with the other > mainline kernel projects? This list accounts for about 10% of the total spam I receive (which is a lot) and there has been a general uptick in spam since the beginning of the week. I think there is a new Windows virus or botnet in the wild since most of it is coming from residential ISPs. Spamprobe catches it, but I still need to browse that daily to make sure there are no false positives. I would vote for tightening the protections here. Cheers, Dan -- Dan Noe (dpn at lampreynetworks.com) Software Engineer Lamprey Networks, Inc. From dotanb at dev.mellanox.co.il Thu Mar 27 08:08:41 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 27 Mar 2008 17:08:41 +0200 Subject: [ofa-general] [PATCH] mlx4: update QP state after query QP Message-ID: <200803271708.41638.dotanb@dev.mellanox.co.il> Update the QP state after a successful query QP command was executed. If the QP was moved to other state (such as SQE) by the CA, the user won't have to set the IBV_QP_CUR_STATE mask in order to execute the modify QP in order to recover from this state. Signed-off-by: Dotan Barak --- diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 958e205..ce685cc 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -1725,7 +1725,9 @@ int mlx4_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr struct mlx4_ib_qp *qp = to_mqp(ibqp); struct mlx4_qp_context context; int mlx4_state; - int err; + int err = 0; + + mutex_lock(&qp->mutex); if (qp->state == IB_QPS_RESET) { qp_attr->qp_state = IB_QPS_RESET; @@ -1733,12 +1735,15 @@ int mlx4_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr } err = mlx4_qp_query(dev->dev, &qp->mqp, &context); - if (err) - return -EINVAL; + if (err) { + err = -EINVAL; + goto out; + } mlx4_state = be32_to_cpu(context.flags) >> 28; - qp_attr->qp_state = to_ib_qp_state(mlx4_state); + qp->state = to_ib_qp_state(mlx4_state); + qp_attr->qp_state = qp->state; qp_attr->path_mtu = context.mtu_msgmax >> 5; qp_attr->path_mig_state = to_ib_mig_state((be32_to_cpu(context.flags) >> 11) & 0x3); @@ -1796,7 +1801,8 @@ done: qp_attr->cap.max_inline_data = 0; qp_init_attr->cap = qp_attr->cap; - - return 0; +out: + mutex_unlock(&qp->mutex); + return err; } From dzieko at cefeid.wcss.wroc.pl Thu Mar 27 08:24:43 2008 From: dzieko at cefeid.wcss.wroc.pl (Pawel Dziekonski) Date: Thu, 27 Mar 2008 16:24:43 +0100 Subject: [ofa-general] a test - ignore pls Message-ID: <20080327152443.DB7B7C91FF6@cefeid.wcss.wroc.pl> Thu Mar 27 16:24:43 CET 2008 From dotanb at dev.mellanox.co.il Thu Mar 27 08:25:53 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 27 Mar 2008 17:25:53 +0200 Subject: [ofa-general] the port numbers in some of the rdmacm examples is a fixed value Message-ID: <47EBBC81.4030501@dev.mellanox.co.il> Hi Sean. It seems that some of the examples that comes with the librdmacm examples (cmatose, udaddy) have a fixed port number, i would like to send you a patch that allows changing the port number in the command line. Any objection? thanks Dotan From pawel.dziekonski at wcss.pl Thu Mar 27 08:27:20 2008 From: pawel.dziekonski at wcss.pl (Pawel Dziekonski) Date: Thu, 27 Mar 2008 16:27:20 +0100 Subject: [ofa-general] Spam on mailing list general@openib.org In-Reply-To: <47EBB5A0.6030000@isomerica.net> References: <47EBB5A0.6030000@isomerica.net> Message-ID: <20080327152720.GB24509@cefeid.wcss.wroc.pl> On Thu, 27 Mar 2008 at 10:56:32AM -0400, Dan Noe wrote: > On 3/27/2008 10:46, Talpey, Thomas wrote: >> I'm getting more than a little sick of the spam that's coming in through >> the old mail alias on openib.org. It has increased greatly in the past >> week. >> Can we either tighten up the protections there, or cut it off? In fact, >> does >> it make sense to move the entire list to vger, as is the norm with the >> other >> mainline kernel projects? > > This list accounts for about 10% of the total spam I receive (which is a > lot) and there has been a general uptick in spam since the beginning of the > week. I think there is a new Windows virus or botnet in the wild since > most of it is coming from residential ISPs. > > Spamprobe catches it, but I still need to browse that daily to make sure > there are no false positives. I would vote for tightening the protections > here. Hi, the problem is that anybody can write to the list. this has advantages, but allows spam to come. so just close access to registered people? cheers, P -- Pawel Dziekonski Wroclaw Centre for Networking & Supercomputing, HPC Department Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl From jlentini at netapp.com Thu Mar 27 08:30:42 2008 From: jlentini at netapp.com (James Lentini) Date: Thu, 27 Mar 2008 11:30:42 -0400 (EDT) Subject: [ofa-general] Spam on mailing list general@openib.org In-Reply-To: <20080327152720.GB24509@cefeid.wcss.wroc.pl> References: <47EBB5A0.6030000@isomerica.net> <20080327152720.GB24509@cefeid.wcss.wroc.pl> Message-ID: On Thu, 27 Mar 2008, Pawel Dziekonski wrote: > On Thu, 27 Mar 2008 at 10:56:32AM -0400, Dan Noe wrote: > > On 3/27/2008 10:46, Talpey, Thomas wrote: > >> I'm getting more than a little sick of the spam that's coming in through > >> the old mail alias on openib.org. It has increased greatly in the past > >> week. > >> Can we either tighten up the protections there, or cut it off? In fact, > >> does > >> it make sense to move the entire list to vger, as is the norm with the > >> other > >> mainline kernel projects? > > > > This list accounts for about 10% of the total spam I receive (which is a > > lot) and there has been a general uptick in spam since the beginning of the > > week. I think there is a new Windows virus or botnet in the wild since > > most of it is coming from residential ISPs. > > > > Spamprobe catches it, but I still need to browse that daily to make sure > > there are no false positives. I would vote for tightening the protections > > here. > > Hi, > > the problem is that anybody can write to the list. this has > advantages, but allows spam to come. so just close access to > registered people? We don't want a closed list. We've discussed this in the past on openfabrics general and openib general. From dpn at isomerica.net Thu Mar 27 08:33:31 2008 From: dpn at isomerica.net (Dan Noe) Date: Thu, 27 Mar 2008 11:33:31 -0400 Subject: [ofa-general] Spam on mailing list general@openib.org In-Reply-To: <20080327152720.GB24509@cefeid.wcss.wroc.pl> References: <47EBB5A0.6030000@isomerica.net> <20080327152720.GB24509@cefeid.wcss.wroc.pl> Message-ID: <47EBBE4B.5090706@isomerica.net> On 3/27/2008 11:27, Pawel Dziekonski wrote: > the problem is that anybody can write to the list. this has > advantages, but allows spam to come. so just close access to > registered people? Using mailman it is also possible to hold rejected mail for moderation, which allows you to catch people who might be sending legit mail from outside. Kinda gives you the best of both worlds. The drawback is that someone needs to moderate it... Also, you have the option of rejecting or dropping the messages. Rejecting sends a response, at the risk of causing backscatter spam. But at least then if someone legit who is not a list member posts to the list, it'll send them a message back telling them they need to join the list. Cheers, Dan -- Dan Noe (dpn at lampreynetworks.com) Software Engineer Lamprey Networks, Inc. From nfukc at boyutbilgi.com.tr Thu Mar 27 08:39:34 2008 From: nfukc at boyutbilgi.com.tr (Kareem Goldsmith) Date: Thu, 27 Mar 2008 16:39:34 +0100 Subject: [ofa-general] No weight - no problems Message-ID: <01c89029$2396bf80$e383084d@nfukc> Never late to try. Watch details attached and know more. -------------- next part -------------- A non-text attachment was scrubbed... Name: mp5.zip Type: application/zip Size: 579 bytes Desc: not available URL: From sean.hefty at intel.com Thu Mar 27 08:54:13 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 27 Mar 2008 08:54:13 -0700 Subject: [ofa-general] RE: the port numbers in some of the rdmacm examples is a fixed value In-Reply-To: <47EBBC81.4030501@dev.mellanox.co.il> References: <47EBBC81.4030501@dev.mellanox.co.il> Message-ID: <000101c89022$ce0b3d30$9c98070a@amr.corp.intel.com> >It seems that some of the examples that comes with the librdmacm >examples (cmatose, udaddy) have >a fixed port number, i would like to send you a patch that allows >changing the port number in the command line. > >Any objection? No objection at all -- just keep the defaults if one isn't provided. Thanks From olga.shern at gmail.com Thu Mar 27 09:16:18 2008 From: olga.shern at gmail.com (Olga Shern) Date: Thu, 27 Mar 2008 18:16:18 +0200 Subject: [ofa-general] Re: [ewg] OFED March 24 meeting summary on OFED 1.4 plans In-Reply-To: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> Message-ID: On Mon, Mar 24, 2008 at 10:45 PM, Tziporet Koren wrote: > > OFED March 24 meeting summary about OFED 1.4 and 1.3.1 plans: > > *1.3.1 Release:* > As we decided we should do a release in 2-3 month after 1.3. > In addition if there are any special fixes as outcome from the interop we > can do a release earlier. > All - please send me your requests for fixed issues and needed time frame > and I will publish 1.3.1 schedule based on this. > Hi Tziporet, Our plans for OFED 1.3.1 are 1. Fix issues related to SM failover: - packets lost in IPoIB - packets lost during bonding failover - some issues with multicast We haven't open bugs in bugzilla about this yet. 2. New bonding rpm with additional fixes - already done 3. We see again issues with "PKey table reordering stops ipoib traffic" - bug 420 4. IPoIB bug fix - multicast packets droped when calling set_multicats_list - already sent to upstream and was applied - I will send this patch to OFED. We would like that the following bugs will be fixed: 1. BUG 985 - IPoIB panic during openind stop 2. SDP kerenel panic bugs (971 , 969) Best Regards, Olga -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jeffrey.C.Becker at nasa.gov Thu Mar 27 09:44:52 2008 From: Jeffrey.C.Becker at nasa.gov (Jeff Becker) Date: Thu, 27 Mar 2008 09:44:52 -0700 Subject: [ofa-general] Spam on mailing list general@openib.org In-Reply-To: <47EBBE4B.5090706@isomerica.net> References: <47EBB5A0.6030000@isomerica.net> <20080327152720.GB24509@cefeid.wcss.wroc.pl> <47EBBE4B.5090706@isomerica.net> Message-ID: <47EBCF04.6040208@nasa.gov> Hi all. I tightened the SPAM level trigger on this list. Hopefully that'll improve things. -jeff Dan Noe wrote: > On 3/27/2008 11:27, Pawel Dziekonski wrote: >> the problem is that anybody can write to the list. this has >> advantages, but allows spam to come. so just close access to >> registered people? > > Using mailman it is also possible to hold rejected mail for > moderation, which allows you to catch people who might be sending > legit mail from outside. Kinda gives you the best of both worlds. > The drawback is that someone needs to moderate it... > > Also, you have the option of rejecting or dropping the messages. > Rejecting sends a response, at the risk of causing backscatter spam. > But at least then if someone legit who is not a list member posts to > the list, it'll send them a message back telling them they need to > join the list. > > Cheers, > Dan > From Brian.Murrell at Sun.COM Thu Mar 27 09:52:09 2008 From: Brian.Murrell at Sun.COM (Brian J. Murrell) Date: Thu, 27 Mar 2008 12:52:09 -0400 Subject: [ofa-general] determine release version of available source In-Reply-To: <1206574698.10075.290.camel@pc.ilinx> References: <1206574698.10075.290.camel@pc.ilinx> Message-ID: <1206636729.10075.327.camel@pc> On Wed, 2008-03-26 at 19:38 -0400, Brian J. Murrell wrote: > Hello, > > How can I determine the release version of a given set of OFED sources? > For example, looking in the SLES 10 2.6.16.54-0.2.3 source tree Further to this, perhaps I can narrow it down a bit. The sources seem to be missing rdma_cm.h and any mention of "rdma_cm_id" in them. Smells like it could be rather old. Is there a bonsai or browsable source repository where I can try to determine where this file was added? b. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From bullshit6 at dalex.de Thu Mar 27 10:46:16 2008 From: bullshit6 at dalex.de (Bonnie Kearney) Date: Thu, 27 Mar 2008 19:46:16 +0200 Subject: [ofa-general] Real sexual preparations which always work Message-ID: <455620227.57501361798688@dalex.de> exp cmz re jk ssher dj ba zg lsGa rd in An Am mmr azi iqp ng 1 to 3 fu tb ll In bi ch nb es! CLI zmj CK HE gk RE!!! -------------- next part -------------- An HTML attachment was scrubbed... URL: From christina.fjellstrom at hjart-lung.se Thu Mar 27 11:20:59 2008 From: christina.fjellstrom at hjart-lung.se (Parker Macias) Date: Thu, 27 Mar 2008 14:20:59 -0400 Subject: [ofa-general] MS Office 2007, AutoCAD 2008, Photoshop CS3 Message-ID: <01c89015$c7ac2600$1fa5dec9@christina.fjellstrom> Sie können unsere Software sofort runterladen! Zahlen Sie und die Downloads werden Ihnen sofort zur Verfügung stehen. Unsere Programme wurden für mehrere OS (Windows und Macintosh) programmiert und sind in allen europäischen Sprachen verfügbar. Wir verkaufen nur originale Vollversionen, haben aber die günstigsten Preise. Sie werden nach dem Kauf nicht alleine gelassen! Unsere freundlichen Kundenberater werden Ihnen bei der Installation helfen, falls Sie es benötigen. Schnelle Antworten und Geld-Zurück-Garantie sind bei uns selbstverständlich. Kaufen Sie die perfekt funktionierte Softwarehttp://geocities.com/theron_wilkinson/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ccornell at cooley.com Thu Mar 27 12:20:33 2008 From: ccornell at cooley.com (Rene Tucker) Date: Thu, 27 Mar 2008 22:20:33 +0300 Subject: [ofa-general] Entdecken Sie die Quelle fuer den Billigkauf Message-ID: <490495458.96395498067019@cooley.com> Sie bekommen unsere Software ohne Wartezeiten! Sofort nach der Bezahlung koennen Sie diese runterladen. Unsere Programme wurden nicht nur fuer Windows, sondern auch fuer Macintosh entwickelt und sind in allen europaeischen Sprachen verfuegbar. Wir haben sehr guenstige Preise und es handelt sich dabei nur um vollstaendige Originalversionen.Bei uns kaufen Sie sicher ein, denn unsere kompetenten Mitarbeiten vom Kundencenter werden Ihnen bei der Softwareinstallation weiterhelfen. Wir antworten unverzueglich und Sie bekommen von uns eine Geld-Zurueck-Garantie.Kaufen Sie die perfekt funktionierte Softwarehttp://geocities.com/buddydennis759/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From philip at cepia.net Thu Mar 27 13:04:28 2008 From: philip at cepia.net (Gordon Kish) Date: Thu, 27 Mar 2008 22:04:28 +0200 Subject: [ofa-general] Belebt Geist und Korper Message-ID: <01c89056$869bbe00$2fbd7b4d@philip> Online anonym bestellen - original Qualitat - 100% wirksam Fakten von unseren Kunden: - Sex ist befriedigender denn je. Stress und Leistungsdruck verschwinden. Sie ist nie wieder frustriert, ich habe keine Angst mehr zu versagen. Es ist ein wundervolles korperliches Erlebnis, dem ein genauso tiefes Gefuhl folgt. - Die Nebenwirkungen sind minimal: manchmal eine verstopfte Nase, kurzzeitig ein roter Kopf - kein Kopfschmerz, sondern das Gefuhl, als wurde man eine Flasche eiskalte Cola in einem Zug trinken. - Interessanterweise macht eine Vi. allein noch keinen Stander. Man(n) muss wenigstens ein bisschen Lust auf Sex mit der Frau haben. Gegen eine Eiserne Jungfrau im Bett hilft auch die grosste Dosis nichts. Wer aber das erste Kribbeln in den Lenden spurt, wird einen stahlharten Stander haben, und das fur wenigstens vier Stunden. - Eine volle 100-mg-Dosis macht den Schwanz zum Schwert. Wer es ubertreibt, ist Schuld, wenn die Herzallerliebste am Ende einen Y-formigen Sarg braucht. Fur die meisten von uns sind 50 mg mehr als genug, wenn man das gute Stuck zwischen den Hohepunkten auch mal hangen lassen will ... zur Not hilft es da vielleicht, sich ein nacktes Grossmutterchen vorzustellen. - Wer noch Zeit und Lust fur eine schnelle Nummer am nachsten Morgen hat, sollte dafur sorgen, dann noch genug Vi. im Blut zu haben - damit es noch fur ein oder zwei "Stehaufmannchen" reicht. - Das Beste an Vi. ist die Sicherheit, dass man "mit Autopilot fliegt", dass man entspannt und ohne Sorgen zur Sache kommen kann, dass der Stander auch halt, auch wenn man unterbrochen wird (die Kinder klopfen an die Schlafzimmertur, der Hund bellt, das Kondom sitzt schlecht). Wenn man Vi. bewusst anwendet, kann es auch der Partnerin gegenuber ein grosses Geschenk sein. Nur ein Rat: Sagen Sie ihr nicht, dass Sie es verwenden, das weibliche Selbstwertgefuhl ist genauso verletzlich wie das unsere. Spezialangebot: Vi. 10 Tab. 100 mg + Ci. 10 Tab. x 20 mg 53,82 Euro Vi. 10 Tab. 26,20 Euro Vi. 30 Tab. 51,97 Euro - Sie sparen: 27,00 Euro Vi. 60 Tab. 95,69 Euro - Sie sparen: 62,00 Euro Vi. 90 Tab. 136,91 Euro - Sie sparen: 100,00 Euro Ci. 10 - 30,00 Euro Ci. 20 - 59,35 Euro - Sie sparen: 2,00 Euro Ci. 30 - 80,30 Euro - Sie sparen: 12,00 Euro - keine versteckte Kosten - kein peinlicher Arztbesuch erforderlich - Visa verifizierter Onlineshop - kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - diskrete Zahlung - bequem und diskret online bestellen. - diskrete Verpackung - kostenlose, arztliche Telefon-Beratung Bestellen Sie jetzt und vergessen Sie Ihre Enttauschungen, anhaltende Versagensaengste und wiederholte peinliche Situationen http://designbefore.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From manny.danby at x-pressfragt.dk Thu Mar 27 13:13:00 2008 From: manny.danby at x-pressfragt.dk (Joesph Oneil) Date: Fri, 28 Mar 2008 00:13:00 +0400 Subject: [ofa-general] Very Strong Message-ID: <01c89068$7b518e00$f45bacd5@manny.danby> -------------- next part -------------- A non-text attachment was scrubbed... Name: it.gif Type: image/gif Size: 5876 bytes Desc: not available URL: From 95eqkchina.comchole at eqkchina.com Thu Mar 27 11:36:32 2008 From: 95eqkchina.comchole at eqkchina.com (dallis arnold) Date: Thu, 27 Mar 2008 18:36:32 +0000 Subject: [ofa-general] For Your Attention, Job Offer Message-ID: <000901c89048$0185013a$a9c9198f@qqxilmk> Opportunity From LUGANO Best Hotels Hello! Are you looking for an interesting job? Do you want to build perfect career with world leading team? Do you have a few available hours a day? Do you want to try part-time position from world leading tour operator? Bingo! We are glad to say that we are hiring. At this time we have various job openings for US citizens only. You do not need to have any tourism education and experiences, you do not need to relocate, you do not need to invest anything and we will cover all possible charges and fees. All you have to do at this time is just to read requirements and send us your CV or RESUME or just a few words about yourself for qualification. Our HR department will inform you with results very soon. Requirements: Strong computer skills (MS Office(MS Word 97/2003/2007) Any POP3 Mail Client) Good communication skills (Verbal and Written) Mobile phone (Just for Urgent Calls) High School, College, University (Any Education) Must Love People Send your resume/cv/profile to this address only: patrick.douson at gmail.com Best Places Best Guides SALARY: $3,500.00/month + % JOB TYPE: Part-Time/Employee PLACE OF WORK: Virtual Office ÷¿ Copyright 2004-2008 | Lugano Turismo -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ted.Kim at Sun.COM Thu Mar 27 13:42:23 2008 From: Ted.Kim at Sun.COM (Ted H. Kim) Date: Thu, 27 Mar 2008 13:42:23 -0700 Subject: [ofa-general] Re: files preamble In-Reply-To: <47E5EF49.9080506@voltaire.com> References: <47E5EF49.9080506@voltaire.com> Message-ID: <47EC06AF.8000309@sun.com> Or, I am not responding directly to Arkady's question, but aren't there some files with other licenses attached? For example, it appears addr.c, cma.c, ib_addr.h, rdma_cm.h and rdma_cm_ib.h -- all have the "Common Public License 1.0" mentioned. Are these special exceptions? Or were they just overlooked? Or is the idea of ONLY BSD and GPLv2 not a strict rule? -ted Or Gerlitz wrote: > Kanevsky, Arkady wrote: >> I noticed that all files in the preambule states that the file can be >> taken under GNU GPLv2 or >> OpenIB BSD licence. >> Should not it state OpenFabrics instead? > My understanding is that when ofa was founded it was decided to go on > this --specific-- dual license, what reasoning you see to change that? > > Or. -- Ted H. Kim Sun Microsystems, Inc. ted.kim at sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX From weiny2 at llnl.gov Thu Mar 27 14:35:34 2008 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 27 Mar 2008 14:35:34 -0700 Subject: [ofa-general] Re: [PATCH 1/6] complib/nodenamemap: add generic parse_node_map() function In-Reply-To: <20080327085429.GA23849@sashak.voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> <1206577637-18226-2-git-send-email-sashak@voltaire.com> <20080326175228.0cee3e5a.weiny2@llnl.gov> <20080327085429.GA23849@sashak.voltaire.com> Message-ID: <20080327143534.01150523.weiny2@llnl.gov> On Thu, 27 Mar 2008 08:54:29 +0000 Sasha Khapyorsky wrote: > Hi Ira, > > On 17:52 Wed 26 Mar , Ira Weiny wrote: > > > diff --git a/opensm/complib/cl_nodenamemap.c b/opensm/complib/cl_nodenamemap.c > > > index af5da39..696d455 100644 > > > --- a/opensm/complib/cl_nodenamemap.c > > > +++ b/opensm/complib/cl_nodenamemap.c > > > @@ -160,3 +160,48 @@ clean_nodedesc(char *nodedesc) > > > > > > return (nodedesc); > > > } > > > + > > > +int parse_node_map(const char *file_name, > > > + int (*create)(void *, uint64_t, char *), void *cxt) > > > > Why do you need this new way of parsing the file format? I think passing this > > callback makes the code more complicated that it needs to be. > > I don't think that it is much more complicated. OTOH it provides a great > flexibility (it is used in later patches). > > > Is there a reason not to use the nn_map_t as the type for root_nodes_list in > > the updn_t struct? > > If you will look at later updn patches you will find that there is no > any root nodes storage at all. The same is true for later up/down IDs > patch. Likely similar (intermediate storage free) improvement is > possible in fat-tree as well. Doesn't the root_nodes_list member of updn_t store the guids? I'm not quite sure what you are avoiding to store. It does not look as though you are searching the file each time the code wants to know if a particular guid is a root. In particular I don't want the node name lookup to search the file each time. That is why I read the file and store it in a map object. I suppose the callback method allows for users to use their own data structure object. > > > We could rename nn_map_t to be more generic if necessary. > > > > I worry about maintaining 2 functions which parse the same file format in the > > same c file. > > I agree with you, just didn't time to merge it. > Ok, however, I think there is a slight difference in the formats the 2 functions parse. I am looking for '"' symbols to delineate the name string. I don't think you are. For example: 0xcafebabe "hello world" Would result in a node guid of 0xcafebabe and a string of "hello world". I think your function would parse it as '"hello'? In fact including the '"'? Ira From info at prejud.com Thu Mar 27 14:43:29 2008 From: info at prejud.com (=?windows-1255?B?4ungIOXn7Ok=?=) Date: Thu, 27 Mar 2008 16:43:29 -0500 Subject: [ofa-general] =?windows-1255?b?4OnqIOnp+ODlIOfp6eog4fL66eM/?= Message-ID: <20080327214436.27BF2E60AF5@openfabrics.org> An HTML attachment was scrubbed... URL: From sean.hefty at intel.com Thu Mar 27 14:15:36 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 27 Mar 2008 14:15:36 -0700 Subject: [ofa-general] Re: files preamble In-Reply-To: <47EC06AF.8000309@sun.com> References: <47E5EF49.9080506@voltaire.com> <47EC06AF.8000309@sun.com> Message-ID: <002f01c8904f$b39e47d0$9c98070a@amr.corp.intel.com> >I am not responding directly to Arkady's question, but >aren't there some files with other licenses attached? > >For example, it appears addr.c, cma.c, ib_addr.h, rdma_cm.h >and rdma_cm_ib.h -- all have the "Common Public License 1.0" >mentioned. > >Are these special exceptions? >Or were they just overlooked? >Or is the idea of ONLY BSD and GPLv2 not a strict rule? I only checked the cma.c file, but it appears that Common Public License 1.0 has always been included. Not sure where that came from. All of the files mentioned likely picked up the same license text from copy-paste from each other. - Sean From akepner at sgi.com Thu Mar 27 14:44:33 2008 From: akepner at sgi.com (akepner at sgi.com) Date: Thu, 27 Mar 2008 14:44:33 -0700 Subject: [ofa-general] smpquery vlarb ... dumps only 1 entry? Message-ID: <20080327214433.GK16973@sgi.com> In ofed 1.2, "smpquery vlarb ... " would produce output like: # VLArbitration tables: Lid 8 port 0 LowCap 8 HighCap 8 # Low priority VL Arbitration Table: VL : |0x0 |0x1 |0x2 |0x3 |0x4 |0x5 |0x6 |0x7 | WEIGHT: |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 | # High priority VL Arbitration Table: VL : |0x0 |0x0 |0x0 |0x0 |0x1 |0x2 |0x3 |0x4 | WEIGHT: |0xFF|0xFF|0xFF|0xFF|0xFF|0x7F|0x3F|0x1F| which is what I expected. But in ofed 1.3, I only get a single entry: # VLArbitration tables: Lid 1 port 0 LowCap 8 HighCap 8 # Low priority VL Arbitration Table: VL : |0x3 | WEIGHT: |0x3 | # High priority VL Arbitration Table: VL : |0x3 | WEIGHT: |0x8 | Anyone know what changed here? -- Arthur From Arkady.Kanevsky at netapp.com Thu Mar 27 15:18:56 2008 From: Arkady.Kanevsky at netapp.com (Kanevsky, Arkady) Date: Thu, 27 Mar 2008 18:18:56 -0400 Subject: [ofa-general] Re: files preamble In-Reply-To: <47EC06AF.8000309@sun.com> References: <47E5EF49.9080506@voltaire.com> <47EC06AF.8000309@sun.com> Message-ID: Ted, my understanding that as long as we have BSD and GPLv2 licences we are fine. If some people would like to add more licences to choose from we are OK. By the way these files are based on DAT files which were triple licenced. By the fact preambule is still states OpenIB and not OpenFabric 2 years after OpenFabric name change is bad. Specifically, I am not sure if OpenIB BSD license exists. I think it is now OpenFabric BSD license. Thanks, Arkady Kanevsky email: arkady at netapp.com Network Appliance Inc. phone: 781-768-5395 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 Waltham, MA 02451 central phone: 781-768-5300 > -----Original Message----- > From: Ted H. Kim [mailto:Ted.Kim at Sun.COM] > Sent: Thursday, March 27, 2008 4:42 PM > To: openib-general at openib.org > Subject: [ofa-general] Re: files preamble > > Or, > > I am not responding directly to Arkady's question, but aren't > there some files with other licenses attached? > > For example, it appears addr.c, cma.c, ib_addr.h, rdma_cm.h > and rdma_cm_ib.h -- all have the "Common Public License 1.0" > mentioned. > > Are these special exceptions? > Or were they just overlooked? > Or is the idea of ONLY BSD and GPLv2 not a strict rule? > > -ted > > > Or Gerlitz wrote: > > Kanevsky, Arkady wrote: > >> I noticed that all files in the preambule states that the > file can be > >> taken under GNU GPLv2 or OpenIB BSD licence. > >> Should not it state OpenFabrics instead? > > My understanding is that when ofa was founded it was > decided to go on > > this --specific-- dual license, what reasoning you see to > change that? > > > > Or. > > > -- > Ted H. Kim > Sun Microsystems, Inc. ted.kim at sun.com > 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 > El Segundo, CA 90245 (310) 341-1120 FAX > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From Patrick.Delmonte at yahoo.com Thu Mar 27 15:56:14 2008 From: Patrick.Delmonte at yahoo.com (Patrick Delmonte) Date: Thu, 27 Mar 2008 14:56:14 -0800 Subject: [ofa-general] Contingent IT Recruitment Services - March 26, 2008 Message-ID: <12065315077a3eacbfe86e87b41d92db133ab85598@yahoo.com> Hi, I was just checking in with you and your company to see if I could assist with filling any of your curent IT positions. I have a strong team of recruiters that specialize exclusively in locating and recruiting qualified IT professionals and would love to work with you to help fill those openings! We work on both permanent direct-hire AND contract position and our fees are extremely reasonable - Most of our consultants have bill rates of 60-70/hr and our permanent fee ranges from 15-25%, depending on the difficulty of the position. A list of candidates/consultants that we currently have available include: IT Directors/Managers Project Managers Software Developers Engineers Analysts Architects Desktop Support Specialists Please email me back if we could assist with any current IT openings. Thanks! Patrick Delmonte Account Manager -------------- next part -------------- An HTML attachment was scrubbed... URL: From tbhainesd at ppresspub.com Thu Mar 27 16:56:01 2008 From: tbhainesd at ppresspub.com (University Degree) Date: Thu, 27 Mar 2008 16:56:01 -0700 Subject: [ofa-general] University Degree - Fast and Easy Message-ID: <680968170.05266250477432@ppresspub.com> 1-410-230-1849 University Degree These are real, genuine degrees that include Bachelors, Masters, MBA and Doctorate Degrees. Confidentiality Assured 1-410-230-1849 24 hours a day, 7 days a week including Sundays and Holidays -------------- next part -------------- An HTML attachment was scrubbed... URL: From teh_iroqois at rocketmail.com Thu Mar 27 18:30:48 2008 From: teh_iroqois at rocketmail.com (Bryce Ruffin) Date: Thu, 27 Mar 2008 20:30:48 -0500 Subject: [ofa-general] Your new way of life Message-ID: <01c89049$70d3cc00$827f9dbe@teh_iroqois> We proudly present you... What? Interested? Treatment to all THAT problems. Take a visit to our web portal at http://sadegikaz [DOT] com From cwcnn at cvmail.cl Thu Mar 27 17:48:48 2008 From: cwcnn at cvmail.cl (ethelred ching-li) Date: Fri, 28 Mar 2008 00:48:48 +0000 Subject: [ofa-general] Rihanna exposed Message-ID: <000501c8907c$020fb494$88d3c0b0@esjgercs> ANtsWznQuOY Download and WatchTsOFANtsWzn -------------- next part -------------- An HTML attachment was scrubbed... URL: From z_margaret.edmonds at jonmcrae.com Thu Mar 27 21:10:49 2008 From: z_margaret.edmonds at jonmcrae.com (EuroSoftware) Date: Fri, 28 Mar 2008 05:10:49 +0100 Subject: [ofa-general] All Possible Kinds of Software here Message-ID: <596082468.89374027142048@jonmcrae.com> Software range expansion-price downfallOur main purpose is to provide low price PC and Mac lawful software and computer solutions for anyone. Whether you are a corporate buyer, a holder of small-scale enterprise, or go shopping for your own PC, we guess that we can assist you.VIEW ALL PRODUCTS http://geocities.com/virgie.delacruz/Most demanding software are:* Microsoft Office Enterprise 2007: Retail price today - $599.95; Our just - $79.95 * Adobe Creative Suite 3 Web Premium: Retail price for now - $1599.95; Our only today - $219.95 * Adobe GoLive CS2: Retail price for now - $399.95; Our only - $49.95 * Autodesk VIZ 2008: Retail price this day - $1999.95; Our just - $149.95 * Adobe Dreamweaver CS3: Retail price for this time - $399.00; Our only - $59.95 * Office System Professional 2003 (5 Cds): Retail price this day - $469.95; Our only - $59.95 * Adobe After Effects CS3: Retail price for now - $999.95; Our just - $99.95 CHECK WHAT WE HAVE TO PROPOSE http://geocities.com/virgie.delacruz/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carolyn.hyden at fcps.edu Thu Mar 27 21:10:57 2008 From: carolyn.hyden at fcps.edu (EuroSoftware) Date: Fri, 28 Mar 2008 05:10:57 +0100 Subject: [ofa-general] All Software at the very Low Prices Message-ID: <272260631.66314742659162@fcps.edu> Just click to buy OEM. Best worldwide soft at increadeable pricesOur main purpose is to provide low price PC and Mac lawful software and computer solutions for anyone. Whether you are a corporate buyer, a holder of small-scale enterprise, or go shopping for your own PC, we guess that we can assist you.ENJOY OF OUR PRODUCTS http://geocities.com/tedmorris76/Most popular software in sight are:* Microsoft Office Enterprise 2007: Retail price for now - $599.95; Our only for today - $79.95 * Adobe Creative Suite 3 Design Premium: Retail price for this time - $1799.95; Our just - $229.95 * Adobe Encore DVD 2.0: Retail price for this time - $349.95; Our only today - $59.95 * Autodesk VIZ 2008: Retail price for now - $1999.95; Our only today - $149.95 * Adobe Fireworks CS3: Retail price this day - $299.00; Our only - $59.95 * Office System Professional 2003 (5 Cds): Retail price today - $469.95; Our now just - $59.95 * Adobe Premiere Pro 2: Retail price this day - $849.95; Our just - $79.95 VIEW ALL PRODUCTS http://geocities.com/tedmorris76/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Thu Mar 27 21:22:34 2008 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 27 Mar 2008 21:22:34 -0700 Subject: [ofa-general] QP connection healthy detection problem with fork/exec References: Message-ID: > > However in Rank 1, fork() is called, and parent exit(), the child call sleep for 5 minutes. But in rank 0, > The hear-beat message is always success untill I kill rank 2's child. > > Further, rank 1 calls fork() and exits, the child calls > execl("/bin/sleep", "sleep", "300", (char *)0); > > In rank 0, the heart-beat is still success untill I kill the 'sleep' process. > > It is easy to understand that if only fork() is called, the child will hold QP resources from parent, rank 0 can NOT detect > anything wrong. But if child calls exec, everything in rank 1 has been destroyed, why can't rank 0 detect the connection is broken ? I think the problem is that exec() does not clean everything up. AFAIK exec doesn't close open file descriptors so there's no way for the kernel uverbs module to know that it should clean up. - R. From dpn at lampreynetworks.com Thu Mar 27 21:38:46 2008 From: dpn at lampreynetworks.com (Daniel Noe) Date: Fri, 28 Mar 2008 00:38:46 -0400 Subject: [ofa-general] QP connection healthy detection problem with fork/exec In-Reply-To: References: Message-ID: <47EC7656.2090607@lampreynetworks.com> Roland Dreier wrote: > I think the problem is that exec() does not clean everything up. > AFAIK exec doesn't close open file descriptors so there's no way for > the kernel uverbs module to know that it should clean up. There is a file descriptor flag that can be set with fcntl(): FD_CLOEXEC If this flag is set on an fd, the kernel closes that file descriptor on execve(). Otherwise, the file descriptor remains open. Cheers, Dan -- Dan Noe (dpn at lampreynetworks.com) Software Engineer Lamprey Networks, Inc. From changquing.tang at hp.com Thu Mar 27 21:55:36 2008 From: changquing.tang at hp.com (Tang, Changqing) Date: Fri, 28 Mar 2008 04:55:36 +0000 Subject: [ofa-general] QP connection healthy detection problem with fork/exec In-Reply-To: <47EC7656.2090607@lampreynetworks.com> References: <47EC7656.2090607@lampreynetworks.com> Message-ID: When a process fork/exec, isn't QP destroyed automatically ? If the QP is destroyed, then RDMA operation is based on QP connection and should fail. Also how do I set FD_CLOEXEC ? which fd in which structure ? --CQ > -----Original Message----- > From: Daniel Noe [mailto:dpn at lampreynetworks.com] > Sent: Thursday, March 27, 2008 11:39 PM > To: Roland Dreier > Cc: Tang, Changqing; general at lists.openfabrics.org > Subject: Re: [ofa-general] QP connection healthy detection > problem with fork/exec > > Roland Dreier wrote: > > I think the problem is that exec() does not clean everything up. > > AFAIK exec doesn't close open file descriptors so there's > no way for > > the kernel uverbs module to know that it should clean up. > > There is a file descriptor flag that can be set with fcntl(): > FD_CLOEXEC > > If this flag is set on an fd, the kernel closes that file > descriptor on execve(). > > Otherwise, the file descriptor remains open. > > Cheers, > Dan > > -- > Dan Noe (dpn at lampreynetworks.com) > Software Engineer > Lamprey Networks, Inc. > From teensx99 at icqmail.com Thu Mar 27 22:06:45 2008 From: teensx99 at icqmail.com (Marisol Thomason) Date: Thu, 27 Mar 2008 21:06:45 -0800 Subject: [ofa-general] No weight - no problems Message-ID: <01c8904e$767fe080$8f88fede@teensx99> Never late to try. Watch details attached and know more. -------------- next part -------------- A non-text attachment was scrubbed... Name: mp5.zip Type: application/zip Size: 579 bytes Desc: not available URL: From dotanb at dev.mellanox.co.il Thu Mar 27 22:33:55 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Fri, 28 Mar 2008 08:33:55 +0300 Subject: [ofa-general] QP connection healthy detection problem with fork/exec In-Reply-To: References: <47EC7656.2090607@lampreynetworks.com> Message-ID: <47EC8343.80404@dev.mellanox.co.il> Hi. I didn't have a chance to check this but i believe that the following file descriptors need to be closed: struct ibv_context { struct ibv_device *device; struct ibv_context_ops ops; int cmd_fd; <-------- (i'm sure that this need is the one) int async_fd; <-------- (i think that you can skip this one....) int num_comp_vectors; pthread_mutex_t mutex; void *abi_compat; struct ibv_xrc_ops *xrc_ops; }; Dotan Tang, Changqing wrote: > When a process fork/exec, isn't QP destroyed automatically ? If the QP is destroyed, then > RDMA operation is based on QP connection and should fail. > > Also how do I set FD_CLOEXEC ? which fd in which structure ? > > > --CQ > > > >> -----Original Message----- >> From: Daniel Noe [mailto:dpn at lampreynetworks.com] >> Sent: Thursday, March 27, 2008 11:39 PM >> To: Roland Dreier >> Cc: Tang, Changqing; general at lists.openfabrics.org >> Subject: Re: [ofa-general] QP connection healthy detection >> problem with fork/exec >> >> Roland Dreier wrote: >> >>> I think the problem is that exec() does not clean everything up. >>> AFAIK exec doesn't close open file descriptors so there's >>> >> no way for >> >>> the kernel uverbs module to know that it should clean up. >>> >> There is a file descriptor flag that can be set with fcntl(): >> FD_CLOEXEC >> >> If this flag is set on an fd, the kernel closes that file >> descriptor on execve(). >> >> Otherwise, the file descriptor remains open. >> >> Cheers, >> Dan >> >> -- >> Dan Noe (dpn at lampreynetworks.com) >> Software Engineer >> Lamprey Networks, Inc. >> >> > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From millenaire3 at grandlyon.org Fri Mar 28 00:30:30 2008 From: millenaire3 at grandlyon.org (Bertie Downs) Date: Fri, 28 Mar 2008 08:30:30 +0100 Subject: [ofa-general] Entdecken Sie die Quelle fuer den Billigkauf Message-ID: <624142313.88856377656444@grandlyon.org> Bekommen Sie Ihre Software unverzueglich. Einfach zahlen und sofort runterladen. Hier sind Programme in allen europaeischen Sprachen verfuegbar, programmiert fuer Windows und Macintosh. Alle Softwaren sind sehr guenstig, es handelt sich dabei garantiert um originale, komplette und voellig funktionale Versionen.Unsere Kundenberater sind immer bereit Ihnen bei der Installation zu helfen. Wir antworten sehr schnell und geben Ihnen auch Geld-Zurueck-Garantie!Sie bekommen bei uns nur die ausgezeichnete Softwarehttp://soptesof.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ZON487 at mayridge.com Fri Mar 28 01:33:50 2008 From: ZON487 at mayridge.com (Louis Nash) Date: Fri, 28 Mar 2008 09:33:50 +0100 Subject: [ofa-general] =?koi8-r?b?89fPyiDGz87EINrBINLVwsXWz80=?= Message-ID: <01c890b6$d4767600$25b1b84f@ZON487> Добрый день, Юрий! Ознакомьтесь: -- Способы и рекомендации в использовании учредительных документов общества для защиты от враждебного поглощения, корпоративного шантажа и воздействия на руководителей и акционеров; Страхование рисков враждебного поглощения и корпоративного шантажа - российская практика. Весь перечень интересных вопросов и решений, о которых Вы хотели знать, пожалуйста, смотрите на: http://lairgiusep.tripod.com С уважением, Сергей. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nibe at sun-star-st.jp Fri Mar 28 02:39:59 2008 From: nibe at sun-star-st.jp (inness sinclair) Date: Fri, 28 Mar 2008 09:39:59 +0000 Subject: [ofa-general] New Britney P*ssy shot Message-ID: <000601c890c6$06774f9d$779183ae@sdwhsxjg> XTgPFTYsXs Download and WatchMWCgQXTgPF -------------- next part -------------- An HTML attachment was scrubbed... URL: From hgldft at brajamusic.com Fri Mar 28 06:07:29 2008 From: hgldft at brajamusic.com (Omar Lynch) Date: Fri, 28 Mar 2008 22:07:29 +0900 Subject: [ofa-general] Be like sex machine Message-ID: <01c89120$1ceb9000$a44f637b@hgldft> Be damn goood in it Please look attached file and know MORE ABOUT THIS! -------------- next part -------------- A non-text attachment was scrubbed... Name: cvv2.zip Type: application/zip Size: 636 bytes Desc: not available URL: From ytvcel at bovkun.com Fri Mar 28 06:29:47 2008 From: ytvcel at bovkun.com (Eleanor Levine) Date: Fri, 28 Mar 2008 15:29:47 +0200 Subject: [ofa-general] No weight - no problems Message-ID: <01c890e8$8e0ba780$8317ea58@ytvcel> Never late to try. Watch details attached and know more. -------------- next part -------------- A non-text attachment was scrubbed... Name: mp5.zip Type: application/zip Size: 579 bytes Desc: not available URL: From Brian.Murrell at Sun.COM Fri Mar 28 08:25:27 2008 From: Brian.Murrell at Sun.COM (Brian J. Murrell) Date: Fri, 28 Mar 2008 11:25:27 -0400 Subject: [ofa-general] KSRC vs. K_SRC in install.pl? Message-ID: <1206717927.25531.19.camel@pc.ilinx> I'm looking into the build process for the various OFED RPMs and I'm seeing that install.pl builds the ofa_kernel RPM with "--define 'K_SRC $kernel_sources'" however looking in the ofa_kernel.spec I see: %define K_SRC /lib/modules/%{KVERSION}/build %{!?KSRC: %define KSRC %{K_SRC}} These are in fact the only two places where K_SRC is even used. Doesn't the "%define K_SRC" make the K_SRC set by install.pl moot? Should the install.pl perhaps be defining KSRC, not K_SRC? b. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From Thomas.Talpey at netapp.com Fri Mar 28 09:01:46 2008 From: Thomas.Talpey at netapp.com (Talpey, Thomas) Date: Fri, 28 Mar 2008 12:01:46 -0400 Subject: [ofa-general] Spam on mailing list general@openib.org In-Reply-To: <47EBCF04.6040208@nasa.gov> References: <47EBB5A0.6030000@isomerica.net> <20080327152720.GB24509@cefeid.wcss.wroc.pl> <47EBBE4B.5090706@isomerica.net> <47EBCF04.6040208@nasa.gov> Message-ID: At 12:44 PM 3/27/2008, Jeff Becker wrote: >Hi all. I tightened the SPAM level trigger on this list. Hopefully >that'll improve things. I'm sure you've noticed this didn't work very well. They got raunchier, in fact. I'll note again that they are entering via the old openib.org lists, not the current openfabrics. Perhaps the filter doesn't apply to the internal redirection? Tom. From divisas1234 at gmail.com Fri Mar 28 09:16:03 2008 From: divisas1234 at gmail.com (divisas1234) Date: Fri, 28 Mar 2008 11:16:03 -0500 Subject: [ofa-general] El secreto ... atraccion Message-ID: <65750-22008352816163875@MAGO> El Secreto financiero Atraccion de dinero La ley de atraccion Atraer dinero Atraer riqueza Atraer utilidades CON SU PENSAMIENTO Y SU INVERSION CON NOSOTROS, ATRAE UTILIDADES Estos son los resultados Marzo 2008 utilidad cliente 22% Febrero 2008 utilidad Cliente 60.4% Enero 2008 utilidad cliente 19.8% Diciembre 2007 utilidad cliente 13.7% Este es el resultado del a�o 2007: Inicio con $5.000 y termino con $15.000 Utilidad $10.371 dolares Month Pips Profit Balance Start -- -- 5,000 Jan 2007 -92 -395 4,605 Feb 2007 460 2,237 6,842 Mar 2007 -137 -717 6,125 Apr 2007 504 2,612 8,737 May 2007 169 735 9,472 Jun 2007 546 2,642 12,114 Jul 2007 503 2,552 14,666 Aug 2007 -321 -1,600 13,066 Sep 2007 559 2,795 15,861 Oct 2007 207 978 16,839 Nov 2007 -457 -2,273 14,566 Dec 2007 185 805 15,371 Total 2,126 10,371 15,371 Como solicita informacion Divisas123 at gmail.com From rdreier at cisco.com Fri Mar 28 10:28:17 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 10:28:17 -0700 Subject: [ofa-general] [PATCH for 2.6.25] RDMA/cxgb3: Program hardware IRD with correct value Message-ID: Because of a typo in iwch_accept_cr(), the cxgb3 connection handling code programs the hardware IRD (incoming RDMA read queue depth) with the value that is passed in for the ORD (outgoing RDMA read queue depth). In particular this means that if an application passes in IRD > 0 and ORD = 0 (which is a completely sane and valid thing to do for an app that expects only incoming RDMA read requests), then the hardware will end up programmed with IRD = 0 and the app will fail in a mysterious way. Fix this by using "ep->ird" instead of "ep->ord" in the intended place. Signed-off-by: Roland Dreier Acked-by: Steve Wise --- Linus, please add this for 2.6.25 if possible -- it's a trivial, "obviously correct" fix that is very low risk and fixes real apps. Thanks, Roland diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c index 320f2b6..99f2f2a 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_cm.c +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c @@ -1745,7 +1745,7 @@ int iwch_accept_cr(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param) /* bind QP to EP and move to RTS */ attrs.mpa_attr = ep->mpa_attr; - attrs.max_ird = ep->ord; + attrs.max_ird = ep->ird; attrs.max_ord = ep->ord; attrs.llp_stream_handle = ep; attrs.next_state = IWCH_QP_STATE_RTS; From rdreier at cisco.com Fri Mar 28 10:33:28 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 10:33:28 -0700 Subject: [ofa-general] QP connection healthy detection problem with fork/exec In-Reply-To: (Changqing Tang's message of "Fri, 28 Mar 2008 04:55:36 +0000") References: <47EC7656.2090607@lampreynetworks.com> Message-ID: > When a process fork/exec, isn't QP destroyed automatically ? If the QP is destroyed, then > RDMA operation is based on QP connection and should fail. No, the QP is not destroyed automatically. The kernel will clean up resources when the user verbs context's file descriptor is closed, and that does not happen on fork/exec. A safe thing to do when you fork/exec is to close any file descriptors that you don't need... you can use a loop up to getdtablesize() to close everything. - R. From phil.newcombe at pagemoy.com Fri Mar 28 09:58:19 2008 From: phil.newcombe at pagemoy.com (dmitri eka) Date: Fri, 28 Mar 2008 16:58:19 +0000 Subject: [ofa-general] Authorised Dealer 3000 Styles Great Prices Message-ID: <000601c89103$04b9bc3e$20b53c8b@siftlqc> Timepieces Rolex Watches Online http://rheumdakragas.com/ From HNGUYEN at de.ibm.com Fri Mar 28 11:53:07 2008 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Fri, 28 Mar 2008 19:53:07 +0100 Subject: [ofa-general] Re: IB/core: Add creation flags to QPs In-Reply-To: <47EB436E.50908@voltaire.com> Message-ID: Hello Roland! > Jack Morgenstein wrote: > > Actually, we did not break the ABI for XRC (we had to do cartwheels to avoid > > this, but what the heck). > > .... > > You can avoid doing this in one of 2 ways: > > Either: create a new QP type > > Or: add new verbs to the core. > Jack - OK, got it, and thanks a lot! for the coaching. > > Roland - in the case of the connectx block loopback feature, it seems > that the simplest way to go would for user space is to create a new QP > type IB_QPT_UD_BL[block-loopback] which is translated by the uverbs > layer to what ever scheme we decide on for the kernel (eg create_flags, > qp_types, bit fields, etc). As for IPoIB, it would create a UD/LSO/BL > QP, thoughts? What is your recommendation wrt/ encoding scheme for qp_type and create_flags? For ehca we need user space support for LL QP. And we can implement that as a flag (my favorite) or as two additional qp_types (IB_QPT_UD_LL and IB_QPT_RC_LL). Anyway I'm flexible in whatever approach you choose. PS: See also this thread http://lists.openfabrics.org/pipermail/general/2008-March/048445.html Thanks! Nam From sean.hefty at intel.com Fri Mar 28 11:57:58 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Fri, 28 Mar 2008 11:57:58 -0700 Subject: [ofa-general] [ANNOUNCE] librdmacm release 1.0.7 Message-ID: <000001c89105$a3da0ee0$b137170a@amr.corp.intel.com> I've published librdmacm release 1.0.7. It contains the following additions over 1.0.6: Add support for rdma_migrate_id(). Kernel support will be in 2.6.25. Several build fixes from Roland. Set reject status correctly. Please pull this package for OFED 1.3.1. The reject status fix has been requested for DAPL. - Sean From rdreier at cisco.com Fri Mar 28 12:25:02 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 12:25:02 -0700 Subject: [ofa-general] Re: [PATCH 6/6] IB/ipath - eeprom support for 7220 devices, robustness improvements, cleanup In-Reply-To: <20080325180638.27615.7220.stgit@eng-46.mv.qlogic.com> (Ralph Campbell's message of "Tue, 25 Mar 2008 11:06:38 -0700") References: <20080325180607.27615.58003.stgit@eng-46.mv.qlogic.com> <20080325180638.27615.7220.stgit@eng-46.mv.qlogic.com> Message-ID: thanks, applied 1-6 From accounts at southwestair.com.pg Fri Mar 28 11:01:12 2008 From: accounts at southwestair.com.pg (flemming joe) Date: Fri, 28 Mar 2008 18:01:12 +0000 Subject: [ofa-general] Cialys, Viagra and Levytra at Bargain Prices - We Have It All! openib-general's discount. Message-ID: <000a01c8910c$01dd7547$79da8885@pelyumjh> We always been known for good support, timely dispatches and Immediate feedback. You will save Your 89%. #PQBJSlYour Low Cost Online Source Of: Quality Brand Name & Generic Drugs; Herbal & Dietary Supplements.Save Money on Name Brand Drugs..VuIvYcVPWeight loss, Men's Health, Women's Health, Sexual Health Affordable, Safe and Convenient Next Day..JSlxHhVWQuickly we will ship Your medicatoins. -------------- next part -------------- An HTML attachment was scrubbed... URL: From changquing.tang at hp.com Fri Mar 28 12:58:12 2008 From: changquing.tang at hp.com (Tang, Changqing) Date: Fri, 28 Mar 2008 19:58:12 +0000 Subject: [ofa-general] QP connection healthy detection problem with fork/exec In-Reply-To: References: <47EC7656.2090607@lampreynetworks.com> Message-ID: Thanks. Closing all unused fd except 0,1,2 is our current solution. --CQ > -----Original Message----- > From: Roland Dreier [mailto:rdreier at cisco.com] > Sent: Friday, March 28, 2008 12:33 PM > To: Tang, Changqing > Cc: dpn at lampreynetworks.com; general at lists.openfabrics.org > Subject: Re: [ofa-general] QP connection healthy detection > problem with fork/exec > > > When a process fork/exec, isn't QP destroyed automatically > ? If the QP is destroyed, then > RDMA operation is based on > QP connection and should fail. > > No, the QP is not destroyed automatically. > > The kernel will clean up resources when the user verbs > context's file descriptor is closed, and that does not happen > on fork/exec. > > A safe thing to do when you fork/exec is to close any file > descriptors that you don't need... you can use a loop up to > getdtablesize() to close everything. > > - R. > From mvislkpasttg at brainbold.com Fri Mar 28 13:24:06 2008 From: mvislkpasttg at brainbold.com (Lincoln Kearney) Date: Fri, 28 Mar 2008 16:24:06 -0400 Subject: [ofa-general] The Cure Message-ID: <01c890f0$24982280$20095ebe@mvislkpasttg> We proudly present you... What? Interested? Treatment to all THAT problems. Take a visit to our web portal at http://sadegikaz?com From rdreier at cisco.com Fri Mar 28 14:32:54 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 14:32:54 -0700 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: <1205767427.25950.137.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:47 +0200") References: <1205767427.25950.137.camel@mtls03> Message-ID: Thanks, I applied this with some extra code in all the low-level drivers to make sure that the create_flags are passed in as 0. Does that make sense to everyone? >From 3835906cb5d2477521b88e7974153505a3d3fdb6 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Mon, 17 Mar 2008 17:23:47 +0200 Subject: [PATCH] IB/core: Add creation flags to struct ib_qp_init_attr Add a create_flags member to struct ib_qp_init_attr that will allow a kernel verbs consumer to create a pass special flags when creating a QP. Add a flag value for telling low-level drivers that a QP will be used for IPoIB UD LSO. The create_flags member will also be useful for XRC and ehca low-latency QP support. Since no create_flags handling is implemented yet, add code to all low-level drivers to return -EINVAL if create_flags is non-zero. Signed-off-by: Eli Cohen Signed-off-by: Roland Dreier --- drivers/infiniband/core/uverbs_cmd.c | 1 + drivers/infiniband/hw/amso1100/c2_provider.c | 3 +++ drivers/infiniband/hw/ehca/ehca_qp.c | 3 +++ drivers/infiniband/hw/ipath/ipath_qp.c | 5 +++++ drivers/infiniband/hw/mlx4/qp.c | 3 +++ drivers/infiniband/hw/mthca/mthca_provider.c | 3 +++ drivers/infiniband/hw/nes/nes_verbs.c | 3 +++ include/rdma/ib_verbs.h | 5 +++++ 8 files changed, 26 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 238680c..8767192 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1065,6 +1065,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, attr.srq = srq; attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; attr.qp_type = cmd.qp_type; + attr.create_flags = 0; attr.cap.max_send_wr = cmd.max_send_wr; attr.cap.max_recv_wr = cmd.max_recv_wr; diff --git a/drivers/infiniband/hw/amso1100/c2_provider.c b/drivers/infiniband/hw/amso1100/c2_provider.c index 21fb46d..e10d27a 100644 --- a/drivers/infiniband/hw/amso1100/c2_provider.c +++ b/drivers/infiniband/hw/amso1100/c2_provider.c @@ -245,6 +245,9 @@ static struct ib_qp *c2_create_qp(struct ib_pd *pd, pr_debug("%s:%u\n", __func__, __LINE__); + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + switch (init_attr->qp_type) { case IB_QPT_RC: qp = kzalloc(sizeof(*qp), GFP_KERNEL); diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index a9fd419..3eb14a5 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -421,6 +421,9 @@ static struct ehca_qp *internal_create_qp( u32 swqe_size = 0, rwqe_size = 0, ib_qp_num; unsigned long flags; + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + memset(&parms, 0, sizeof(parms)); qp_type = init_attr->qp_type; diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 6a4a5e3..812b42c 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -747,6 +747,11 @@ struct ib_qp *ipath_create_qp(struct ib_pd *ibpd, size_t sz; struct ib_qp *ret; + if (init_attr->create_flags) { + ret = ERR_PTR(-EINVAL); + goto bail; + } + if (init_attr->cap.max_send_sge > ib_ipath_max_sges || init_attr->cap.max_send_wr > ib_ipath_max_qp_wrs) { ret = ERR_PTR(-EINVAL); diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 31b2b5b..320c25f 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -673,6 +673,9 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, struct mlx4_ib_qp *qp; int err; + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + switch (init_attr->qp_type) { case IB_QPT_RC: case IB_QPT_UC: diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c index ee9bc14..81b257e 100644 --- a/drivers/infiniband/hw/mthca/mthca_provider.c +++ b/drivers/infiniband/hw/mthca/mthca_provider.c @@ -540,6 +540,9 @@ static struct ib_qp *mthca_create_qp(struct ib_pd *pd, struct mthca_qp *qp; int err; + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + switch (init_attr->qp_type) { case IB_QPT_RC: case IB_QPT_UC: diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c index 46b3b8e..7c27420 100644 --- a/drivers/infiniband/hw/nes/nes_verbs.c +++ b/drivers/infiniband/hw/nes/nes_verbs.c @@ -1252,6 +1252,9 @@ static struct ib_qp *nes_create_qp(struct ib_pd *ibpd, u8 rq_encoded_size; /* int counter; */ + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + atomic_inc(&qps_created); switch (init_attr->qp_type) { case IB_QPT_RC: diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 51fc1f7..3ac7371 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -496,6 +496,10 @@ enum ib_qp_type { IB_QPT_RAW_ETY }; +enum ib_qp_create_flags { + IB_QP_CREATE_IPOIB_UD_LSO = 1 << 0, +}; + struct ib_qp_init_attr { void (*event_handler)(struct ib_event *, void *); void *qp_context; @@ -505,6 +509,7 @@ struct ib_qp_init_attr { struct ib_qp_cap cap; enum ib_sig_type sq_sig_type; enum ib_qp_type qp_type; + enum ib_qp_create_flags create_flags; u8 port_num; /* special QP types only */ }; -- 1.5.4.3 From rdreier at cisco.com Fri Mar 28 14:32:54 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 14:32:54 -0700 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: <1205767427.25950.137.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:47 +0200") References: <1205767427.25950.137.camel@mtls03> Message-ID: Thanks, I applied this with some extra code in all the low-level drivers to make sure that the create_flags are passed in as 0. Does that make sense to everyone? >From 3835906cb5d2477521b88e7974153505a3d3fdb6 Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Mon, 17 Mar 2008 17:23:47 +0200 Subject: [PATCH] IB/core: Add creation flags to struct ib_qp_init_attr Add a create_flags member to struct ib_qp_init_attr that will allow a kernel verbs consumer to create a pass special flags when creating a QP. Add a flag value for telling low-level drivers that a QP will be used for IPoIB UD LSO. The create_flags member will also be useful for XRC and ehca low-latency QP support. Since no create_flags handling is implemented yet, add code to all low-level drivers to return -EINVAL if create_flags is non-zero. Signed-off-by: Eli Cohen Signed-off-by: Roland Dreier --- drivers/infiniband/core/uverbs_cmd.c | 1 + drivers/infiniband/hw/amso1100/c2_provider.c | 3 +++ drivers/infiniband/hw/ehca/ehca_qp.c | 3 +++ drivers/infiniband/hw/ipath/ipath_qp.c | 5 +++++ drivers/infiniband/hw/mlx4/qp.c | 3 +++ drivers/infiniband/hw/mthca/mthca_provider.c | 3 +++ drivers/infiniband/hw/nes/nes_verbs.c | 3 +++ include/rdma/ib_verbs.h | 5 +++++ 8 files changed, 26 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 238680c..8767192 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1065,6 +1065,7 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file, attr.srq = srq; attr.sq_sig_type = cmd.sq_sig_all ? IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR; attr.qp_type = cmd.qp_type; + attr.create_flags = 0; attr.cap.max_send_wr = cmd.max_send_wr; attr.cap.max_recv_wr = cmd.max_recv_wr; diff --git a/drivers/infiniband/hw/amso1100/c2_provider.c b/drivers/infiniband/hw/amso1100/c2_provider.c index 21fb46d..e10d27a 100644 --- a/drivers/infiniband/hw/amso1100/c2_provider.c +++ b/drivers/infiniband/hw/amso1100/c2_provider.c @@ -245,6 +245,9 @@ static struct ib_qp *c2_create_qp(struct ib_pd *pd, pr_debug("%s:%u\n", __func__, __LINE__); + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + switch (init_attr->qp_type) { case IB_QPT_RC: qp = kzalloc(sizeof(*qp), GFP_KERNEL); diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index a9fd419..3eb14a5 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -421,6 +421,9 @@ static struct ehca_qp *internal_create_qp( u32 swqe_size = 0, rwqe_size = 0, ib_qp_num; unsigned long flags; + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + memset(&parms, 0, sizeof(parms)); qp_type = init_attr->qp_type; diff --git a/drivers/infiniband/hw/ipath/ipath_qp.c b/drivers/infiniband/hw/ipath/ipath_qp.c index 6a4a5e3..812b42c 100644 --- a/drivers/infiniband/hw/ipath/ipath_qp.c +++ b/drivers/infiniband/hw/ipath/ipath_qp.c @@ -747,6 +747,11 @@ struct ib_qp *ipath_create_qp(struct ib_pd *ibpd, size_t sz; struct ib_qp *ret; + if (init_attr->create_flags) { + ret = ERR_PTR(-EINVAL); + goto bail; + } + if (init_attr->cap.max_send_sge > ib_ipath_max_sges || init_attr->cap.max_send_wr > ib_ipath_max_qp_wrs) { ret = ERR_PTR(-EINVAL); diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index 31b2b5b..320c25f 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -673,6 +673,9 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd, struct mlx4_ib_qp *qp; int err; + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + switch (init_attr->qp_type) { case IB_QPT_RC: case IB_QPT_UC: diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c index ee9bc14..81b257e 100644 --- a/drivers/infiniband/hw/mthca/mthca_provider.c +++ b/drivers/infiniband/hw/mthca/mthca_provider.c @@ -540,6 +540,9 @@ static struct ib_qp *mthca_create_qp(struct ib_pd *pd, struct mthca_qp *qp; int err; + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + switch (init_attr->qp_type) { case IB_QPT_RC: case IB_QPT_UC: diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c index 46b3b8e..7c27420 100644 --- a/drivers/infiniband/hw/nes/nes_verbs.c +++ b/drivers/infiniband/hw/nes/nes_verbs.c @@ -1252,6 +1252,9 @@ static struct ib_qp *nes_create_qp(struct ib_pd *ibpd, u8 rq_encoded_size; /* int counter; */ + if (init_attr->create_flags) + return ERR_PTR(-EINVAL); + atomic_inc(&qps_created); switch (init_attr->qp_type) { case IB_QPT_RC: diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 51fc1f7..3ac7371 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -496,6 +496,10 @@ enum ib_qp_type { IB_QPT_RAW_ETY }; +enum ib_qp_create_flags { + IB_QP_CREATE_IPOIB_UD_LSO = 1 << 0, +}; + struct ib_qp_init_attr { void (*event_handler)(struct ib_event *, void *); void *qp_context; @@ -505,6 +509,7 @@ struct ib_qp_init_attr { struct ib_qp_cap cap; enum ib_sig_type sq_sig_type; enum ib_qp_type qp_type; + enum ib_qp_create_flags create_flags; u8 port_num; /* special QP types only */ }; -- 1.5.4.3 From rdreier at cisco.com Fri Mar 28 14:39:37 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 14:39:37 -0700 Subject: [ofa-general] Re: [PATCH 3/10] IB/core: Add LSO support In-Reply-To: <1205767431.25950.138.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:51 +0200") References: <1205767431.25950.138.camel@mtls03> Message-ID: thanks, applied as below. For now I left the IB_WR_LSO opcode rather than a send flag, since the mlx4 internal implementation is as a new opcode. However since this is kernel-internal we can revisit this and I'm happy if the discussion continues. >From 86a0dd93c39739a39d6b5f7f67d4b2456c5f45ae Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Mon, 17 Mar 2008 17:23:51 +0200 Subject: [PATCH] IB/core: Add IPoIB UD LSO support LSO (large send offload) allows the networking stack to pass SKBs with data size larger than the MTU to the IPoIB driver and have the HCA HW fragment the data to multiple MSS-sized packets. Add a device capability flag IB_DEVICE_UD_TSO for devices that can perform TCP segmentation offload, a new send work request opcode IB_WR_LSO, header, hlen and mss fields for the work request structure, and a new IB_WC_LSO completion type. Signed-off-by: Eli Cohen Signed-off-by: Roland Dreier --- include/rdma/ib_verbs.h | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 3ac7371..5fe7723 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -104,6 +104,7 @@ enum ib_device_cap_flags { * IPoIB driver may set NETIF_F_IP_CSUM for datagram mode. */ IB_DEVICE_UD_IP_CSUM = (1<<18), + IB_DEVICE_UD_TSO = (1<<19), IB_DEVICE_SEND_W_INV = (1<<21), }; @@ -412,6 +413,7 @@ enum ib_wc_opcode { IB_WC_COMP_SWAP, IB_WC_FETCH_ADD, IB_WC_BIND_MW, + IB_WC_LSO, /* * Set value of IB_WC_RECV so consumers can test if a completion is a * receive by testing (opcode & IB_WC_RECV). @@ -623,7 +625,8 @@ enum ib_wr_opcode { IB_WR_SEND_WITH_IMM, IB_WR_RDMA_READ, IB_WR_ATOMIC_CMP_AND_SWP, - IB_WR_ATOMIC_FETCH_AND_ADD + IB_WR_ATOMIC_FETCH_AND_ADD, + IB_WR_LSO }; enum ib_send_flags { @@ -662,6 +665,9 @@ struct ib_send_wr { } atomic; struct { struct ib_ah *ah; + void *header; + int hlen; + int mss; u32 remote_qpn; u32 remote_qkey; u16 pkey_index; /* valid for GSI only */ -- 1.5.4.3 From rdreier at cisco.com Fri Mar 28 14:39:37 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 14:39:37 -0700 Subject: [ofa-general] Re: [PATCH 3/10] IB/core: Add LSO support In-Reply-To: <1205767431.25950.138.camel@mtls03> (Eli Cohen's message of "Mon, 17 Mar 2008 17:23:51 +0200") References: <1205767431.25950.138.camel@mtls03> Message-ID: thanks, applied as below. For now I left the IB_WR_LSO opcode rather than a send flag, since the mlx4 internal implementation is as a new opcode. However since this is kernel-internal we can revisit this and I'm happy if the discussion continues. >From 86a0dd93c39739a39d6b5f7f67d4b2456c5f45ae Mon Sep 17 00:00:00 2001 From: Eli Cohen Date: Mon, 17 Mar 2008 17:23:51 +0200 Subject: [PATCH] IB/core: Add IPoIB UD LSO support LSO (large send offload) allows the networking stack to pass SKBs with data size larger than the MTU to the IPoIB driver and have the HCA HW fragment the data to multiple MSS-sized packets. Add a device capability flag IB_DEVICE_UD_TSO for devices that can perform TCP segmentation offload, a new send work request opcode IB_WR_LSO, header, hlen and mss fields for the work request structure, and a new IB_WC_LSO completion type. Signed-off-by: Eli Cohen Signed-off-by: Roland Dreier --- include/rdma/ib_verbs.h | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 3ac7371..5fe7723 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -104,6 +104,7 @@ enum ib_device_cap_flags { * IPoIB driver may set NETIF_F_IP_CSUM for datagram mode. */ IB_DEVICE_UD_IP_CSUM = (1<<18), + IB_DEVICE_UD_TSO = (1<<19), IB_DEVICE_SEND_W_INV = (1<<21), }; @@ -412,6 +413,7 @@ enum ib_wc_opcode { IB_WC_COMP_SWAP, IB_WC_FETCH_ADD, IB_WC_BIND_MW, + IB_WC_LSO, /* * Set value of IB_WC_RECV so consumers can test if a completion is a * receive by testing (opcode & IB_WC_RECV). @@ -623,7 +625,8 @@ enum ib_wr_opcode { IB_WR_SEND_WITH_IMM, IB_WR_RDMA_READ, IB_WR_ATOMIC_CMP_AND_SWP, - IB_WR_ATOMIC_FETCH_AND_ADD + IB_WR_ATOMIC_FETCH_AND_ADD, + IB_WR_LSO }; enum ib_send_flags { @@ -662,6 +665,9 @@ struct ib_send_wr { } atomic; struct { struct ib_ah *ah; + void *header; + int hlen; + int mss; u32 remote_qpn; u32 remote_qkey; u16 pkey_index; /* valid for GSI only */ -- 1.5.4.3 From balineserac at gostafford.com Fri Mar 28 17:51:11 2008 From: balineserac at gostafford.com (University Degree) Date: Fri, 28 Mar 2008 18:51:11 -0600 Subject: [ofa-general] University Degree brings success Message-ID: <401159956.26499058759870@gostafford.com> 1-410-230-1849 University Degree Get University Degree within weeks without stress, exams, and long years of studying. Confidentiality Assured 1-410-230-1849 24 hours a day, 7 days a week including Sundays and Holidays -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at rheadm.com Fri Mar 28 19:47:37 2008 From: info at rheadm.com (=?windows-1255?B?4/jqIOT59PI=?=) Date: Fri, 28 Mar 2008 21:47:37 -0500 Subject: [ofa-general] =?windows-1255?b?9+Xg9ifp8OIg7OvsIOTn6ent?= Message-ID: <20080329024844.4CA2CE60974@openfabrics.org> An HTML attachment was scrubbed... URL: From rdreier at cisco.com Fri Mar 28 20:22:42 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 20:22:42 -0700 Subject: [ofa-general] Re: [PATCH 4/10 v1] IB/ipoib: Add LSO support to ipoib In-Reply-To: <1206451839.25950.356.camel@mtls03> (Eli Cohen's message of "Tue, 25 Mar 2008 15:30:39 +0200") References: <1206451839.25950.356.camel@mtls03> Message-ID: OK, applied, but: > + if (!skb_is_gso(skb)) { > + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { > + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", > + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); can this ever really happen? Is it worth having code to check it? presumably we'll just get a local length error on the send and find out anyway. > + if (unlikely(!skb_pull(skb, hlen))) { > + ipoib_warn(priv, "linear data too small\n"); > + goto drop; > + } similarly can this ever trigger? - R. From rdreier at cisco.com Fri Mar 28 20:22:42 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 20:22:42 -0700 Subject: [ofa-general] Re: [PATCH 4/10 v1] IB/ipoib: Add LSO support to ipoib In-Reply-To: <1206451839.25950.356.camel@mtls03> (Eli Cohen's message of "Tue, 25 Mar 2008 15:30:39 +0200") References: <1206451839.25950.356.camel@mtls03> Message-ID: OK, applied, but: > + if (!skb_is_gso(skb)) { > + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { > + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", > + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); can this ever really happen? Is it worth having code to check it? presumably we'll just get a local length error on the send and find out anyway. > + if (unlikely(!skb_pull(skb, hlen))) { > + ipoib_warn(priv, "linear data too small\n"); > + goto drop; > + } similarly can this ever trigger? - R. From rdreier at cisco.com Fri Mar 28 20:35:22 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 20:35:22 -0700 Subject: [ofa-general] Re: [PATCH 1/1] MLX4: Added resize_cq capability. In-Reply-To: <47E923CA.90804@dev.mellanox.co.il> (Vladimir Sokolovsky's message of "Tue, 25 Mar 2008 18:09:46 +0200") References: <47E923CA.90804@dev.mellanox.co.il> Message-ID: > +enum { > + MLX4_MAX_DIRECT_CQ_SIZE = 2 * PAGE_SIZE > +}; If you're going to introduce this, please replace the existing use of 2 * PAGE_SIZE with the constant. > +enum { > + MLX4_CQ_ENTRY_SIZE = 0x20 > +}; Just use sizeof (struct mlx4_cqe) the way the rest of the code does. > + cq->is_kernel = !context; I don't think there's any need for this new member... we can just test cq->ibcq->uobject the way the rest of the existing code does. > + context = kzalloc(sizeof *context, GFP_KERNEL); > + memset(context, 0, sizeof *context); no point in doing this memset if you use kzalloc(). - R. From rdreier at cisco.com Fri Mar 28 20:37:30 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 20:37:30 -0700 Subject: [ofa-general] Re: [PATCH] libmlx4: Added resize CQ capability. In-Reply-To: <47E92539.7030908@dev.mellanox.co.il> (Vladimir Sokolovsky's message of "Tue, 25 Mar 2008 18:15:53 +0200") References: <47E92539.7030908@dev.mellanox.co.il> Message-ID: > struct mlx4_cq { > + int cqe; I'm probably missing something but I don't see why we can't continue using cq->ibv_cq.cqe, which avoids a bunch of changes scattered around. The whole resize operation is inside the CQ's spinlock so we don't have to worry about keeping the size of the CQ consistent or anything like that. - R. From a-17-m at adimodel.com Fri Mar 28 21:29:15 2008 From: a-17-m at adimodel.com (Virginia Singer) Date: Sat, 29 Mar 2008 12:29:15 +0800 Subject: [ofa-general] hope you remember me Message-ID: <367292142.21968748625744@adimodel.com> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at Yvonne at jolasite.com only, because I am using my friend's email to write this. Mind me sending some of my pictures to you? From rdreier at cisco.com Fri Mar 28 22:35:19 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 22:35:19 -0700 Subject: [ofa-general] error with ibv_poll_cq() call In-Reply-To: <200803260901.25918.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Wed, 26 Mar 2008 09:01:25 +0200") References: <200803260901.25918.jackm@dev.mellanox.co.il> Message-ID: > It looks like we have a race condition in mlx4_destroy_qp. We clean the > cq BEFORE modifying the QP to reset (done in kernel as part of > the ibv_cmd_destroy_qp() flow). ouch, you're right. however I think your patch is not the best way to handle this: > - mlx4_cq_clean(to_mcq(ibqp->recv_cq), ibqp->qp_num, > - ibqp->srq ? to_msrq(ibqp->srq) : NULL); > - if (ibqp->send_cq != ibqp->recv_cq) > - mlx4_cq_clean(to_mcq(ibqp->send_cq), ibqp->qp_num, NULL); > - > mlx4_lock_cqs(ibqp); > mlx4_clear_qp(to_mctx(ibqp->context), ibqp->qp_num); > mlx4_unlock_cqs(ibqp); > @@ -576,6 +571,11 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) > return ret; > } > > + mlx4_cq_clean(to_mcq(ibqp->recv_cq), ibqp->qp_num, > + ibqp->srq ? to_msrq(ibqp->srq) : NULL); > + if (ibqp->send_cq != ibqp->recv_cq) > + mlx4_cq_clean(to_mcq(ibqp->send_cq), ibqp->qp_num, NULL); > + because you leave the clear_qp call before the cq_clean and they are not within the same cq lock, there is a window where a consumer could poll a valid CQE for a qp that is no longer in the table. I think it's better to really match the logic of the kernel driver (which I think is correct). untested patch below: diff --git a/src/cq.c b/src/cq.c index 91297e4..53c6e06 100644 --- a/src/cq.c +++ b/src/cq.c @@ -381,15 +381,13 @@ void mlx4_cq_event(struct ibv_cq *cq) to_mcq(cq)->arm_sn++; } -void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) +void __mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) { struct mlx4_cqe *cqe, *dest; uint32_t prod_index; uint8_t owner_bit; int nfreed = 0; - pthread_spin_lock(&cq->lock); - /* * First we need to find the current producer index, so we * know where to start cleaning from. It doesn't matter if HW @@ -429,7 +427,12 @@ void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) wmb(); update_cons_index(cq); } +} +void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) +{ + pthread_spin_lock(&cq->lock); + __mlx4_cq_clean(cq, qpn, srq); pthread_spin_unlock(&cq->lock); } diff --git a/src/mlx4.h b/src/mlx4.h index 3710a17..9b0ad6a 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -312,8 +312,8 @@ int mlx4_destroy_cq(struct ibv_cq *cq); int mlx4_poll_cq(struct ibv_cq *cq, int ne, struct ibv_wc *wc); int mlx4_arm_cq(struct ibv_cq *cq, int solicited); void mlx4_cq_event(struct ibv_cq *cq); -void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, - struct mlx4_srq *srq); +__void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq); +void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq); void mlx4_cq_resize_copy_cqes(struct mlx4_cq *cq, void *buf, int new_cqe); struct ibv_srq *mlx4_create_srq(struct ibv_pd *pd, diff --git a/src/verbs.c b/src/verbs.c index 50e0947..26174f8 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -532,23 +532,20 @@ int mlx4_destroy_qp(struct ibv_qp *ibqp) struct mlx4_qp *qp = to_mqp(ibqp); int ret; - mlx4_cq_clean(to_mcq(ibqp->recv_cq), ibqp->qp_num, - ibqp->srq ? to_msrq(ibqp->srq) : NULL); - if (ibqp->send_cq != ibqp->recv_cq) - mlx4_cq_clean(to_mcq(ibqp->send_cq), ibqp->qp_num, NULL); + ret = ibv_cmd_destroy_qp(ibqp); + if (ret) + return ret; mlx4_lock_cqs(ibqp); - mlx4_clear_qp(to_mctx(ibqp->context), ibqp->qp_num); - mlx4_unlock_cqs(ibqp); - ret = ibv_cmd_destroy_qp(ibqp); - if (ret) { - mlx4_lock_cqs(ibqp); - mlx4_store_qp(to_mctx(ibqp->context), ibqp->qp_num, qp); - mlx4_unlock_cqs(ibqp); + __mlx4_cq_clean(to_mcq(ibqp->recv_cq), ibqp->qp_num, + ibqp->srq ? to_msrq(ibqp->srq) : NULL); + if (ibqp->send_cq != ibqp->recv_cq) + __mlx4_cq_clean(to_mcq(ibqp->send_cq), ibqp->qp_num, NULL); - return ret; - } + mlx4_clear_qp(to_mctx(ibqp->context), ibqp->qp_num); + + mlx4_unlock_cqs(ibqp); if (!ibqp->srq) mlx4_free_db(to_mctx(ibqp->context), MLX4_DB_TYPE_RQ, qp->db); From rdreier at cisco.com Fri Mar 28 22:44:54 2008 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 28 Mar 2008 22:44:54 -0700 Subject: [ofa-general] BUG() in iw_cm:iw_cm:cm_work_handler Message-ID: I've seen the following crash a few times using iWARP on cxgb3. The crash is at the BUG() here: switch (cm_id_priv->state) { /* ... */ default: BUG(); } I added a print to dump the state, and it was 2 (IW_CM_STATE_CONN_RECV) both times I hit the crash after that. the way I've been able to reproduce it is with some buggy code that does an RDMA read into a memory region without remote write permission (which works on IB but not iWARP). The active side issues an RDMA read, which fails and causes the connection to be torn down, and the passive side crashes as below sometimes. Anyone have any suggestions on how to track this down? [11481.273827] ------------[ cut here ]------------ [11481.273827] kernel BUG at drivers/infiniband/core/iwcm.c:790! [11481.273827] invalid opcode: 0000 [1] SMP [11481.273827] CPU 3 [11481.273827] Modules linked in: rdma_ucm ib_uverbs nfs lockd nfs_acl fan ac battery ipv6 fuse dm_snapshot dm_mirror dm_mod iw_cxgb3 svcrdma xprtrdma sunrpc rdma_cm ib_cm iw_cm ib_sa ib_mad ib_addr loop ide_cd_mod cdrom ide_pci_generic piix iw_nes serio_raw ide_core cxgb3 ib_core evdev thermal pcspkr psmouse ipmi_si ipmi_msghandler ehci_hcd bnx2 zlib_inflate libcrc32c firmware_class container button uhci_hcd processor [11481.273827] Pid: 2518, comm: iw_cm_wq Not tainted 2.6.25-rc7 #84 [11481.273827] RIP: 0010:[] [] :iw_cm:cm_work_handler+0x36e/0x42a [11481.273827] RSP: 0018:ffff8100798e9dc0 EFLAGS: 00010093 [11481.273827] RAX: 0000000000000002 RBX: 0000000000000246 RCX: ffffffff80622f04 [11481.273827] RDX: ffff810079520000 RSI: 0000000000000001 RDI: 0000000000000086 [11481.273828] RBP: ffff8100798e9e50 R08: ffff81007e588d08 R09: ffff81000100a030 [11481.273828] R10: 0000000000000001 R11: 00000000798e9a70 R12: ffff81007e588c00 [11481.273828] R13: 0000000000000000 R14: ffff81007e588d08 R15: ffff8100798e9de0 [11481.273828] FS: 0000000000000000(0000) GS:ffff81007fc02e00(0000) knlGS:0000000000000000 [11481.273828] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b [11481.273828] CR2: 00007fed6a22e7c0 CR3: 0000000079596000 CR4: 00000000000006e0 [11481.273828] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [11481.273828] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [11481.273828] Process iw_cm_wq (pid: 2518, threadinfo ffff8100798e8000, task ffff810079520000) [11481.273828] Stack: ffff810079dec518 ffff81007e588cb8 ffff81007e588cf8 ffff81007e588cf8 [11481.273828] 0000000000000005 0000000000000000 0000000000000000 0000000000000000 [11481.273828] 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [11481.273828] Call Trace: [11481.273828] [] ? :iw_cm:cm_work_handler+0x0/0x42a [11481.273828] [] run_workqueue+0xeb/0x200 [11481.273828] [] worker_thread+0xa4/0xb5 [11481.273828] [] ? autoremove_wake_function+0x0/0x38 [11481.273828] [] ? worker_thread+0x0/0xb5 [11481.273828] [] kthread+0x49/0x78 [11481.273828] [] child_rip+0xa/0x12 [11481.273828] [] ? restore_args+0x0/0x30 [11481.273828] [] ? kthreadd+0x157/0x17c [11481.273828] [] ? kthread+0x0/0x78 [11481.273828] [] ? child_rip+0x0/0x12 [11481.273828] [11481.273828] [11481.273828] Code: 4c 89 f7 41 c7 44 24 58 00 00 00 00 e8 38 a4 2c f8 4c 89 fe 4c 89 e7 41 ff 14 24 4c 89 f7 41 89 c5 e8 6c a3 2c f8 48 89 c3 eb 07 <0f> 0b eb fe 45 31 ed 48 89 de 4c 89 f7 e8 0c a4 2c f8 eb 04 0f [11481.273828] RIP [] :iw_cm:cm_work_handler+0x36e/0x42a [11481.273828] RSP [11481.282720] ---[ end trace 2667de3bacfc5e84 ]--- From vodkkecihew at kkeci.com Fri Mar 28 23:34:30 2008 From: vodkkecihew at kkeci.com (Kara Seay) Date: Sat, 29 Mar 2008 13:34:30 +0700 Subject: [ofa-general] Software Message-ID: <555496487.76625917048389@kkeci.com> Are you searhing for the better prices in discount software? Right now you'll find the opportunity to have the softwares you dream of from very long ago. And the better thing is, all softwares are exceedingly cheap. Confirm it by yourself and have the softwares for cheapest rates! http://terriemiaheu541.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Sat Mar 29 02:45:53 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 29 Mar 2008 09:45:53 +0000 Subject: [ofa-general] Re: [PATCH 1/6] complib/nodenamemap: add generic parse_node_map() function In-Reply-To: <20080327143534.01150523.weiny2@llnl.gov> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> <1206577637-18226-2-git-send-email-sashak@voltaire.com> <20080326175228.0cee3e5a.weiny2@llnl.gov> <20080327085429.GA23849@sashak.voltaire.com> <20080327143534.01150523.weiny2@llnl.gov> Message-ID: <20080329094553.GV13708@sashak.voltaire.com> On 14:35 Thu 27 Mar , Ira Weiny wrote: > > Doesn't the root_nodes_list member of updn_t store the guids? No, it is removed in patch 3/6. > I'm not quite > sure what you are avoiding to store. It does not look as though you are > searching the file each time the code wants to know if a particular guid is a > root. Correct, and this is the point there - each heavy sweep Up/Down re-reads the root guids file and catches possible changes there. > In particular I don't want the node name lookup to search the file each time. > That is why I read the file and store it in a map object. I suppose the > callback method allows for users to use their own data structure object. > > > > > > We could rename nn_map_t to be more generic if necessary. > > > > > > I worry about maintaining 2 functions which parse the same file format in the > > > same c file. > > > > I agree with you, just didn't time to merge it. > > > > Ok, however, I think there is a slight difference in the formats the 2 > functions parse. I am looking for '"' symbols to delineate the name string. I > don't think you are. For example: > > 0xcafebabe "hello world" > > Would result in a node guid of 0xcafebabe and a string of "hello world". I > think your function would parse it as '"hello'? In fact including the '"'? Correct, it is slightly different today. By "merging" I meant format merge as well. Sasha From vaughannn at burin.com Sat Mar 29 06:19:52 2008 From: vaughannn at burin.com (Bradley Hobbs) Date: Sat, 29 Mar 2008 14:19:52 +0100 Subject: [ofa-general] Die laecherlichen Preise fuer legale Software Message-ID: <687665609.13863499361192@burin.com> Sie können unsere Software sofort runterladen! Zahlen Sie und die Downloads werden Ihnen sofort zur Verfügung stehen. Unsere Programme wurden für mehrere OS (Windows und Macintosh) programmiert und sind in allen europäischen Sprachen verfügbar. Wir verkaufen nur originale Vollversionen, haben aber die günstigsten Preise. Sie werden nach dem Kauf nicht alleine gelassen! Unsere freundlichen Kundenberater werden Ihnen bei der Installation helfen, falls Sie es benötigen. Schnelle Antworten und Geld-Zurück-Garantie sind bei uns selbstverständlich. Bestellen Sie die feinste Softwarehttp://geocities.com/damonjacobs602/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Sat Mar 29 03:27:22 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 29 Mar 2008 10:27:22 +0000 Subject: [ofa-general] smpquery vlarb ... dumps only 1 entry? In-Reply-To: <20080327214433.GK16973@sgi.com> References: <20080327214433.GK16973@sgi.com> Message-ID: <20080329102722.GW13708@sashak.voltaire.com> Hi, On 14:44 Thu 27 Mar , akepner at sgi.com wrote: > > In ofed 1.2, "smpquery vlarb ... " would produce output like: > > # VLArbitration tables: Lid 8 port 0 LowCap 8 HighCap 8 > # Low priority VL Arbitration Table: > VL : |0x0 |0x1 |0x2 |0x3 |0x4 |0x5 |0x6 |0x7 | > WEIGHT: |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 | > # High priority VL Arbitration Table: > VL : |0x0 |0x0 |0x0 |0x0 |0x1 |0x2 |0x3 |0x4 | > WEIGHT: |0xFF|0xFF|0xFF|0xFF|0xFF|0x7F|0x3F|0x1F| > > which is what I expected. > > But in ofed 1.3, I only get a single entry: > > # VLArbitration tables: Lid 1 port 0 LowCap 8 HighCap 8 > # Low priority VL Arbitration Table: > VL : |0x3 | > WEIGHT: |0x3 | > # High priority VL Arbitration Table: > VL : |0x3 | > WEIGHT: |0x8 | > > Anyone know what changed here? I'm not aware about any of such changes. And it looks wrong (I cannot see this in my setup). mad_dump_vlarbitration() (from libibmad/src/dump.c) has this: sprintf(buf_line1, "%s0x%-2X|", buf_line1, vl); Any chance that you have different sprintf() implementations with OFED 1.2 and 1.3? Sasha From sashak at voltaire.com Sat Mar 29 03:37:39 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 29 Mar 2008 10:37:39 +0000 Subject: [ofa-general] smpquery vlarb ... dumps only 1 entry? In-Reply-To: <20080329102722.GW13708@sashak.voltaire.com> References: <20080327214433.GK16973@sgi.com> <20080329102722.GW13708@sashak.voltaire.com> Message-ID: <20080329103739.GX13708@sashak.voltaire.com> On 10:27 Sat 29 Mar , Sasha Khapyorsky wrote: > Hi, > > On 14:44 Thu 27 Mar , akepner at sgi.com wrote: > > > > In ofed 1.2, "smpquery vlarb ... " would produce output like: > > > > # VLArbitration tables: Lid 8 port 0 LowCap 8 HighCap 8 > > # Low priority VL Arbitration Table: > > VL : |0x0 |0x1 |0x2 |0x3 |0x4 |0x5 |0x6 |0x7 | > > WEIGHT: |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 |0x0 | > > # High priority VL Arbitration Table: > > VL : |0x0 |0x0 |0x0 |0x0 |0x1 |0x2 |0x3 |0x4 | > > WEIGHT: |0xFF|0xFF|0xFF|0xFF|0xFF|0x7F|0x3F|0x1F| > > > > which is what I expected. > > > > But in ofed 1.3, I only get a single entry: > > > > # VLArbitration tables: Lid 1 port 0 LowCap 8 HighCap 8 > > # Low priority VL Arbitration Table: > > VL : |0x3 | > > WEIGHT: |0x3 | > > # High priority VL Arbitration Table: > > VL : |0x3 | > > WEIGHT: |0x8 | > > > > Anyone know what changed here? > > I'm not aware about any of such changes. And it looks wrong (I cannot > see this in my setup). > > mad_dump_vlarbitration() (from libibmad/src/dump.c) has this: > > sprintf(buf_line1, "%s0x%-2X|", buf_line1, vl); > > Any chance that you have different sprintf() implementations with OFED > 1.2 and 1.3? Also how output of smpdump 1 0x18 0x00010000 smpdump 1 0x18 0x00030000 looks? Sasha From sashak at voltaire.com Sat Mar 29 05:12:52 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 29 Mar 2008 12:12:52 +0000 Subject: [ofa-general] [PATCH] libibmad/dump: support VLArb table size, fix printing Message-ID: <20080329121252.GY13708@sashak.voltaire.com> Add support for VLArb table size. Fix printing, eliminate intermediate buffers, some other cleanups. Signed-off-by: Sasha Khapyorsky --- Arthur, could you try this? libibmad/src/dump.c | 54 +++++++++++++++----------------------------------- 1 files changed, 16 insertions(+), 38 deletions(-) diff --git a/libibmad/src/dump.c b/libibmad/src/dump.c index 254106b..b2798b8 100644 --- a/libibmad/src/dump.c +++ b/libibmad/src/dump.c @@ -588,35 +588,14 @@ ib_slvl_get_i(ib_slvl_table_t *tbl, int i, uint8_t *vl) *vl = (tbl->vl_by_sl_num[i >> 1] >> ((!(i&1)) << 2)) & 0xf; } -typedef struct _ib_vl_arb_element { - uint8_t res_vl; - uint8_t weight; -} __attribute__((packed)) ib_vl_arb_element_t; - #define IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK 32 -#define IB_NUM_VL_ARB_ELEMENTS_CONF_SUPPORT 8 -#define IB_NUM_VL_ARB_BLOCKS_IN_TBL 2 typedef struct _ib_vl_arb_table { - ib_vl_arb_element_t vl_entry[IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]; -} ib_vl_arb_table_t; - -typedef struct _ib_vl_arb_table_two_blocks { - ib_vl_arb_table_t tbl_blocks[IB_NUM_VL_ARB_BLOCKS_IN_TBL]; -} ib_vl_arb_table_two_blocks_t; - -static inline uint8_t -ib_vl_arb_get_vl_entries_num_in_table(ib_vl_arb_table_t *tbl) -{ - uint8_t i; - - for (i = 0; i < 32; i++) { - if (!(tbl->vl_entry[i].res_vl || tbl->vl_entry[i].weight)) - break; - } - - return i; -} + struct { + uint8_t res_vl; + uint8_t weight; + } vl_entry[IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]; +} __attribute__((packed)) ib_vl_arb_table_t; static inline void ib_vl_arb_get_vl(uint8_t res_vl, uint8_t *const vl ) @@ -639,27 +618,26 @@ mad_dump_sltovl(char *buf, int bufsz, void *val, int valsz) } void -mad_dump_vlarbitration(char *buf, int bufsz, void *val, int valsz) +mad_dump_vlarbitration(char *buf, int bufsz, void *val, int num) { ib_vl_arb_table_t* p_vla_tbl = val; - char buf_line1[1024]; - char buf_line2[1024]; + unsigned i, n; uint8_t vl; - int i; - buf_line1[0] = 0; - buf_line2[0] = 0; + num /= sizeof(p_vla_tbl->vl_entry[0]); - for (i = 0; i<8; i++) { + n = snprintf(buf, bufsz, "\nVL : |"); + for (i = 0; i < num; i++) { ib_vl_arb_get_vl(p_vla_tbl->vl_entry[i].res_vl, &vl); - sprintf(buf_line1, "%s0x%-2X|", buf_line1, vl); + n += snprintf(buf + n, bufsz - n, "0x%-2X|", vl); } - for (i = 0; i<8; i++) - sprintf(buf_line2, "%s0x%-2X|", - buf_line2, p_vla_tbl->vl_entry[i].weight); + n += snprintf(buf + n, bufsz - n, "\nWEIGHT: |"); + for (i = 0; i < num; i++) + n += snprintf(buf + n, bufsz - n, "0x%-2X|", + p_vla_tbl->vl_entry[i].weight); - snprintf(buf, bufsz, "\nVL : |%s\nWEIGHT: |%s\n", buf_line1, buf_line2); + snprintf(buf + n, bufsz - n, "\n"); } static int -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Sat Mar 29 05:13:54 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 29 Mar 2008 12:13:54 +0000 Subject: [ofa-general] [PATCH] infiniband-diags: pass valid VLArb table size to dump func In-Reply-To: <20080329121252.GY13708@sashak.voltaire.com> References: <20080329121252.GY13708@sashak.voltaire.com> Message-ID: <20080329121354.GZ13708@sashak.voltaire.com> Pass valid VLArb table size to bump function from libibmad. Signed-off-by: Sasha Khapyorsky --- infiniband-diags/src/smpquery.c | 15 ++++++++------- 1 files changed, 8 insertions(+), 7 deletions(-) diff --git a/infiniband-diags/src/smpquery.c b/infiniband-diags/src/smpquery.c index 7535f37..8e2efa0 100644 --- a/infiniband-diags/src/smpquery.c +++ b/infiniband-diags/src/smpquery.c @@ -275,7 +275,7 @@ sl2vl_table(ib_portid_t *dest, char **argv, int argc) return 0; } -static char *vlarb_dump_table_entry(ib_portid_t *dest, int portnum, int offset) +static char *vlarb_dump_table_entry(ib_portid_t *dest, int portnum, int offset, unsigned cap) { char buf[2048]; char data[IB_SMP_DATA_SIZE]; @@ -283,7 +283,7 @@ static char *vlarb_dump_table_entry(ib_portid_t *dest, int portnum, int offset) if (!smp_query(data, dest, IB_ATTR_VL_ARBITRATION, (offset << 16) | portnum, 0)) return "vl arb query failed"; - mad_dump_vlarbitration(buf, sizeof(buf), data, sizeof(data)); + mad_dump_vlarbitration(buf, sizeof(buf), data, cap * 2); printf("%s", buf); return 0; } @@ -294,11 +294,12 @@ static char *vlarb_dump_table(ib_portid_t *dest, int portnum, char *ret; printf("# %s priority VL Arbitration Table:", name); - if ( (ret = vlarb_dump_table_entry(dest, portnum, offset)) || - (cap > 32 && - (ret = vlarb_dump_table_entry(dest, portnum, offset + 1))) ) - return ret; - return 0; + ret = vlarb_dump_table_entry(dest, portnum, offset, + cap < 32 ? cap : 32); + if (!ret && cap > 32) + ret = vlarb_dump_table_entry(dest, portnum, offset + 1, + cap - 32); + return ret; } static char * -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Sat Mar 29 05:22:49 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 29 Mar 2008 12:22:49 +0000 Subject: [ofa-general] [general-bounces@lists.openfabrics.org: Your message to general awaits moderator approval] Message-ID: <20080329122249.GA13708@sashak.voltaire.com> Hi, Now I get such messages on each post to . Is our list moderated unconditionally? Sasha ----- Forwarded message from general-bounces at lists.openfabrics.org ----- Date: Sat, 29 Mar 2008 03:12:59 -0700 From: general-bounces at lists.openfabrics.org To: sashak at voltaire.com Subject: Your message to general awaits moderator approval MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-ID: Precedence: bulk Sender: general-bounces at lists.openfabrics.org Errors-To: general-bounces at lists.openfabrics.org Return-Path: general-bounces at lists.openfabrics.org Your mail to 'general' with the subject Re: [ofa-general] smpquery vlarb ... dumps only 1 entry? Is being held until the list moderator can review it for approval. The reason it is being held: The message headers matched a filter rule Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: ----- End forwarded message ----- ----- Forwarded message from general-bounces at lists.openfabrics.org ----- Date: Sat, 29 Mar 2008 03:23:11 -0700 From: general-bounces at lists.openfabrics.org To: sashak at voltaire.com Subject: Your message to general awaits moderator approval MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-ID: Precedence: bulk Sender: general-bounces at lists.openfabrics.org Errors-To: general-bounces at lists.openfabrics.org Return-Path: general-bounces at lists.openfabrics.org Your mail to 'general' with the subject Re: [ofa-general] smpquery vlarb ... dumps only 1 entry? Is being held until the list moderator can review it for approval. The reason it is being held: The message headers matched a filter rule Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: ----- End forwarded message ----- ----- Forwarded message from general-bounces at lists.openfabrics.org ----- Date: Sat, 29 Mar 2008 04:58:27 -0700 From: general-bounces at lists.openfabrics.org To: sashak at voltaire.com Subject: Your message to general awaits moderator approval MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-ID: Precedence: bulk Sender: general-bounces at lists.openfabrics.org Errors-To: general-bounces at lists.openfabrics.org Return-Path: general-bounces at lists.openfabrics.org Your mail to 'general' with the subject [PATCH] libibmad/dump: support VLArb table size, fix printing Is being held until the list moderator can review it for approval. The reason it is being held: The message headers matched a filter rule Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: ----- End forwarded message ----- ----- Forwarded message from general-bounces at lists.openfabrics.org ----- Date: Sat, 29 Mar 2008 04:59:27 -0700 From: general-bounces at lists.openfabrics.org To: sashak at voltaire.com Subject: Your message to general awaits moderator approval MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-ID: Precedence: bulk Sender: general-bounces at lists.openfabrics.org Errors-To: general-bounces at lists.openfabrics.org Return-Path: general-bounces at lists.openfabrics.org Your mail to 'general' with the subject [PATCH] infiniband-diags: pass valid VLArb table size to dump func Is being held until the list moderator can review it for approval. The reason it is being held: The message headers matched a filter rule Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: ----- End forwarded message ----- From akstcacecomnsdgs at aceco.org Sat Mar 29 09:28:21 2008 From: akstcacecomnsdgs at aceco.org (Abdul Plummer) Date: , 30 Mar 2008 01:28:21 +0900 Subject: [ofa-general] Abdul Message-ID: <01c89205$56e29200$6f255676@akstcacecomnsdgs> Buy meds safely: secure and confidential purchase only. http://jennafajardopc.blogspot.com From diskscdil at bobys.com Sat Mar 29 09:27:02 2008 From: diskscdil at bobys.com (Roxanne Bray) Date: Sat, 29 Mar 2008 08:27:02 -0800 Subject: [ofa-general] Take her to love heaven Message-ID: <01c89176$a9bd8f00$3c61acc9@diskscdil> On top all night Please look attached file and know MORE ABOUT THIS! -------------- next part -------------- A non-text attachment was scrubbed... Name: secrets.zip Type: application/zip Size: 634 bytes Desc: not available URL: From info at webheretic.com Sat Mar 29 08:13:24 2008 From: info at webheretic.com (jean-pierre suranet) Date: Sat, 29 Mar 2008 15:13:24 +0000 Subject: [ofa-general] Paris gives a BJ to Justin Timberlake Message-ID: <000501c891be$0543c96b$ccb11786@wkmnxe> XjvvYAvBDq Download and WatchkeiNeXjvvYA -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeaninenn at estradaroma.com Sat Mar 29 09:45:24 2008 From: jeaninenn at estradaroma.com (ashlin wong) Date: Sat, 29 Mar 2008 16:45:24 +0000 Subject: [ofa-general] Watches Online Message-ID: <000401c891cb$0463ee9a$943f0091@gujkbrl> High Quality Watches Available Now Directory Of Watches Providers. Find Watches Quickly. http://fraysmemsteep.com/ From rdreier at cisco.com Sat Mar 29 16:33:17 2008 From: rdreier at cisco.com (Roland Dreier) Date: Sat, 29 Mar 2008 16:33:17 -0700 Subject: [ofa-general] Re: [ANNOUNCE] librdmacm release 1.0.7 In-Reply-To: <000001c89105$a3da0ee0$b137170a@amr.corp.intel.com> (Sean Hefty's message of "Fri, 28 Mar 2008 11:57:58 -0700") References: <000001c89105$a3da0ee0$b137170a@amr.corp.intel.com> Message-ID: FWIW I've updated the Debian and Fedora librdmacm packages as well as my Ubuntu PPA to librdmacm-1.0.7. - R. From teichp at quick.cz Sat Mar 29 20:41:28 2008 From: teichp at quick.cz (Keri Sellers) Date: Sat, 29 Mar 2008 22:41:28 -0500 Subject: [ofa-general] Smart in bed games Message-ID: <01c891ee$06a85c00$c45841be@teichp> She wants you more now Please look attached file and know MORE ABOUT THIS! -------------- next part -------------- A non-text attachment was scrubbed... Name: secrets.zip Type: application/zip Size: 634 bytes Desc: not available URL: From dotanb at dev.mellanox.co.il Sat Mar 29 22:52:00 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Sun, 30 Mar 2008 08:52:00 +0300 Subject: [ofa-general] Re: the port numbers in some of the rdmacm examples is a fixed value In-Reply-To: <000101c89022$ce0b3d30$9c98070a@amr.corp.intel.com> References: <47EBBC81.4030501@dev.mellanox.co.il> <000101c89022$ce0b3d30$9c98070a@amr.corp.intel.com> Message-ID: <47EF2A80.1020804@dev.mellanox.co.il> Sean Hefty wrote: >> It seems that some of the examples that comes with the librdmacm >> examples (cmatose, udaddy) have >> a fixed port number, i would like to send you a patch that allows >> changing the port number in the command line. >> >> Any objection? >> > > No objection at all -- just keep the defaults if one isn't provided. Thanks > Of course. I started to work on this patch and for ucmatose everything is fine. The problem is with udaddy: the parameter "-p" is only being used for the port space ... (I really would like to have the same parameter for controlling the port number for ALL of the examples of the librdmacm, but i must admit that without doing some changes it won't happen) ... what do you think? (do you want me to send you the changes i made so far?) thanks Dotan From a-340 at acs-mail.com Sat Mar 29 22:59:09 2008 From: a-340 at acs-mail.com (Kristina Lloyd) Date: , 30 Mar 2008 14:59:09 +0900 Subject: [ofa-general] good luck Message-ID: <01c89276$9b569c80$7ca7c874@a-340> Hello! I am bored tonight. I am nice girl that would like to chat with you. Email me at Elsa at jolasite.com only, because I am using my friend's email to write this. Hope you like my pictures. From RigobertonucleolusKnox at webopedia.com Sat Mar 29 12:41:25 2008 From: RigobertonucleolusKnox at webopedia.com (Vern Mcclure) Date: Sun, 30 Mar 2008 01:41:25 +0600 Subject: [ofa-general] Dont miss this chance to ride a multibagger Message-ID: <7IX789EJXVWDA505@webopedia.com> Introducing the company with a unique concept Red Branch Technologies Symbol: RBTI Red Branch Technologies, Inc. makes business travel easier, more secure and more responsive for both the hard-charging business traveler and the corporation by meeting travel needs at each point in the travel cycle. The company's innovative my/mTravel(r) and mTravel(r) products automate the business travel process from planning and booking to en route services and support, thru post travel reporting and unused ticket redemption. Red Branch's Magellan360 provides agency and net-delivered back office services to independent professional travel marketers. The company has real clients and income stream. Look at their recent PR's - Red Branch Technologies Teams With DeskPort Technologies to Introduce 60,000 Business Travelers to my/mTravel - Red Branch Technologies Gives Business Travelers Credit Card Fraud and Identity Theft Protection With Introduction of IdentiFlyer - Red Branch Technologies Wholly-Owned Subsidiary, Magellan360, Releases Unaudited Financial Performance for 2007; Revenue Up 12 Percent to $22 Million Do your due diligence on RBTI, and read their conferece call details you'll see why we think this is the undiscovered gem for double digit gains. Forward looking statements are based on expectations, estimates and projections at the time the statements are made that involve a number of risks and uncertainties which could cause actual results or events to differ materially from those presently anticipated From askaterchix at golfagent.com Sat Mar 29 22:05:50 2008 From: askaterchix at golfagent.com (devlen beverley) Date: Sun, 30 Mar 2008 05:05:50 +0000 Subject: [ofa-general] 70% discount. Code #KZiF Message-ID: <93637.daryl@wai> Hello general, be wise, purchase your drugs from the most well-known online store since 1997. http://www.google.co.uk/pagead/iclk?sa=l&ai=xcbMSm&num=43130&adurl=http://PGpD.timeminute.com keefer pawan From eli at dev.mellanox.co.il Sun Mar 30 00:41:13 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Sun, 30 Mar 2008 10:41:13 +0300 Subject: [ofa-general] Re: [PATCH 4/10 v1] IB/ipoib: Add LSO support to ipoib In-Reply-To: References: <1206451839.25950.356.camel@mtls03> Message-ID: <1206862873.19448.18.camel@mtls03> On Fri, 2008-03-28 at 20:22 -0700, Roland Dreier wrote: > OK, applied, but: > > > + if (!skb_is_gso(skb)) { > > + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { > > + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", > > + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); > > can this ever really happen? I saw the above warning a few times. > Is it worth having code to check it? > presumably we'll just get a local length error on the send and find > out anyway. Why do you think we would get a local length error? Looks to me like the message will be transmitted. When this condition becomes true an ICMP message will be sent to the other host to reduce the path MTU? Shouldn't we keep the above condition for this? > > > + if (unlikely(!skb_pull(skb, hlen))) { > > + ipoib_warn(priv, "linear data too small\n"); > > + goto drop; > > + } > > similarly can this ever trigger? > I was not 100% sure that this won't happen so I put the condition. I never saw this warning anyway so I think we can just use this: __skb_pull(skb, hlen); What do you think? From eli at dev.mellanox.co.il Sun Mar 30 00:41:13 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Sun, 30 Mar 2008 10:41:13 +0300 Subject: [ofa-general] Re: [PATCH 4/10 v1] IB/ipoib: Add LSO support to ipoib In-Reply-To: References: <1206451839.25950.356.camel@mtls03> Message-ID: <1206862873.19448.18.camel@mtls03> On Fri, 2008-03-28 at 20:22 -0700, Roland Dreier wrote: > OK, applied, but: > > > + if (!skb_is_gso(skb)) { > > + if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { > > + ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", > > + skb->len, priv->mcast_mtu + IPOIB_ENCAP_LEN); > > can this ever really happen? I saw the above warning a few times. > Is it worth having code to check it? > presumably we'll just get a local length error on the send and find > out anyway. Why do you think we would get a local length error? Looks to me like the message will be transmitted. When this condition becomes true an ICMP message will be sent to the other host to reduce the path MTU? Shouldn't we keep the above condition for this? > > > + if (unlikely(!skb_pull(skb, hlen))) { > > + ipoib_warn(priv, "linear data too small\n"); > > + goto drop; > > + } > > similarly can this ever trigger? > I was not 100% sure that this won't happen so I put the condition. I never saw this warning anyway so I think we can just use this: __skb_pull(skb, hlen); What do you think? From otk0x5f8ly at gmail.com Sun Mar 30 00:49:53 2008 From: otk0x5f8ly at gmail.com (=?BIG5?B?ufmuYbr7?=) Date: Sun, 30 Mar 2008 15:49:53 +0800 Subject: [ofa-general] =?big5?b?sXrB2ahTprOkQK3TsU3E3aq6pfi3frr0r7i23KFI?= =?big5?b?ob8=?= Message-ID: 2008-3-30 - 15:49 山鉧科技網頁設計 我們的宗旨：客戶的每ㄧ件小事情，都是山鉧的大事情 我們現在推出企業形象網站包含前台網頁美工+後台管理程式 限時限量專案價只要$24,900 (在送ㄧ年100MB不限流量網站空間) 我們做的不只是網站，而是您企業的入口 ㄧ個好的企業網站資料即時更新的速度是很重要的 選擇山鉧成就您的夢想 歡迎來電洽詢黃專員：0-9-3-8-7-6-4-3-9-5 ZlKTaa From eli at dev.mellanox.co.il Sun Mar 30 02:34:10 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Sun, 30 Mar 2008 12:34:10 +0300 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: References: <1205767427.25950.137.camel@mtls03> Message-ID: <1206869650.19448.29.camel@mtls03> On Fri, 2008-03-28 at 14:32 -0700, Roland Dreier wrote: > Thanks, I applied this with some extra code in all the low-level > drivers to make sure that the create_flags are passed in as 0. Does > that make sense to everyone? > As far as I could see, all kernel consumers clear struct ib_qp_init_attr before calling ib_create_qp() and for user space we clear this too. The only hole is that rdma_create_qp() is exported and someone may be calling this function without clearing struct ib_qp_init_attr. But I think your approach is safer and should be used. From eli at dev.mellanox.co.il Sun Mar 30 02:34:10 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Sun, 30 Mar 2008 12:34:10 +0300 Subject: [ofa-general] Re: [PATCH 2/10] IB/core: Add creation flags to QPs In-Reply-To: References: <1205767427.25950.137.camel@mtls03> Message-ID: <1206869650.19448.29.camel@mtls03> On Fri, 2008-03-28 at 14:32 -0700, Roland Dreier wrote: > Thanks, I applied this with some extra code in all the low-level > drivers to make sure that the create_flags are passed in as 0. Does > that make sense to everyone? > As far as I could see, all kernel consumers clear struct ib_qp_init_attr before calling ib_create_qp() and for user space we clear this too. The only hole is that rdma_create_qp() is exported and someone may be calling this function without clearing struct ib_qp_init_attr. But I think your approach is safer and should be used. From ogerlitz at voltaire.com Sun Mar 30 03:28:39 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 30 Mar 2008 13:28:39 +0300 Subject: [ofa-general] Re: files preamble In-Reply-To: <47EC06AF.8000309@sun.com> References: <47E5EF49.9080506@voltaire.com> <47EC06AF.8000309@sun.com> Message-ID: <47EF6B57.40502@voltaire.com> Ted H. Kim wrote: > Or, > > I am not responding directly to Arkady's question, but > aren't there some files with other licenses attached? > > For example, it appears addr.c, cma.c, ib_addr.h, rdma_cm.h > and rdma_cm_ib.h -- all have the "Common Public License 1.0" > mentioned. > I have no idea what the common public license is, generally speaking I think it would be fine if you send a patch that removes it from all the files under drivers/infiniband and include/rdma. Or. From ogerlitz at voltaire.com Sun Mar 30 03:49:56 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 30 Mar 2008 13:49:56 +0300 Subject: [ofa-general] Re: files preamble In-Reply-To: <47EF6B57.40502@voltaire.com> References: <47E5EF49.9080506@voltaire.com> <47EC06AF.8000309@sun.com> <47EF6B57.40502@voltaire.com> Message-ID: <47EF7054.6020503@voltaire.com> Or Gerlitz wrote: > Ted H. Kim wrote: >> For example, it appears addr.c, cma.c, ib_addr.h, rdma_cm.h >> and rdma_cm_ib.h -- all have the "Common Public License 1.0" >> mentioned. >> > I have no idea what the common public license is, generally speaking I > think it would be fine if you send a patch that removes it from all > the files under drivers/infiniband and include/rdma. Hi Ted, OK, as its about legals, if you want to drive the removal of this license from the files, best if its first being discussed in an appropriate forum which I am not sure if this list is, so in that respect, I take back my proposal for you to send a patch... Or. From jackm at dev.mellanox.co.il Sun Mar 30 05:07:03 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Sun, 30 Mar 2008 14:07:03 +0200 Subject: [ofa-general] [PATCH] core: ib_uverbs_reg_xrc_rcv_qp thinko fix. Message-ID: <200803301507.03265.jackm@dev.mellanox.co.il> core: ib_uverbs_reg_xrc_rcv_qp thinko fix. When caller calls ib_uverbs_reg_xrc_rcv_qp more than once for a given receive QP, should return in_len, and should not call put_xrcd_read again. Found By: Ronni Zimmerman of Mellanox QA. Signed-off-by: Jack Morgenstein diff --git a/kernel_patches/fixes/core_0110_xrc_rcv.patch b/kernel_patches/fixes/core_0110_xrc_rcv.patch index 56b9ac4..c5b5af4 100644 --- a/kernel_patches/fixes/core_0110_xrc_rcv.patch +++ b/kernel_patches/fixes/core_0110_xrc_rcv.patch @@ -183,7 +183,7 @@ Index: ofed_kernel/drivers/infiniband/core/uverbs_cmd.c { struct inode *inode = NULL; int ret = 0; -@@ -2625,4 +2640,352 @@ void ib_uverbs_dealloc_xrcd(struct ib_de +@@ -2625,4 +2640,351 @@ void ib_uverbs_dealloc_xrcd(struct ib_de xrcd_table_delete(ib_dev, inode); } @@ -475,8 +475,7 @@ Index: ofed_kernel/drivers/infiniband/core/uverbs_cmd.c + if (cmd.qp_num == tmp->qp_num) { + kfree(qp_obj); + mutex_unlock(&file->mutex); -+ put_xrcd_read(xrcd_uobj); -+ return 0; ++ return in_len; + } + qp_obj->qp_num = cmd.qp_num; + qp_obj->domain_handle = cmd.xrc_domain_handle; From AmeliasaxonyRichter at jacksonimmuno.com Sun Mar 30 06:29:38 2008 From: AmeliasaxonyRichter at jacksonimmuno.com (Iris Hannah) Date: Sun, 30 Mar 2008 13:29:38 +0000 Subject: [ofa-general] Aggressive traders alert Message-ID: <3IX858EJXVWDA052@jacksonimmuno.com> An HTML attachment was scrubbed... URL: From MirandanarbonneDoherty at lifehacker.com Sun Mar 30 10:40:28 2008 From: MirandanarbonneDoherty at lifehacker.com (Patti Melvin) Date: Sun, 30 Mar 2008 15:40:28 -0200 Subject: [ofa-general] Superstar stock report Message-ID: <5IX287EJXVWDA167@lifehacker.com> Introducing the company with a unique concept Red Branch Technologies Symbol: RBTI Red Branch Technologies, Inc. makes business travel easier, more secure and more responsive for both the hard-charging business traveler and the corporation by meeting travel needs at each point in the travel cycle. The company's innovative my/mTravel(r) and mTravel(r) products automate the business travel process from planning and booking to en route services and support, thru post travel reporting and unused ticket redemption. Red Branch's Magellan360 provides agency and net-delivered back office services to independent professional travel marketers. The company has real clients and income stream. Look at their recent PR's - Red Branch Technologies Teams With DeskPort Technologies to Introduce 60,000 Business Travelers to my/mTravel - Red Branch Technologies Gives Business Travelers Credit Card Fraud and Identity Theft Protection With Introduction of IdentiFlyer - Red Branch Technologies Wholly-Owned Subsidiary, Magellan360, Releases Unaudited Financial Performance for 2007; Revenue Up 12 Percent to $22 Million Do your due diligence on RBTI, and read their conferece call details you'll see why we think this is the undiscovered gem for double digit gains. Forward looking statements are based on expectations, estimates and projections at the time the statements are made that involve a number of risks and uncertainties which could cause actual results or events to differ materially from those presently anticipated From sashak at voltaire.com Sun Mar 30 07:27:51 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 30 Mar 2008 14:27:51 +0000 Subject: [ofa-general] [PATCH] complib/nodenamemap: merge file parsers In-Reply-To: <20080330133201.GI13708@sashak.voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> <1206577637-18226-2-git-send-email-sashak@voltaire.com> <20080326175228.0cee3e5a.weiny2@llnl.gov> <20080327085429.GA23849@sashak.voltaire.com> <20080327143534.01150523.weiny2@llnl.gov> <20080329094553.GV13708@sashak.voltaire.com> <20080330133201.GI13708@sashak.voltaire.com> Message-ID: <20080330142751.GK13708@sashak.voltaire.com> Merge nodenamemap style format parsers. Signed-off-by: Sasha Khapyorsky --- opensm/complib/cl_nodenamemap.c | 100 +++++++++++++----------------- opensm/include/complib/cl_nodenamemap.h | 6 +- 2 files changed, 45 insertions(+), 61 deletions(-) diff --git a/opensm/complib/cl_nodenamemap.c b/opensm/complib/cl_nodenamemap.c index d09863a..bafaf5c 100644 --- a/opensm/complib/cl_nodenamemap.c +++ b/opensm/complib/cl_nodenamemap.c @@ -38,73 +38,61 @@ #include #include #include +#include #include #include #include #include -static void -read_names(nn_map_t *map) +static int map_name(void *cxt, uint64_t guid, char *p) { - char *line = NULL; - size_t len = 0; + cl_qmap_t *map = cxt; name_map_item_t *item; - rewind(map->fp); - while (getline(&line, &len, map->fp) != -1) { - char *guid_str = NULL; - char *name = NULL; - line[len-1] = '\0'; - if (line[0] == '#') - continue; - - guid_str = strtok(line, "\"#"); - name = strtok(NULL, "\"#"); - if (!guid_str || !name) - continue; - - item = malloc(sizeof(*item)); - if (!item) { - goto error; - } - item->guid = strtoull(guid_str, NULL, 0); - item->name = strdup(name); - cl_qmap_insert(&(map->map), item->guid, (cl_map_item_t *)item); - } + p = strtok(p, "\"#"); + if (!p) + return 0; -error: - free (line); + item = malloc(sizeof(*item)); + if (!item) + return -1; + item->guid = guid; + item->name = strdup(p); + cl_qmap_insert(map, item->guid, (cl_map_item_t *)item); + return 0; } nn_map_t * open_node_name_map(char *node_name_map) { - FILE *tmp_fp = NULL; - nn_map_t *rc = NULL; - - if (node_name_map != NULL) { - tmp_fp = fopen(node_name_map, "r"); - if (tmp_fp == NULL) { - fprintf(stderr, - "WARNING failed to open switch map \"%s\" (%s)\n", - node_name_map, strerror(errno)); - } + nn_map_t *map; + + if (!node_name_map) { #ifdef HAVE_DEFAULT_NODENAME_MAP - } else { - tmp_fp = fopen(HAVE_DEFAULT_NODENAME_MAP, "r"); + struct stat buf; + node_name_map = HAVE_DEFAULT_NODENAME_MAP; + if (stat(node_name_map, &buf)) + return NULL; +#else + return NULL; #endif /* HAVE_DEFAULT_NODENAME_MAP */ } - if (!tmp_fp) - return (NULL); - - rc = malloc(sizeof(*rc)); - if (!rc) - return (NULL); - rc->fp = tmp_fp; - cl_qmap_init(&(rc->map)); - read_names(rc); - return (rc); + + map = malloc(sizeof(*map)); + if (!map) + return NULL; + cl_qmap_init(map); + + if (parse_node_map(node_name_map, map_name, map)) { + fprintf(stderr, + "WARNING failed to open node name map \"%s\" (%s)\n", + node_name_map, strerror(errno)); + close_node_name_map(map); + return NULL; + } + + return map; } void @@ -115,15 +103,13 @@ close_node_name_map(nn_map_t *map) if (!map) return; - item = (name_map_item_t *)cl_qmap_head(&(map->map)); - while (item != (name_map_item_t *)cl_qmap_end(&(map->map))) { - item = (name_map_item_t *)cl_qmap_remove(&(map->map), item->guid); + item = (name_map_item_t *)cl_qmap_head(map); + while (item != (name_map_item_t *)cl_qmap_end(map)) { + item = (name_map_item_t *)cl_qmap_remove(map, item->guid); free(item->name); free(item); - item = (name_map_item_t *)cl_qmap_head(&(map->map)); + item = (name_map_item_t *)cl_qmap_head(map); } - if (map->fp) - fclose(map->fp); free(map); } @@ -136,8 +122,8 @@ remap_node_name(nn_map_t *map, uint64_t target_guid, char *nodedesc) if (!map) goto done; - item = (name_map_item_t *)cl_qmap_get(&(map->map), target_guid); - if (item != (name_map_item_t *)cl_qmap_end(&(map->map))) + item = (name_map_item_t *)cl_qmap_get(map, target_guid); + if (item != (name_map_item_t *)cl_qmap_end(map)) rc = strdup(item->name); done: diff --git a/opensm/include/complib/cl_nodenamemap.h b/opensm/include/complib/cl_nodenamemap.h index 5f4a026..9b2ada4 100644 --- a/opensm/include/complib/cl_nodenamemap.h +++ b/opensm/include/complib/cl_nodenamemap.h @@ -47,10 +47,8 @@ typedef struct _name_map_item { uint64_t guid; char *name; } name_map_item_t; -typedef struct _node_name_map { - FILE *fp; - cl_qmap_t map; -} nn_map_t; + +typedef cl_qmap_t nn_map_t; /** * Node name map interface. -- 1.5.4.1.122.gaa8d From sashak at voltaire.com Sun Mar 30 07:29:28 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 30 Mar 2008 14:29:28 +0000 Subject: [ofa-general] Re: [PATCH 1/6] complib/nodenamemap: add generic parse_node_map() function In-Reply-To: <20080329094553.GV13708@sashak.voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> <1206577637-18226-2-git-send-email-sashak@voltaire.com> <20080326175228.0cee3e5a.weiny2@llnl.gov> <20080327085429.GA23849@sashak.voltaire.com> <20080327143534.01150523.weiny2@llnl.gov> <20080329094553.GV13708@sashak.voltaire.com> Message-ID: <20080330142928.GL13708@sashak.voltaire.com> On 09:45 Sat 29 Mar , Sasha Khapyorsky wrote: > > Correct, it is slightly different today. By "merging" I meant format > merge as well. Finally I found time for doing this. The patches to the list. Sahsa From sashak at voltaire.com Sun Mar 30 06:32:01 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 30 Mar 2008 13:32:01 +0000 Subject: [ofa-general] [PATCH] opensm: make formats of node map names and up/down guid ids files identical In-Reply-To: <20080329094553.GV13708@sashak.voltaire.com> References: <1206577637-18226-1-git-send-email-sashak@voltaire.com> <1206577637-18226-2-git-send-email-sashak@voltaire.com> <20080326175228.0cee3e5a.weiny2@llnl.gov> <20080327085429.GA23849@sashak.voltaire.com> <20080327143534.01150523.weiny2@llnl.gov> <20080329094553.GV13708@sashak.voltaire.com> Message-ID: <20080330133201.GI13708@sashak.voltaire.com> Signed-off-by: Sasha Khapyorsky --- opensm/complib/cl_nodenamemap.c | 8 +++----- opensm/opensm/osm_ucast_updn.c | 3 ++- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/opensm/complib/cl_nodenamemap.c b/opensm/complib/cl_nodenamemap.c index 696d455..d09863a 100644 --- a/opensm/complib/cl_nodenamemap.c +++ b/opensm/complib/cl_nodenamemap.c @@ -170,7 +170,7 @@ int parse_node_map(const char *file_name, if (!(f = fopen(file_name, "r"))) return -1; - while (fgets(line, sizeof(line),f)) { + while (fgets(line, sizeof(line), f)) { uint64_t guid; char *p, *e; @@ -181,18 +181,16 @@ int parse_node_map(const char *file_name, continue; guid = strtoull(p, &e, 0); - if (e == p || !isspace(*e)) { + if (e == p || (!isspace(*e) && *e != '#' && *e != '\0')) { fclose(f); return -1; } p = e; - if (*e) - e++; while (isspace(*p)) p++; - e = strpbrk(p, "# \t\n"); + e = strpbrk(p, "\n"); if (e) *e = '\0'; diff --git a/opensm/opensm/osm_ucast_updn.c b/opensm/opensm/osm_ucast_updn.c index 5e778c2..0c4df5d 100644 --- a/opensm/opensm/osm_ucast_updn.c +++ b/opensm/opensm/osm_ucast_updn.c @@ -49,6 +49,7 @@ #endif /* HAVE_CONFIG_H */ #include +#include #include #include #include @@ -446,7 +447,7 @@ static int update_id(void *cxt, uint64_t guid, char *p) } id = strtoull(p, &e, 0); - if (*e) { + if (*e && !isspace(*e)) { OSM_LOG(&osm->log, OSM_LOG_ERROR, "ERR: cannot parse node id \'%s\'", p); return -1; -- 1.5.4.1.122.gaa8d From richardn at theinternet.net Sun Mar 30 05:15:32 2008 From: richardn at theinternet.net (aldous abner) Date: Sun, 30 Mar 2008 12:15:32 +0000 Subject: [ofa-general] Paris gives a BJ to Justin Timberlake Message-ID: <000501c8926e$0284fe64$a7f2beb9@urkdomnr> fRSaaVaSMlX Download and WatchokFHfRSaaV -------------- next part -------------- An HTML attachment was scrubbed... URL: From ferricyanic at eiserloh.net Sun Mar 30 08:05:42 2008 From: ferricyanic at eiserloh.net (Vassos Wong) Date: Sun, 30 Mar 2008 17:05:42 +0200 Subject: [ofa-general] Ado6e Master Sui+e for XP|Vls+a 299, Retail 2499 (save 2199) Message-ID: <000801c89276$179cb000$0100007f@hbmsymt> v!sit newxpdeal. com in Internet Exp|orer 2007microsoft office enterprise - 79 microsoft sql server developer edition 2005 - 69 microsoft sql server developer edition 2005 - 69 solid edge v17 - 69 sony sound forge 9.0 - 49 adobe encore dvd 2 - 49 sony vegas 6 - 69 autodesk architectural studio 3.0 - 39 2003 microsoft office professional with business contact manager for outlook - 69 macrovision installshield premier edition 2008 - 199 creative suite standard - 99 autodesk 3ds max 9.0 - 149 ibm lotus smartsuite millenium edition release 9.8 - 39 adobe creative suite 3 master collection for win - 299 adobe dreamweaver cs3 - 59 From wlouis at ms3.hinet.net Sun Mar 30 06:21:10 2008 From: wlouis at ms3.hinet.net (chancey saifalla) Date: Sun, 30 Mar 2008 13:21:10 +0000 Subject: [ofa-general] top of the top Message-ID: <000501c89277$02ed8806$2788a98f@prruhh> " My order arrived yesterday via registered mail in good order THE WATCH IS BEAUTIFUL AND EVEN BETTER THAN I EXPECTED." Try it for yourself - u will be amazed!! - The worlds largest online retailer of luxury products, including: Rolex Sports Models Rolex Datejusts Breitling Cartier Porsche Design Dolce & Gabbana Dior Gucci Hermes Watches Patek Philippe Visit - www.wathceaa.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mejiadd at allegheny.com Sun Mar 30 06:21:43 2008 From: mejiadd at allegheny.com (anders hilton) Date: Sun, 30 Mar 2008 13:21:43 +0000 Subject: [ofa-general] New Britney P*ssy shot Message-ID: <000801c89277$04287417$5c00378c@qiygusg> DrEWcGwGYX Download and WatchbNRSvDrEWc -------------- next part -------------- An HTML attachment was scrubbed... URL: From HilaryeternitySwenson at merriam-webster.com Sun Mar 30 09:45:02 2008 From: HilaryeternitySwenson at merriam-webster.com (Terrie Purvis) Date: Sun, 30 Mar 2008 16:45:02 +0000 Subject: [ofa-general] Breaking News Message-ID: <4IX693EJXVWDA060@merriam-webster.com> An HTML attachment was scrubbed... URL: From JamesdarwinianArredondo at firstmonday.org Sun Mar 30 01:51:07 2008 From: JamesdarwinianArredondo at firstmonday.org (Chasity Bonds) Date: Sun, 30 Mar 2008 12:51:07 +0400 Subject: [ofa-general] Rocket Stock Report Message-ID: <4IX416EJXVWDA600@firstmonday.org> Introducing the company with a unique concept Red Branch Technologies Symbol: RBTI Red Branch Technologies, Inc. makes business travel easier, more secure and more responsive for both the hard-charging business traveler and the corporation by meeting travel needs at each point in the travel cycle. The company's innovative my/mTravel(r) and mTravel(r) products automate the business travel process from planning and booking to en route services and support, thru post travel reporting and unused ticket redemption. Red Branch's Magellan360 provides agency and net-delivered back office services to independent professional travel marketers. The company has real clients and income stream. Look at their recent PR's - Red Branch Technologies Teams With DeskPort Technologies to Introduce 60,000 Business Travelers to my/mTravel - Red Branch Technologies Gives Business Travelers Credit Card Fraud and Identity Theft Protection With Introduction of IdentiFlyer - Red Branch Technologies Wholly-Owned Subsidiary, Magellan360, Releases Unaudited Financial Performance for 2007; Revenue Up 12 Percent to $22 Million Do your due diligence on RBTI, and read their conferece call details you'll see why we think this is the undiscovered gem for double digit gains. Forward looking statements are based on expectations, estimates and projections at the time the statements are made that involve a number of risks and uncertainties which could cause actual results or events to differ materially from those presently anticipated From jameskirkjs9828222 at gmail.com Sun Mar 30 11:08:03 2008 From: jameskirkjs9828222 at gmail.com (James Kirk) Date: Sun, 30 Mar 2008 19:08:03 +0100 Subject: [ofa-general] James Kirk Message-ID: Dear Good Friend, Sorry for my late response.Check the attached file for update. Best Regards James Kirk -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: James Kirk.rtf Type: application/rtf Size: 5827 bytes Desc: not available URL: From sashak at voltaire.com Sun Mar 30 16:21:19 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 30 Mar 2008 23:21:19 +0000 Subject: [ofa-general] [RFC][PATCH] opensm/configure.in: improve readability of configured config files Message-ID: <20080330232119.GM13708@sashak.voltaire.com> When ./configure script is executed it will show the values which are used for those config files, like this: checking for --with-opensm-conf-sub-dir... opensm checking for --with-node-name-map ... ib-node-name-map checking for --with-partitions-conf... partitions.conf checking for --with-qos-policy-conf... qos-policy.conf And not just that it was or wasn't redefined from its default values (checking for --with-partitions-conf... no , etc). Signed-off-by: Sasha Khapyorsky --- Another ideas? Maybe to print full path at least for opensm-conf-sub-dir, like: /usr/local/etc/opensm? Sasha opensm/configure.in | 12 ++++-------- 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/opensm/configure.in b/opensm/configure.in index a5b7c5a..d88bfd4 100644 --- a/opensm/configure.in +++ b/opensm/configure.in @@ -92,12 +92,11 @@ AC_ARG_WITH(opensm-conf-sub-dir, no) ;; *) - withopensmconfsubdir=yes OPENSM_CONF_SUB_DIR=$withval ;; esac ] ) -AC_MSG_RESULT(${withopensmconfsubdir=no}) +AC_MSG_RESULT($OPENSM_CONF_SUB_DIR) AC_SUBST(OPENSM_CONF_SUB_DIR) dnl Set up /opensm config dir. @@ -120,12 +119,11 @@ AC_ARG_WITH(node-name-map, no) ;; *) - withnodenamemap=yes NODENAMEMAPFILE=$withval ;; esac ] ) -AC_MSG_RESULT(${withnodenamemap=no}) +AC_MSG_RESULT($NODENAMEMAPFILE) AC_DEFINE_UNQUOTED(HAVE_DEFAULT_NODENAME_MAP, ["$CONF_DIR/$NODENAMEMAPFILE"], [Define a default node name map file]) @@ -141,12 +139,11 @@ AC_ARG_WITH(partitions-conf, no) ;; *) - withpartitionsconf=yes PARTITION_CONFIG_FILE=$withval ;; esac ] ) -AC_MSG_RESULT(${withpartitionsconf=no}) +AC_MSG_RESULT($PARTITION_CONFIG_FILE) AC_DEFINE_UNQUOTED(HAVE_DEFAULT_PARTITION_CONFIG_FILE, ["$CONF_DIR/$PARTITION_CONFIG_FILE"], [Define a Partition config file]) @@ -162,12 +159,11 @@ AC_ARG_WITH(qos-policy-conf, no) ;; *) - withqospolicyconf=yes QOS_POLICY_FILE=$withval ;; esac ] ) -AC_MSG_RESULT(${withqospolicyconf=no}) +AC_MSG_RESULT($QOS_POLICY_FILE) AC_DEFINE_UNQUOTED(HAVE_DEFAULT_QOS_POLICY_FILE, ["$CONF_DIR/$QOS_POLICY_FILE"], [Define a QOS policy config file]) -- 1.5.4.1.122.gaa8d From Uosrdjana.janosevic at sbc.sc Sun Mar 30 14:40:15 2008 From: Uosrdjana.janosevic at sbc.sc (fons charlton) Date: Sun, 30 Mar 2008 21:40:15 +0000 Subject: [ofa-general] To: general Message-ID: <000601c892bd$0275c93a$0d347495@nfpevxr> Voted the #1 most effective male enlargement supplement by FHM readers: V_P_X_L is guaranteed to add inches within just a few short weeks. In addition to permanent, massive gains to length, you can also expect: 1. up to 20-30% gains in girth 2. Longer sex, never subside penis 3. More volume of ejaculate Satisfaction and results are absolutely guaranteed. Why hesitate?Try the only Doctor endorsed Herbal enlargement supplement today! Click here: http://burieuir.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From interest at mtroncomponents.com Sun Mar 30 18:12:24 2008 From: interest at mtroncomponents.com (interest at mtroncomponents.com) Date: Mon, 31 Mar 2008 09:12:24 +0800 Subject: [ofa-general] Hear her screaming your name in passion! Message-ID: <002e01c892cc$47734720$ec209e9c@hscd> A safe, reliable anti-ED. http://jbovrydxutdaa.blogspot.com From kliteyn at mellanox.co.il Sun Mar 30 16:35:18 2008 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 31 Mar 2008 02:35:18 +0300 Subject: [ofa-general] [RFC][PATCH] opensm/configure.in: improve readability of configured config files In-Reply-To: <20080330232119.GM13708@sashak.voltaire.com> References: <20080330232119.GM13708@sashak.voltaire.com> Message-ID: <47F023B6.2070302@mellanox.co.il> Hi Sasha, Sasha Khapyorsky wrote: > When ./configure script is executed it will show the values which are > used for those config files, like this: > > checking for --with-opensm-conf-sub-dir... opensm > checking for --with-node-name-map ... ib-node-name-map > checking for --with-partitions-conf... partitions.conf > checking for --with-qos-policy-conf... qos-policy.conf > > And not just that it was or wasn't redefined from its default values > (checking for --with-partitions-conf... no , etc). > > Signed-off-by: Sasha Khapyorsky > --- > > Another ideas? Maybe to print full path at least for > opensm-conf-sub-dir, like: /usr/local/etc/opensm? > Actually, printing full path for opensm-conf-sub-dir is a great idea! Moreover, I wouldn't mind seeing the full path for all the config files, but that would be less important than the config dir. -- Yevgeny > Sasha > > opensm/configure.in | 12 ++++-------- > 1 files changed, 4 insertions(+), 8 deletions(-) > > diff --git a/opensm/configure.in b/opensm/configure.in > index a5b7c5a..d88bfd4 100644 > --- a/opensm/configure.in > +++ b/opensm/configure.in > @@ -92,12 +92,11 @@ AC_ARG_WITH(opensm-conf-sub-dir, > no) > ;; > *) > - withopensmconfsubdir=yes > OPENSM_CONF_SUB_DIR=$withval > ;; > esac ] > ) > -AC_MSG_RESULT(${withopensmconfsubdir=no}) > +AC_MSG_RESULT($OPENSM_CONF_SUB_DIR) > AC_SUBST(OPENSM_CONF_SUB_DIR) > > dnl Set up /opensm config dir. > @@ -120,12 +119,11 @@ AC_ARG_WITH(node-name-map, > no) > ;; > *) > - withnodenamemap=yes > NODENAMEMAPFILE=$withval > ;; > esac ] > ) > -AC_MSG_RESULT(${withnodenamemap=no}) > +AC_MSG_RESULT($NODENAMEMAPFILE) > AC_DEFINE_UNQUOTED(HAVE_DEFAULT_NODENAME_MAP, > ["$CONF_DIR/$NODENAMEMAPFILE"], > [Define a default node name map file]) > @@ -141,12 +139,11 @@ AC_ARG_WITH(partitions-conf, > no) > ;; > *) > - withpartitionsconf=yes > PARTITION_CONFIG_FILE=$withval > ;; > esac ] > ) > -AC_MSG_RESULT(${withpartitionsconf=no}) > +AC_MSG_RESULT($PARTITION_CONFIG_FILE) > AC_DEFINE_UNQUOTED(HAVE_DEFAULT_PARTITION_CONFIG_FILE, > ["$CONF_DIR/$PARTITION_CONFIG_FILE"], > [Define a Partition config file]) > @@ -162,12 +159,11 @@ AC_ARG_WITH(qos-policy-conf, > no) > ;; > *) > - withqospolicyconf=yes > QOS_POLICY_FILE=$withval > ;; > esac ] > ) > -AC_MSG_RESULT(${withqospolicyconf=no}) > +AC_MSG_RESULT($QOS_POLICY_FILE) > AC_DEFINE_UNQUOTED(HAVE_DEFAULT_QOS_POLICY_FILE, > ["$CONF_DIR/$QOS_POLICY_FILE"], > [Define a QOS policy config file]) > From qu66jw43y64dpb at gmail.com Sun Mar 30 22:15:43 2008 From: qu66jw43y64dpb at gmail.com (=?BIG5?B?s6+mTq3r?=) Date: Mon, 31 Mar 2008 13:15:43 +0800 Subject: [ofa-general] =?big5?b?t1GwZaX3r1OnT6q6wqeqq6FBtbmxerPMr1OnT6q6?= =?big5?b?saGkSLbcoUg=?= Message-ID: <6574daa00803302215t21491edfu1fd5907921179b86@mail.gmail.com> 您想給心愛的另ㄧ半ㄧ份驚喜嗎？ 近來看看，試試這個喔^.^ 珍珠貝殼項鍊禮盒　∼　日本九州空運來台 網址在這→203.70.179.39/ju←複製這行網址貼上即可 ∴侔裸F7iBb From melvinl at nutraceuticals.net Sun Mar 30 21:50:22 2008 From: melvinl at nutraceuticals.net (barnie bridgett) Date: Mon, 31 Mar 2008 04:50:22 +0000 Subject: [ofa-general] Rihanna exposed Message-ID: <000701c892f9$0645e8df$7f0556b2@pyeqjpa> ClOxmRnErv Download and WatchFcguUClOxm -------------- next part -------------- An HTML attachment was scrubbed... URL: From a-akin at 7flagscarwash.com Mon Mar 31 00:07:07 2008 From: a-akin at 7flagscarwash.com (Orlando Albert) Date: Mon, 31 Mar 2008 09:07:07 +0200 Subject: [ofa-general] I'd like to show you my pic Message-ID: <01c8930e$980eef80$bb45b04e@a-akin> Hello! I am tired tonight. I am nice girl that would like to chat with you. Email me at Anita at jolasite.com only, because I am using my friend's email to write this. I will show you some of my private pictures From phggm at blondeguy.com Mon Mar 31 00:55:27 2008 From: phggm at blondeguy.com (Aida Wilson) Date: Mon, 31 Mar 2008 14:55:27 +0700 Subject: [ofa-general] Be like sex machine Message-ID: <01c8933f$416dd980$1102167b@phggm> Take her to love heaven Please look attached file and know MORE ABOUT THIS! -------------- next part -------------- A non-text attachment was scrubbed... Name: secrets.zip Type: application/zip Size: 634 bytes Desc: not available URL: From draw4 at web.de Mon Mar 31 01:51:21 2008 From: draw4 at web.de (Draw Lot) Date: Mon, 31 Mar 2008 10:51:21 +0200 Subject: [ofa-general] Uk National Lottery Office You Won. Message-ID: <589561985@web.de> >From Uk National Lottery Office [A]. P O Box 1610 Liverpool, L70 1NL UNITED KINGDOM Ref: UK/9420X/07 Batch: 074/05/ZY369 Congratulation! Attn: Beneficiary, We happily announce to you the draw of the UK NATIONAL LOTTERY held on 31 of March 2008. Your e-mail address attached to ticket number: 5647560054 188 with Serial number 5368/02 jack pot lotto winners drew the lucky numbers: 15 19 29 35 44 45 bonus no 7. Which subsequently won you the lottery in the 2nd category i.e. match 5 plus bonus. All participants for the online version were selected randomly from World Wide Web sites through computer email draw system and extracted from over 100,000 unions, associations, and corporate bodies that are listed online. This promotion takes place weekly, and your lucky winning number falls within our European booklet representative office in Europe as indicated in your play coupon. You have therefore been approved to claim a total sum of £809,000.00 (Eight Hundred and Nine Thousand British Pounds Sterling) in cash credited to file KTU/9023118308/03 in a total sum of £3,236,000.00 (Three million Two Hundred and thirty Six Thousand British pounds sterling) lotto jackpot shared amongst the first four (4) lucky winners in this category. Please do contact your Claim Agent in London United Kingdom: Name: Dr. Theo Klaas (Claim Agent). Address: P O Box 1610 Liverpool, L70 1NL UNITED KINGDOM. Tel. Number +44 7031 862564 Tel. Number +44 7031 862562 Fax. Number +44 8709740605 Email: dr.theo20105 at yahoo.co.uk [mailto:dr.theo20105 at yahoo.co.uk] As you are contacting your claim agent; please quote your reference, batch and winning number which can be found below; left corner of this notification as well as your full names and address to help locate your file easily. 1.Name in full.........................................., 2.Address..............................................., 3.Age...................................................., 4.Sex...................................................., 5.Occupation.............................................., 6.Phone/Fax................................................, Ref: UK/9420X/07 Batch: 074/05/ZY369 Congratulation! Mrs. Rose Allen (Coordinator). Unbegrenzter Speicherplatz für Ihr E-Mail Postfach? Jetzt aktivieren! *http://freemail.web.de/club/landingpage.htm/?mc=025555* [http://freemail.web.de/club/landingpage.htm/?mc=025555] -------------- next part -------------- An HTML attachment was scrubbed... URL: From abbott at angelsforum.com Mon Mar 31 03:15:04 2008 From: abbott at angelsforum.com (Leah Harmon) Date: Mon, 31 Mar 2008 11:15:04 +0100 Subject: [ofa-general] Nur die Billige Software Message-ID: <01c89320$77e82c00$63386852@abbott> MS Office cheap as chipsBekommen Sie Ihre Software unverzueglich. Einfach zahlen und sofort runterladen. Hier sind Programme in allen europaeischen Sprachen verfuegbar, programmiert fuer Windows und Macintosh. Alle Softwaren sind sehr guenstig, es handelt sich dabei garantiert um originale, komplette und voellig funktionale Versionen. http://geocities.com/gailfoster65/* Adobe Acrobat 8.0 Professional: $69.95 * Office Enterprise 2007: $79.95 * Office System Professional 2003 (5 Cds): $59.95 * Office System Professional 2003 (5 Cds): $59.95 http://geocities.com/gailfoster65/Unsere Kundenberater sind immer bereit Ihnen bei der Installation zu helfen. Wir antworten sehr schnell und geben Ihnen auch Geld-Zurueck-Garantie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From CarissagaggleNaquin at prisontalk.com Sun Mar 30 16:16:55 2008 From: CarissagaggleNaquin at prisontalk.com (Lauri Flint) Date: Mon, 31 Mar 2008 05:16:55 +0600 Subject: [ofa-general] Hot Stock Talk Message-ID: <9IX910EJXVWDA490@prisontalk.com> An HTML attachment was scrubbed... URL: From Lucas702 at mayridge.com Mon Mar 31 03:39:30 2008 From: Lucas702 at mayridge.com (=?koi8-r?B?IuHMxcvTycXOy88g6S72LiI=?=) Date: Mon, 31 Mar 2008 11:39:30 +0100 Subject: [ofa-general] =?koi8-r?b?8NLFxNDSydHUydEg68nUwdE=?= Message-ID: <01c89323$e1b60500$c3090753@Lucas702> Справочник заводов и фабрик Китая за 2007 - 2008 год. Найдите своего поставщика, партнера, интересный товар. http://orackristin.tripod.com Приглашаются распространители. -------------- next part -------------- An HTML attachment was scrubbed... URL: From RonaldbrazierAllen at ibiblio.org Mon Mar 31 08:58:01 2008 From: RonaldbrazierAllen at ibiblio.org (Jeffrey Nelson) Date: Mon, 31 Mar 2008 13:58:01 -0200 Subject: [ofa-general] Investor Insight Message-ID: <2IX657EJXVWDA658@ibiblio.org> Introducing the new tech giant in the making DNC MultiMedia Corp Symbol: DCNM This is a hugely undervalued Gem with high profile clients such as Microsoft, Samsung, LG, and Napster the huge volume and price spike today on the news release is a good indicator of short and long term movement. See the company's outstanding pr and pull the trigger on DCNM From a-amyhu at abeka.com Mon Mar 31 04:31:59 2008 From: a-amyhu at abeka.com (Bud Landers) Date: Mon, 31 Mar 2008 19:31:59 +0800 Subject: [ofa-general] we met online Message-ID: <01c89365$e3082180$69bfb63c@a-amyhu> Hello! I am bored this evening. I am nice girl that would like to chat with you. Email me at Madeleine at jolasite.com only, because I am using my friend's email to write this. To see my pics From koen.segers at vrt.be Mon Mar 31 04:39:56 2008 From: koen.segers at vrt.be (Koen Segers) Date: Mon, 31 Mar 2008 13:39:56 +0200 Subject: [ofa-general] PCI-Express payload Message-ID: <1206963596.8457.17.camel@koenVRT> Hi, I'm interested in computing the overhead coming from PCI-Express to an IB HCA. Therefore I need to know the payload size of different types of HCA. We have InfiniHost III Mellanox cards of 4x SDR and DDR IB. According the lspci -vvv command on SLES 10 SP1, the PCI-e payload of these cards is maximum 128 bytes. Can someone give my the payload size for a ConnectX 4x SDR and DDR IB card? Thank you, Koen This is my output: 41:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (rev a0) Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- From TheodorecupfulReyes at sourcewatch.org Mon Mar 31 18:54:01 2008 From: TheodorecupfulReyes at sourcewatch.org (Lloyd Mills) Date: Mon, 31 Mar 2008 18:54:01 -0700 Subject: [ofa-general] SuperNova Stocks Message-ID: <3IX283EJXVWDA987@sourcewatch.org> Introducing the new tech giant in the making DNC MultiMedia Corp Symbol: DCNM This is a hugely undervalued Gem with high profile clients such as Microsoft, Samsung, LG, and Napster the huge volume and price spike today on the news release is a good indicator of short and long term movement. See the company's outstanding pr and pull the trigger on DCNM From rdreier at cisco.com Mon Mar 31 04:56:21 2008 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 31 Mar 2008 04:56:21 -0700 Subject: [ofa-general] Re: linux-next: infiniband merge conflicts References: <20080331130632.bd08315f.sfr@canb.auug.org.au> Message-ID: > Today's linux-next merge of the infiniband tree produced conflicts in > drivers/infiniband/core/cma.c and drivers/infiniband/core/cm.c. The > conflicts were against Al Viro's patch "trivial endianness annotations: > infiniband core" (Linus' commit id > 1b90c137cc2a0e9b813a8ae316827c493c664146). They are reasonably trivial. OK, I've rebased my tree on top of Linus's current tree and fixed up the conflicts. I do wonder why endianness annotations are being merged post -rc7. If anyone had sent them to me I would have waited for the 2.6.26 merge window to send them to Linus. Which also leads to the question of why these patches, which are a subset of the annotations I've had sitting in my tree (and in -next) for quite a while, were so urgent that they had to bypass my tree... Oh well, water under the bridge. As you said the conflicts were trivial so it's not a big deal to fix them up. - R. From dotanb at dev.mellanox.co.il Mon Mar 31 05:05:24 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Mon, 31 Mar 2008 15:05:24 +0300 Subject: [ofa-general] PCI-Express payload In-Reply-To: <1206963596.8457.17.camel@koenVRT> References: <1206963596.8457.17.camel@koenVRT> Message-ID: <47F0D384.9040706@dev.mellanox.co.il> Hi. You can check what is the maximum supported MTU of your HCA using ibv_devinfo. Most of the HCAs supports 2KB MTU and some of them 4KB MTU. The MTU is the maximum payload size that can be sent/received (the space for the IB headers are not part of the MTU) Every connected QP (RC/UC) can define which MTU value he is using (according to the used path). I hope that is the info you asked for .. Dotan Koen Segers wrote: > Hi, > > I'm interested in computing the overhead coming from PCI-Express to an > IB HCA. Therefore I need to know the payload size of different types > of HCA. > > We have InfiniHost III Mellanox cards of 4x SDR and DDR IB. According > the lspci -vvv command on SLES 10 SP1, the PCI-e payload of these > cards is maximum 128 bytes. > > Can someone give my the payload size for a ConnectX 4x SDR and DDR IB > card? > > Thank you, > > Koen > > This is my output: > 41:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > (rev a0) > Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > ParErr+ Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > SERR- Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 209 > Region 0: Memory at e1700000 (64-bit, non-prefetchable) [size=1M] > Region 2: Memory at e1800000 (64-bit, prefetchable) [size=8M] > Capabilities: [40] Power Management version 2 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA > PME(D0-,D1-,D2-,D3hot-,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [48] Vital Product Data > Capabilities: [90] Message Signalled Interrupts: Mask- 64bit+ > Queue=0/5 Enable- > Address: 0000000000000000 Data: 0000 > Capabilities: [84] MSI-X: Enable+ Mask- TabSize=32 > Vector table: BAR=0 offset=00082000 > PBA: BAR=0 offset=00082200 > Capabilities: [60] Express Endpoint IRQ 0 > * Device: Supported: MaxPayload 128 bytes, PhantFunc 0, > ExtTag+* > Device: Latency L0s <64ns, L1 unlimited > Device: AtnBtn- AtnInd- PwrInd- > Device: Errors: Correctable- Non-Fatal+ Fatal+ > Unsupported- > Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- > Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes > Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s, Port 8 > Link: Latency L0s unlimited, L1 unlimited > Link: ASPM Disabled RCB 128 bytes CommClk- ExtSynch- > Link: Speed 2.5Gb/s, Width x8 > > > *** Disclaimer *** > > Vlaamse Radio- en Televisieomroep > Auguste Reyerslaan 52, 1043 Brussel > > nv van publiek recht > BTW BE 0244.142.664 > RPR Brussel > http://www.vrt.be/disclaimer > ------------------------------------------------------------------------ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From pronunciationq5 at mosesmotorco.com Mon Mar 31 05:08:14 2008 From: pronunciationq5 at mosesmotorco.com (Dianne Medeiros) Date: Mon, 31 Mar 2008 12:08:14 +0000 Subject: [ofa-general] Women prefer big penis for more sexual satisfaction. Message-ID: <01c89327$e54b8b00$5310f059@pronunciationq5> Purified Trivang 20 mg http://lariverpo.com From hrosenstock at xsigo.com Mon Mar 31 05:37:02 2008 From: hrosenstock at xsigo.com (Hal Rosenstock) Date: Mon, 31 Mar 2008 05:37:02 -0700 Subject: [ofa-general] RE: [PATCH] ofed/docs: some fixes in OpenSM release notes In-Reply-To: <6C2C79E72C305246B504CBA17B5500C903A7A1CD@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C903A7A1CD@mtlexch01.mtl.com> Message-ID: <1206967022.8048.32.camel@hrosenstock-ws.xsigo.com> On Mon, 2008-03-31 at 13:45 +0300, Tziporet Koren wrote: > Done > Tziporet Shouldn't this come from Sasha's OFED 1.3 OpenSM release notes ? (There were some other minor changes). Also, I think Sasha's latest OFED 1.3 branch needs pulling into OFED by Vlad. -- Hal > -----Original Message----- > From: Yevgeny Kliteynik [mailto:kliteyn at dev.mellanox.co.il] > Sent: Tuesday, March 25, 2008 11:08 AM > To: Tziporet Koren; Tziporet Koren > Cc: Hal Rosenstock; Sasha Khapyorsky; OpenIB > Subject: [PATCH] ofed/docs: some fixes in OpenSM release notes > > Hi Tziporet, > > Added ConnectX to list of supported devices, fixed FW versions > to match OFED release notes, and removed device IDs (no reason > for them to be there, OFED release notes doesn't have them). > > Please apply OFED 1.3 docs. > > Signed-off-by: Yevgeny Kliteynik > --- > opensm_release_notes.txt | 17 +++++++++-------- > 1 files changed, 9 insertions(+), 8 deletions(-) > > diff --git a/opensm_release_notes.txt b/opensm_release_notes.txt > index 825d758..bdefc11 100644 > --- a/opensm_release_notes.txt > +++ b/opensm_release_notes.txt > @@ -459,14 +459,15 @@ Table 3 - Qualified Devices and Corresponding > Firmware > ====================================================== > > Mellanox > -Device | FW versions > ---------|----------------------------------------------------------- > -MT43132 | InfiniScale - fw-43132 5.2.0 (and later) > -MT47396 | InfiniScale III - fw-47396 0.5.0 (and later) > -MT23108 | InfiniHost - fw-23108 3.3.2 (and later) > -MT25204 | InfiniHost III Lx - fw-25204 1.0.1i (and later) > -MT25208 | InfiniHost III Ex (InfiniHost Mode) - fw-25208 4.6.2 (and > later) > -MT25208 | InfiniHost III Ex (MemFree Mode) - fw-25218 5.0.1 (and > later) > +Device | FW versions > +------------------------------------|------------------------------- > +InfiniScale | fw-43132 5.2.000 (and later) > +InfiniScale III | fw-47396 0.5.000 (and later) > +InfiniHost | fw-23108 3.5.000 (and later) > +InfiniHost III Lx | fw-25204 1.2.000 (and later) > +InfiniHost III Ex (InfiniHost Mode) | fw-25208 4.8.200 (and later) > +InfiniHost III Ex (MemFree Mode) | fw-25218 5.3.000 (and later) > +ConnectX IB | fw-25408 2.3.000 (and later) > > QLogic/PathScale > Device | Note From sfr at canb.auug.org.au Sun Mar 30 19:06:32 2008 From: sfr at canb.auug.org.au (Stephen Rothwell) Date: Mon, 31 Mar 2008 13:06:32 +1100 Subject: [ofa-general] linux-next: infiniband merge conflicts Message-ID: <20080331130632.bd08315f.sfr@canb.auug.org.au> Hi Roland, Today's linux-next merge of the infiniband tree produced conflicts in drivers/infiniband/core/cma.c and drivers/infiniband/core/cm.c. The conflicts were against Al Viro's patch "trivial endianness annotations: infiniband core" (Linus' commit id 1b90c137cc2a0e9b813a8ae316827c493c664146). They are reasonably trivial. -- Cheers, Stephen Rothwell sfr at canb.auug.org.au http://www.canb.auug.org.au/~sfr/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From kliteyn at dev.mellanox.co.il Sun Mar 30 23:29:54 2008 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 31 Mar 2008 09:29:54 +0300 Subject: [ofa-general] [general-bounces@lists.openfabrics.org: Your message to general awaits moderator approval] In-Reply-To: <20080329122249.GA13708@sashak.voltaire.com> References: <20080329122249.GA13708@sashak.voltaire.com> Message-ID: <47F084E2.1000800@dev.mellanox.co.il> Sasha Khapyorsky wrote: > Hi, > > Now I get such messages on each post to . Same here. Any news on this? -- Yevgeny > Is our list moderated unconditionally? > > Sasha > > ----- Forwarded message from general-bounces at lists.openfabrics.org ----- > > Date: Sat, 29 Mar 2008 03:12:59 -0700 > From: general-bounces at lists.openfabrics.org > To: sashak at voltaire.com > Subject: Your message to general awaits moderator approval > MIME-Version: 1.0 > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: 7bit > Message-ID: > Precedence: bulk > Sender: general-bounces at lists.openfabrics.org > Errors-To: general-bounces at lists.openfabrics.org > Return-Path: general-bounces at lists.openfabrics.org > > Your mail to 'general' with the subject > > Re: [ofa-general] smpquery vlarb ... dumps only 1 entry? > > Is being held until the list moderator can review it for approval. > > The reason it is being held: > > The message headers matched a filter rule > > Either the message will get posted to the list, or you will receive > notification of the moderator's decision. If you would like to cancel > this posting, please visit the following URL: > > > ----- End forwarded message ----- > ----- Forwarded message from general-bounces at lists.openfabrics.org ----- > > Date: Sat, 29 Mar 2008 03:23:11 -0700 > From: general-bounces at lists.openfabrics.org > To: sashak at voltaire.com > Subject: Your message to general awaits moderator approval > MIME-Version: 1.0 > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: 7bit > Message-ID: > Precedence: bulk > Sender: general-bounces at lists.openfabrics.org > Errors-To: general-bounces at lists.openfabrics.org > Return-Path: general-bounces at lists.openfabrics.org > > Your mail to 'general' with the subject > > Re: [ofa-general] smpquery vlarb ... dumps only 1 entry? > > Is being held until the list moderator can review it for approval. > > The reason it is being held: > > The message headers matched a filter rule > > Either the message will get posted to the list, or you will receive > notification of the moderator's decision. If you would like to cancel > this posting, please visit the following URL: > > > ----- End forwarded message ----- > ----- Forwarded message from general-bounces at lists.openfabrics.org ----- > > Date: Sat, 29 Mar 2008 04:58:27 -0700 > From: general-bounces at lists.openfabrics.org > To: sashak at voltaire.com > Subject: Your message to general awaits moderator approval > MIME-Version: 1.0 > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: 7bit > Message-ID: > Precedence: bulk > Sender: general-bounces at lists.openfabrics.org > Errors-To: general-bounces at lists.openfabrics.org > Return-Path: general-bounces at lists.openfabrics.org > > Your mail to 'general' with the subject > > [PATCH] libibmad/dump: support VLArb table size, fix printing > > Is being held until the list moderator can review it for approval. > > The reason it is being held: > > The message headers matched a filter rule > > Either the message will get posted to the list, or you will receive > notification of the moderator's decision. If you would like to cancel > this posting, please visit the following URL: > > > ----- End forwarded message ----- > ----- Forwarded message from general-bounces at lists.openfabrics.org ----- > > Date: Sat, 29 Mar 2008 04:59:27 -0700 > From: general-bounces at lists.openfabrics.org > To: sashak at voltaire.com > Subject: Your message to general awaits moderator approval > MIME-Version: 1.0 > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: 7bit > Message-ID: > Precedence: bulk > Sender: general-bounces at lists.openfabrics.org > Errors-To: general-bounces at lists.openfabrics.org > Return-Path: general-bounces at lists.openfabrics.org > > Your mail to 'general' with the subject > > [PATCH] infiniband-diags: pass valid VLArb table size to dump func > > Is being held until the list moderator can review it for approval. > > The reason it is being held: > > The message headers matched a filter rule > > Either the message will get posted to the list, or you will receive > notification of the moderator's decision. If you would like to cancel > this posting, please visit the following URL: > > > ----- End forwarded message ----- > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From jackm at dev.mellanox.co.il Mon Mar 31 01:07:39 2008 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 31 Mar 2008 10:07:39 +0200 Subject: [ofa-general] error with ibv_poll_cq() call In-Reply-To: References: <200803260901.25918.jackm@dev.mellanox.co.il> Message-ID: <200803311107.39607.jackm@dev.mellanox.co.il> On Saturday 29 March 2008 08:35, Roland Dreier wrote: > because you leave the clear_qp call before the cq_clean and they are > not within the same cq lock, there is a window where a consumer could > poll a valid CQE for a qp that is no longer in the table.  I think > it's better to really match the logic of the kernel driver (which I > think is correct). > Ouch! You're right, too. BTW, we need the same fix for libmthca. - Jack From vlad at dev.mellanox.co.il Mon Mar 31 01:49:41 2008 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Mon, 31 Mar 2008 11:49:41 +0300 Subject: [ofa-general] [PATCH 1/1 v1] MLX4: Added resize_cq capability. In-Reply-To: References: <47E923CA.90804@dev.mellanox.co.il> Message-ID: <47F0A5A5.2010208@dev.mellanox.co.il> From 4edc320d145f3eaa8d45d7aa7edddb5198a6fd7d Mon Sep 17 00:00:00 2001 From: Vladimir Sokolovsky Date: Tue, 25 Mar 2008 10:23:09 +0200 Subject: [PATCH] MLX4: Added resize_cq capability. Signed-off-by: Vladimir Sokolovsky --- drivers/infiniband/hw/mlx4/cq.c | 287 ++++++++++++++++++++++++++++++++++ drivers/infiniband/hw/mlx4/main.c | 2 + drivers/infiniband/hw/mlx4/mlx4_ib.h | 13 ++ 3 files changed, 302 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 401178f..a9c45c9 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -124,7 +124,10 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector entries = roundup_pow_of_two(entries + 1); cq->ibcq.cqe = entries - 1; buf_size = entries * sizeof (struct mlx4_cqe); + mutex_init(&cq->resize_mutex); spin_lock_init(&cq->lock); + cq->resize_buf = NULL; + cq->resize_umem = NULL; if (context) { struct mlx4_ib_create_cq ucmd; @@ -223,6 +226,236 @@ err_cq: return ERR_PTR(err); } +static int mlx4_alloc_resize_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq *cq, + int entries) +{ + int err; + + if (cq->resize_buf) { + err = -EBUSY; + goto out; + } + + cq->resize_buf = kmalloc(sizeof *cq->resize_buf, GFP_ATOMIC); + if (!cq->resize_buf) { + err = -ENOMEM; + goto out; + } + + err = mlx4_alloc_cq_buf(dev, &cq->resize_buf->buf, entries); + if (err) { + spin_lock_irq(&cq->lock); + kfree(cq->resize_buf); + cq->resize_buf = NULL; + spin_unlock_irq(&cq->lock); + goto out; + } + + cq->resize_buf->cqe = entries - 1; + + return 0; + +out: + return err; +} + +static int mlx4_alloc_resize_umem(struct mlx4_ib_dev *dev, struct mlx4_ib_cq *cq, + int entries, struct ib_udata *udata) +{ + struct mlx4_ib_resize_cq ucmd; + int buf_size; + int err; + + if (cq->resize_umem) { + err = -EBUSY; + goto out; + } + + if (ib_copy_from_udata(&ucmd, udata, sizeof ucmd)) { + err = -EFAULT; + goto out; + } + + cq->resize_buf = kmalloc(sizeof *cq->resize_buf, GFP_ATOMIC); + if (!cq->resize_buf) { + err = -ENOMEM; + goto out; + } + + buf_size = entries * sizeof(struct mlx4_cqe); + + cq->resize_umem = ib_umem_get(cq->umem->context, ucmd.buf_addr, buf_size, + IB_ACCESS_LOCAL_WRITE); + if (IS_ERR(cq->resize_umem)) { + err = PTR_ERR(cq->resize_umem); + goto err_umem; + } + + err = mlx4_mtt_init(dev->dev, ib_umem_page_count(cq->resize_umem), + ilog2(cq->resize_umem->page_size), &cq->resize_buf->buf.mtt); + if (err) + goto err_buf; + + err = mlx4_ib_umem_write_mtt(dev, &cq->resize_buf->buf.mtt, cq->resize_umem); + if (err) + goto err_mtt; + + cq->resize_buf->cqe = entries - 1; + + return 0; + +err_mtt: + mlx4_mtt_cleanup(dev->dev, &cq->resize_buf->buf.mtt); + +err_buf: + ib_umem_release(cq->resize_umem); + cq->resize_umem = NULL; + +err_umem: + kfree(cq->resize_buf); + cq->resize_buf = NULL; + +out: + return err; +} + +int mlx4_get_outstanding_cqes(struct mlx4_ib_cq *cq) +{ + int i; + + i = cq->mcq.cons_index; + while (get_sw_cqe(cq, i & cq->ibcq.cqe)) + ++i; + + return i - cq->mcq.cons_index; +} + +void mlx4_cq_resize_copy_cqes(struct mlx4_ib_cq *cq) +{ + struct mlx4_cqe *cqe; + int i; + + i = cq->mcq.cons_index; + cqe = get_cqe(cq, i & cq->ibcq.cqe); + while ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) != MLX4_CQE_OPCODE_RESIZE) { + memcpy(get_cqe_from_buf(&cq->resize_buf->buf, + (i + 1) & cq->resize_buf->cqe), + get_cqe(cq, i & cq->ibcq.cqe), sizeof(struct mlx4_cqe)); + cqe = get_cqe(cq, ++i & cq->ibcq.cqe); + } + ++cq->mcq.cons_index; +} + +int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) +{ + struct mlx4_ib_dev *dev = to_mdev(ibcq->device); + struct mlx4_ib_cq *cq = to_mcq(ibcq); + struct mlx4_cq_context *context; + struct ib_umem *tumem = NULL; + u64 mtt_addr; + int outst_cqe; + int err; + + mutex_lock(&cq->resize_mutex); + + if (entries < 1 || entries > dev->dev->caps.max_cqes) { + err = -EINVAL; + goto out; + } + + entries = roundup_pow_of_two(entries + 1); + if (entries == ibcq->cqe + 1) { + err = 0; + goto out; + } + + if (ibcq->uobject) { + err = mlx4_alloc_resize_umem(dev, cq, entries, udata); + if (err) + goto out; + } else { + /* Can't be smaller then the number of outstanding CQEs */ + outst_cqe = mlx4_get_outstanding_cqes(cq); + if (entries < outst_cqe + 1) { + err = 0; + goto out; + } + + err = mlx4_alloc_resize_buf(dev, cq, entries); + if (err) + goto out; + } + + context = kzalloc(sizeof *context, GFP_KERNEL); + + if (!context) { + err = -ENOMEM; + goto err_buf; + } + + context->logsize_usrpage = cpu_to_be32(ilog2(entries) << 24); + context->log_page_size = cq->resize_buf->buf.mtt.page_shift - 12; + mtt_addr = mlx4_mtt_addr(dev->dev, &cq->resize_buf->buf.mtt); + context->mtt_base_addr_h = mtt_addr >> 32; + context->mtt_base_addr_l = cpu_to_be32(mtt_addr & 0xffffffff); + + err = mlx4_cq_modify(dev->dev, &cq->mcq, context, 0); + if (err) + goto err_buf; + + spin_lock_irq(&cq->lock); + if (ibcq->uobject) { + cq->buf = cq->resize_buf->buf; + cq->ibcq.cqe = cq->resize_buf->cqe; + tumem = cq->umem; + cq->umem = cq->resize_umem; + + kfree(cq->resize_buf); + cq->resize_buf = NULL; + cq->resize_umem = NULL; + } else { + if (cq->resize_buf) { + mlx4_cq_resize_copy_cqes(cq); + mlx4_free_cq_buf(dev, &cq->buf, cq->ibcq.cqe); + cq->buf = cq->resize_buf->buf; + cq->ibcq.cqe = cq->resize_buf->cqe; + + kfree(cq->resize_buf); + cq->resize_buf = NULL; + } + } + spin_unlock_irq(&cq->lock); + + if (ibcq->uobject) + if (tumem) + ib_umem_release(tumem); + + goto free_context; + +err_buf: + if (cq->resize_buf) { + if (!ibcq->uobject) + mlx4_free_cq_buf(dev, &cq->resize_buf->buf, + cq->resize_buf->cqe); + + spin_lock_irq(&cq->lock); + kfree(cq->resize_buf); + cq->resize_buf = NULL; + spin_unlock_irq(&cq->lock); + } + if (cq->resize_umem) { + ib_umem_release(cq->resize_umem); + cq->resize_umem = NULL; + } + +free_context: + kfree(context); + +out: + mutex_unlock(&cq->resize_mutex); + return err; +} + int mlx4_ib_destroy_cq(struct ib_cq *cq) { struct mlx4_ib_dev *dev = to_mdev(cq->device); @@ -255,6 +488,44 @@ static void dump_cqe(void *cqe) be32_to_cpu(buf[6]), be32_to_cpu(buf[7])); } +int mlx4_alloc_cq_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq_buf *buf, int nent) +{ + int err; + + err = mlx4_buf_alloc(dev->dev, nent * sizeof(struct mlx4_cqe), + PAGE_SIZE * 2, + &buf->buf); + + if (err) + goto out; + + err = mlx4_mtt_init(dev->dev, buf->buf.npages, buf->buf.page_shift, + &buf->mtt); + if (err) + goto err_buf; + + err = mlx4_buf_write_mtt(dev->dev, &buf->mtt, &buf->buf); + if (err) + goto err_mtt; + + return 0; + +err_mtt: + mlx4_mtt_cleanup(dev->dev, &buf->mtt); + +err_buf: + mlx4_buf_free(dev->dev, nent * sizeof(struct mlx4_cqe), + &buf->buf); + +out: + return err; +} + +void mlx4_free_cq_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq_buf *buf, int cqe) +{ + mlx4_buf_free(dev->dev, (cqe + 1) * sizeof(struct mlx4_cqe), &buf->buf); +} + static void mlx4_ib_handle_error_cqe(struct mlx4_err_cqe *cqe, struct ib_wc *wc) { @@ -343,6 +614,7 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, u32 g_mlpath_rqpn; u16 wqe_ctr; +repoll: cqe = next_cqe_sw(cq); if (!cqe) return -EAGAIN; @@ -364,6 +636,21 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, printk(KERN_WARNING "Completion for NOP opcode detected!\n"); return -EINVAL; } + /* Resize CQ */ + if (unlikely((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_CQE_OPCODE_RESIZE)) { + if (cq->resize_buf) { + struct mlx4_ib_dev *dev = to_mdev(cq->ibcq.device); + + mlx4_free_cq_buf(dev, &cq->buf, cq->ibcq.cqe); + cq->buf = cq->resize_buf->buf; + cq->ibcq.cqe = cq->resize_buf->cqe; + + kfree(cq->resize_buf); + cq->resize_buf = NULL; + } + + goto repoll; + } if (!*cur_qp || (be32_to_cpu(cqe->my_qpn) & 0xffffff) != (*cur_qp)->mqp.qpn) { diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index f770610..6ee7f46 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -569,6 +569,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) (1ull << IB_USER_VERBS_CMD_DEREG_MR) | (1ull << IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL) | (1ull << IB_USER_VERBS_CMD_CREATE_CQ) | + (1ull << IB_USER_VERBS_CMD_RESIZE_CQ) | (1ull << IB_USER_VERBS_CMD_DESTROY_CQ) | (1ull << IB_USER_VERBS_CMD_CREATE_QP) | (1ull << IB_USER_VERBS_CMD_MODIFY_QP) | @@ -608,6 +609,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.post_recv = mlx4_ib_post_recv; ibdev->ib_dev.create_cq = mlx4_ib_create_cq; ibdev->ib_dev.modify_cq = mlx4_ib_modify_cq; + ibdev->ib_dev.resize_cq = mlx4_ib_resize_cq; ibdev->ib_dev.destroy_cq = mlx4_ib_destroy_cq; ibdev->ib_dev.poll_cq = mlx4_ib_poll_cq; ibdev->ib_dev.req_notify_cq = mlx4_ib_arm_cq; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 83eb071..425970f 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -78,13 +78,22 @@ struct mlx4_ib_cq_buf { struct mlx4_mtt mtt; }; +struct mlx4_ib_cq_resize { + struct mlx4_ib_cq_buf buf; + int cqe; +}; + struct mlx4_ib_cq { struct ib_cq ibcq; struct mlx4_cq mcq; struct mlx4_ib_cq_buf buf; + struct mlx4_ib_cq_resize *resize_buf; + int is_kernel; struct mlx4_ib_db db; spinlock_t lock; + struct mutex resize_mutex; struct ib_umem *umem; + struct ib_umem *resize_umem; }; struct mlx4_ib_mr { @@ -250,9 +259,13 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, int mlx4_ib_dereg_mr(struct ib_mr *mr); int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); +void mlx4_cq_resize_copy_cqes(struct mlx4_ib_cq *cq); +int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata); struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector, struct ib_ucontext *context, struct ib_udata *udata); +int mlx4_alloc_cq_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq_buf *buf, int nent); +void mlx4_free_cq_buf(struct mlx4_ib_dev *dev, struct mlx4_ib_cq_buf *buf, int cqe); int mlx4_ib_destroy_cq(struct ib_cq *cq); int mlx4_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); int mlx4_ib_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); -- 1.5.4.2 From vlad at dev.mellanox.co.il Mon Mar 31 01:51:18 2008 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Mon, 31 Mar 2008 11:51:18 +0300 Subject: [ofa-general] [PATCH v1] libmlx4: Added resize CQ capability. In-Reply-To: References: <47E92539.7030908@dev.mellanox.co.il> Message-ID: <47F0A606.2060500@dev.mellanox.co.il> From 012952a11c8e984a7ab70a5f85ad4ac845a25220 Mon Sep 17 00:00:00 2001 From: Vladimir Sokolovsky Date: Tue, 25 Mar 2008 17:09:08 +0200 Subject: [PATCH] Added resize CQ capability. Signed-off-by: Vladimir Sokolovsky --- src/cq.c | 38 ++++++++++++++++++++++++++++++++++++++ src/mlx4.h | 3 +++ src/verbs.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 94 insertions(+), 2 deletions(-) diff --git a/src/cq.c b/src/cq.c index 91297e4..e5436d4 100644 --- a/src/cq.c +++ b/src/cq.c @@ -201,6 +201,7 @@ static int mlx4_poll_one(struct mlx4_cq *cq, int is_error; int is_send; +repoll: cqe = next_cqe_sw(cq); if (!cqe) return CQ_EMPTY; @@ -215,6 +216,9 @@ static int mlx4_poll_one(struct mlx4_cq *cq, */ rmb(); + if ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_CQE_OPCODE_RESIZE) + goto repoll; + qpn = ntohl(cqe->my_qpn); is_send = cqe->owner_sr_opcode & MLX4_CQE_IS_SEND_MASK; @@ -433,6 +437,40 @@ void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq) pthread_spin_unlock(&cq->lock); } +int mlx4_get_outstanding_cqes(struct mlx4_cq *cq) +{ + int i; + + for (i = cq->cons_index; get_sw_cqe(cq, (i & cq->ibv_cq.cqe)); ++i) + ; + + return i - cq->cons_index; +} + void mlx4_cq_resize_copy_cqes(struct mlx4_cq *cq, void *buf, int old_cqe) { + struct mlx4_cqe *cqe; + int i; + + i = cq->cons_index; + cqe = get_cqe(cq, (i & old_cqe)); + + while ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) != MLX4_CQE_OPCODE_RESIZE) { + memcpy(buf + ((i + 1) & cq->ibv_cq.cqe) * MLX4_CQ_ENTRY_SIZE, + cqe, MLX4_CQ_ENTRY_SIZE); + ++i; + cqe = get_cqe(cq, (i & old_cqe)); + } + + ++cq->cons_index; +} + +int mlx4_alloc_cq_buf(struct mlx4_device *dev, struct mlx4_buf *buf, int nent) +{ + if (mlx4_alloc_buf(buf, align(nent * MLX4_CQ_ENTRY_SIZE, dev->page_size), + dev->page_size)) + return -1; + memset(buf->buf, 0, nent * MLX4_CQ_ENTRY_SIZE); + + return 0; } diff --git a/src/mlx4.h b/src/mlx4.h index 3710a17..7d88f74 100644 --- a/src/mlx4.h +++ b/src/mlx4.h @@ -174,6 +174,7 @@ struct mlx4_pd { struct mlx4_cq { struct ibv_cq ibv_cq; struct mlx4_buf buf; + struct mlx4_buf resize_buf; pthread_spinlock_t lock; uint32_t cqn; uint32_t cons_index; @@ -307,6 +308,7 @@ int mlx4_dereg_mr(struct ibv_mr *mr); struct ibv_cq *mlx4_create_cq(struct ibv_context *context, int cqe, struct ibv_comp_channel *channel, int comp_vector); +int mlx4_alloc_cq_buf(struct mlx4_device *dev, struct mlx4_buf *buf, int nent); int mlx4_resize_cq(struct ibv_cq *cq, int cqe); int mlx4_destroy_cq(struct ibv_cq *cq); int mlx4_poll_cq(struct ibv_cq *cq, int ne, struct ibv_wc *wc); @@ -314,6 +316,7 @@ int mlx4_arm_cq(struct ibv_cq *cq, int solicited); void mlx4_cq_event(struct ibv_cq *cq); void mlx4_cq_clean(struct mlx4_cq *cq, uint32_t qpn, struct mlx4_srq *srq); +int mlx4_get_outstanding_cqes(struct mlx4_cq *cq); void mlx4_cq_resize_copy_cqes(struct mlx4_cq *cq, void *buf, int new_cqe); struct ibv_srq *mlx4_create_srq(struct ibv_pd *pd, diff --git a/src/verbs.c b/src/verbs.c index 50e0947..6e0b479 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -226,8 +226,59 @@ err: int mlx4_resize_cq(struct ibv_cq *ibcq, int cqe) { - /* XXX resize CQ not implemented */ - return ENOSYS; + struct mlx4_cq *cq = to_mcq(ibcq); + struct mlx4_resize_cq cmd; + struct mlx4_buf buf; + int old_cqe, outst_cqe, ret; + + /* Sanity check CQ size before proceeding */ + if (cqe > 0x3fffff) + return EINVAL; + + pthread_spin_lock(&cq->lock); + + cqe = align_queue_size(cqe); + if (cqe == ibcq->cqe + 1) { + ret = 0; + goto out; + } + + /* Can't be smaller then the number of outstanding CQEs */ + outst_cqe = mlx4_get_outstanding_cqes(cq); + if (cqe < outst_cqe + 1) { + ret = 0; + goto out; + } + + ret = mlx4_alloc_cq_buf(to_mdev(ibcq->context->device), &buf, cqe); + if (ret) + goto out; + + old_cqe = ibcq->cqe; + cmd.buf_addr = (uintptr_t) buf.buf; + +#ifdef IBV_CMD_RESIZE_CQ_HAS_RESP_PARAMS + { + struct ibv_resize_cq_resp resp; + ret = ibv_cmd_resize_cq(ibcq, cqe - 1, &cmd.ibv_cmd, sizeof cmd, + &resp, sizeof resp); + } +#else + ret = ibv_cmd_resize_cq(ibcq, cqe - 1, &cmd.ibv_cmd, sizeof cmd); +#endif + if (ret) { + mlx4_free_buf(&buf); + goto out; + } + + mlx4_cq_resize_copy_cqes(cq, buf.buf, old_cqe); + mlx4_free_buf(&cq->buf); + + cq->buf = buf; + +out: + pthread_spin_unlock(&cq->lock); + return ret; } int mlx4_destroy_cq(struct ibv_cq *cq) -- 1.5.4.2 From sashak at voltaire.com Mon Mar 31 03:52:13 2008 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 31 Mar 2008 10:52:13 +0000 Subject: [ofa-general] Re: [PATCH] opensm/man: Adding QoS-related info to opensm man pages In-Reply-To: <47E99D0C.7040403@dev.mellanox.co.il> References: <47E99D0C.7040403@dev.mellanox.co.il> Message-ID: <20080331105213.GE30617@sashak.voltaire.com> Hi Yevgeny, On 02:47 Wed 26 Mar , Yevgeny Kliteynik wrote: > > I've added QoS related info to opensm man pages: enhanced > existing part (that was talking about VL arbitration) I see that this part was fully rewritten. And IMO it is less clear now than originally was (some comments are below). Any reason to not start enhancements from existing text? > and > added description of QoS manager in accordance with QoS annex. > > Please apply to ofed_1_3 and master. Comments are below. > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/man/opensm.8.in | 501 +++++++++++++++++++++++++++++++++++++++++++----- > 1 files changed, 457 insertions(+), 44 deletions(-) > > diff --git a/opensm/man/opensm.8.in b/opensm/man/opensm.8.in > index 5322ab7..1d9c5b7 100644 > --- a/opensm/man/opensm.8.in > +++ b/opensm/man/opensm.8.in > @@ -35,7 +35,8 @@ to initialize the InfiniBand hardware (at least one per each > InfiniBand subnet). > > opensm also now contains an experimental version of a performance > -manager as well. > +manager and an experimental version QoS manager (in accordance with > +IBA QoS Annex). > > opensm defaults were designed to meet the common case usage on clusters with up to a few hundred nodes. Thus, in this default mode, opensm will scan the IB > fabric, initialize it, and sweep occasionally for changes. > @@ -433,51 +434,463 @@ partition manager: > > Default=0x7fff,ipoib:ALL=full; > > -.SH QOS CONFIGURATION > +.SH QUALITY OF SERVICE > .PP > -There are a set of QoS related low-level configuration parameters. > -All these parameter names are prefixed by "qos_" string. Here is a full > -list of these parameters: > - > - qos_max_vls - The maximum number of VLs that will be on the subnet > - qos_high_limit - The limit of High Priority component of VL > - Arbitration table (IBA 7.6.9) > - qos_vlarb_low - Low priority VL Arbitration table (IBA 7.6.9) > - template > - qos_vlarb_high - High priority VL Arbitration table (IBA 7.6.9) > - template > - Both VL arbitration templates are pairs of > - VL and weight > - qos_sl2vl - SL2VL Mapping table (IBA 7.6.6) template. It is > - a list of VLs corresponding to SLs 0-15 (Note > - that VL15 used here means drop this SL) > - > -Typical default values (hard-coded in OpenSM initialization) are: > - > - qos_max_vls=15 > - qos_high_limit=0 > - qos_vlarb_low=0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4 > - qos_vlarb_high=0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0 > - qos_sl2vl=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 > - > -The syntax is compatible with rest of OpenSM configuration options and > -values may be stored in OpenSM config file (cached options file). > - > -In addition to the above, we may define separate QoS configuration > -parameters sets for various target types. As targets, we currently support > -CAs, routers, switch external ports, and switch's enhanced port 0. The > -names of such specialized parameters are prefixed by "qos__" > -string. Here is a full list of the currently supported sets: > - > - qos_ca_ - QoS configuration parameters set for CAs. > - qos_rtr_ - parameters set for routers. > - qos_sw0_ - parameters set for switches' port 0. > - qos_swe_ - parameters set for switches' external ports. > +OpenSM QoS support comprises of two parts: > > -Examples: > - qos_sw0_max_vls=2 > - qos_ca_sl2vl=0,1,2,3,5,5,5,12,12,0, > - qos_swe_high_limit=0 > + 1. \fBQoS manager in accordance with IBA QoS Annex\fP (experimental) > +.P > + 2. \fBSL2VL and VL Arbitration tables configuration\fP > +.P > +.SS QoS Manager (experimental) > +.PP > +When Quality of Service in OpenSM is enabled (-Q or --qos), OpenSM looks > +for QoS Policy file. The default name of this file is > +\fB\%@CONF_DIR@/@QOS_POLICY_FILE@\fP. The default may be changed by using > +-Y or --qos_policy_file option with OpenSM. > + > +During fabric initialization and at every heavy sweep OpenSM parses the > +QoS policy file, applies its settings to the discovered fabric elements, > +and enforces the provided policy on client requests. The overall flow for > +such requests is as follows: > + - The request is matched against the defined matching rules such that > + the QoS Level definition is found. > + - Given the QoS Level, path(s) search is performed with the given > + restrictions imposed by that level. > + > +There are two ways to define QoS policy: > + - \fBFull\fP: the full policy file syntax provides the administrator various > + ways to match a PathRecord/MultiPathRecord (PR/MPR) request, and to > + enforce various QoS constraints on the requested PR/MPR. > + - \fBSimplified\fP: the simplified policy file syntax enables the administrator > + match PR/MPR requests by various ULPs and applications running on top of > + these ULPs. > + > +While the full policy syntax is very flexible, in many cases the simplified > +policy definition would be sufficient. > +.PP > +.B Full QoS Policy File > +.PP > +QoS policy file has the following sections: > + > +.B I) > +Port Groups (denoted by port-groups). > +This section defines zero or more port groups that can be referred later by > +matching rules (see below). Port group lists ports by: > + - Port GUID > + - Port name, which is a combination of NodeDescription and IB port number > + - PKey, which means that all the ports in the subnet that belong to > + partition with a given PKey belong to this port group > + - Partition name, which means that all the ports in the subnet that belong > + to partition with a given name belong to this port group > + - Node type, where possible node types are: CA, SWITCH, ROUTER, ALL, and > + SELF (SM's port). > + > +.B II) > +QoS Setup (denoted by qos-setup). > +This section describes how to set up SL2VL and VL Arbitration tables on > +various nodes in the fabric. > +However, this is not supported in OFED 1.3. Here and below. I would prefer to not refer OFED versions (OpenSM can be used as part of OFED, independently or OFED/OpenSM versions can be mixed), something like "this version of OpenSM" looks more appropriate here for me. > +SL2VL and VLArb tables should be configured in the OpenSM options file. > + > +.B III) > +QoS Levels (denoted by qos-levels). > +Each QoS Level defines Service Level (SL) and a few optional fields: > + - MTU limit > + - Rate limit > + - PKey > + - Packet lifetime > + > +When path(s) search is performed, it is done with regards to restriction that > +these QoS Level parameters impose. > +One QoS level that is mandatory to define is a DEFAULT QoS level. It is > +applied to a PR/MPR query that does not match any existing match rule. > +Similar to any other QoS Level, it can also be explicitly referred by any > +match rule. Shouldn't this paragraph be placed after IV) - it refers matching rules which defined below? Or maybe even merged with IV? > + > +.B IV) > +QoS Matching Rules (denoted by qos-match-rules). > +Each PathRecord/MultiPathRecord query that OpenSM receives is matched against > +the set of matching rules. Rules are scanned in order of appearance in the QoS > +policy file such as the first match takes precedence. > +Each rule has a name of QoS level that will be applied to the matching query. > +A default QoS level is applied to a query that did not match any rule. > +Queries can be matched by: > + - Source port group (whether a source port is a member of a specified group) > + - Destination port group (same as above, only for destination port) > + - PKey > + - QoS class > + - Service ID > + > +To match a certain matching rule, PR/MPR query has to match ALL the rule's > +criteria. However, not all the fields of the PR/MPR query have to appear in > +the matching rule. > +For instance, if the rule has a single criterion - Service ID, it will match > +any query that has this Service ID, disregarding rest of the query fields. > +However, if a certain query has only Service ID (which means that this is the > +only bit in the PR/MPR component mask that is on), it will not match any rule > +that has other matching criteria besides Service ID. > +.PP > +.B Simplified QoS Policy Definition > +.PP > +Simplified QoS policy definition comprises of a single section denoted by > +qos-ulps. Similar to the full QoS policy, it has a list of match rules and > +their QoS Level, but in this case a match rule has only one criterion - its > +goal is to match a certain ULP (or a certain application on top of this ULP) > +PR/MPR request, and QoS Level has only one constraint - Service Level (SL). > +The simplified policy section may appear in the policy file in combine with > +the full policy, or as a stand-alone policy definition. > +See more details and list of match rule criteria below. What about to merge this paragraph with Simplified QoS policy description below? Here it looks like duplication. > +.PP > +.B Policy File Syntax Guidelines > +.PP > +Empty lines are ignored. > +Leading and trailing blanks, as well as empty lines, are ignored, so the > +indentation in the example is just for better readability. > +Comments are started with the pound sign (#) and terminated by EOL. > +Any keyword should be the first non-blank in the line, unless it's a comment. > +Keywords that denote section/subsection start have matching closing keywords. > +Having a QoS Level named "DEFAULT" is a must - it is applied to PR/MPR > +requests that didn't match any of the matching rules. > +Any section/subsection of the policy file is optional. And this paragraph to move above 'Full QoS Policy File' section? > + > +.PP > +.B Examples of Full Policy File > +.PP > +As mentioned earlier, any section of the policy file is optional, and > +the only mandatory part of the policy file is a default QoS Level. > +Here's an example of the shortest policy file: > + > + qos-levels > + qos-level > + name: DEFAULT > + sl: 0 > + end-qos-level > + end-qos-levels > + > +Port groups section is missing because there are no match rules, which means > +that port groups are not referred anywhere, and there is no need defining > +them. And since this policy file doesn't have any matching rules, PR/MPR query > +won't match any rule, and OpenSM will enforce default QoS level. > +Essentially, the above example is equivalent to not having QoS policy file > +at all. > + > +The following example shows all the possible options and keywords in the > +policy file and their syntax: > + > + # > + # See the comments in the following example. > + # They explain different keywords and their meaning. > + # > + port-groups > + port-group > + name: Storage > + # "use" is just a description that is used for logging > + # Other than that, it is just a comment > + use: SRP Targets > + port-guid: 0x10000000000001, 0x10000000000005-0x1000000000FFFA > + port-guid: 0x1000000000FFFF > + end-port-group > + > + port-group > + name: Virtual Servers > + # The syntax of the port name is as follows: > + # "node_description/Pnum". > + # node_description is compared to the NodeDescription of the node, > + # and "Pnum" is a port number on that node. > + port-name: vs1 HCA-1/P1, vs2 HCA-1/P1 > + end-port-group > + > + # using partitions defined in the partition policy > + port-group > + name: Partitions > + partition: Part1 > + pkey: 0x1234 > + end-port-group > + > + # using node types: CA, ROUTER, SWITCH, SELF (for node that runs SM) > + # or ALL (for all the nodes in the subnet) > + port-group > + name: CAs and SM > + node-type: CA, SELF > + end-port-group > + > + end-port-groups > + > + qos-setup > + # This section of the policy file describes how to set up SL2VL and VL > + # Arbitration tables on various nodes in the fabric. > + # However, this is not supported in OFED 1.3 - the section is parsed > + # and ignored. SL2VL and VLArb tables should be configured in the > + # OpenSM options file (by default - /var/cache/opensm/opensm.opts). > + end-qos-setup > + > + qos-levels > + > + # Having a QoS Level named "DEFAULT" is a must - it is applied to > + # PR/MPR requests that didn't match any of the matching rules. > + qos-level > + name: DEFAULT > + use: default QoS Level > + sl: 0 > + end-qos-level > + > + # the whole set: SL, MTU-Limit, Rate-Limit, PKey, Packet Lifetime > + qos-level > + name: WholeSet > + sl: 1 > + mtu-limit: 4 > + rate-limit: 5 > + pkey: 0x1234 > + packet-life: 8 > + end-qos-level > + > + end-qos-levels > + > + # Match rules are scanned in order of their apperance in the policy file. > + # First matched rule takes precedence. > + qos-match-rules > + > + # matching by single criteria: QoS class > + qos-match-rule > + use: by QoS class > + qos-class: 7-9,11 > + # Name of qos-level to apply to the matching PR/MPR > + qos-level-name: WholeSet > + end-qos-match-rule > + > + # show matching by destination group and service id > + qos-match-rule > + use: Storage targets > + destination: Storage > + service-id: 0x10000000000001, 0x10000000000008-0x10000000000FFF > + qos-level-name: WholeSet > + end-qos-match-rule > + > + qos-match-rule > + source: Storage > + use: match by source group only > + qos-level-name: DEFAULT > + end-qos-match-rule > + > + qos-match-rule > + use: match by all parameters > + qos-class: 7-9,11 > + source: Virtual Servers > + destination: Storage > + service-id: 0x0000000000010000-0x000000000001FFFF > + pkey: 0x0F00-0x0FFF > + qos-level-name: WholeSet > + end-qos-match-rule > + > + end-qos-match-rules > + > +.PP > +.B Simplified QoS Policy - Details and Examples > +.PP > +Simplified QoS policy match rules are tailored for matching ULPs (or > +some application on top of a ULP) PR/MPR requests. It has a list of > +per-ULP (or per-application) match rules and the SL that should be > +enforced on the matched PR/MPR query. > + > +Match rules include: > + - Default match rule that is applied to PR/MPR query that didn't > + match any of the other match rules > + - SDP > + - SDP application with a specific target TCP/IP port range > + - SRP with a specific target IB port GUID > + - RDS > + - iSER > + - iSER application with a specific target TCP/IP port range > + - IPoIB with a default PKey > + - IPoIB with a specific PKey > + - any ULP/application with a specific Service ID in the PR/MPR query > + - any ULP/application with a specific PKey in the PR/MPR query > + - any ULP/application with a specific target IB port GUID in the PR/MPR query Are there duplicated entries (SDP, iSER, IPoIB)? > + > +Since any section of the policy file is optional, as long as basic rules > +of the file are kept (such as no referring to nonexisting port group, > +having default QoS Level, etc), the simplified policy section (qos-ulps) > +can serve as a complete QoS policy file. > +The shortest policy file in this case would be as follows: > + > + qos-ulps > + default : 0 #default SL > + end-qos-ulps > + > +It is equivalent to not having policy file at all. > + > +Below is an example of simplified QoS policy with all the possible keywords: > + > + qos-ulps > + default : 0 # default SL > + sdp, port-num 30000 : 0 # SL for application running on top > + # of SDP when a destination > + # TCP/IPport is 30000 > + sdp, port-num 10000-20000 : 0 > + sdp : 1 # default SL for any other > + # application running on top of SDP > + rds : 2 # SL for RDS traffic > + iser, port-num 900 : 0 # SL for iSER with a specific target > + # port > + iser : 3 # default SL for iSER > + ipoib, pkey 0x0001 : 0 # SL for IPoIB on partition with > + # pkey 0x0001 > + ipoib : 4 # default IPoIB partition, > + # pkey=0x7FFF > + any, service-id 0x6234 : 6 # match any PR/MPR query with a > + # specific Service ID > + any, pkey 0x0ABC : 6 # match any PR/MPR query with a > + # specific PKey > + srp, target-port-guid 0x1234 : 5 # SRP when SRP Target is located on > + # a specified IB port GUID > + any, target-port-guid 0x0ABC-0xFFFFF : 6 # match any PR/MPR query with > + # a specific target port GUID > + end-qos-ulps > + > + > +Similar to the full policy definition, matching of PR/MPR queries is done in > +order of appearance in the QoS policy file such as the first match takes > +precedence, except for the "default" rule, which is applied only if the query > +didn't match any other rule. > + > +All other sections of the QoS policy file take precedence over the qos-ulps > +section. That is, if a policy file has both qos-match-rules and qos-ulps > +sections, then any query is matched first against the rules in the > +qos-match-rules section, and only if there was no match, the query is matched > +against the rules in qos-ulps section. > + > +Note that some of these match rules may overlap, so in order to use the > +simplified QoS definition effectively, it is important to understand how each > +of the ULPs is matched: > + > +.B IPoIB: > +PR query is matched by PKey. Default PKey for IPoIB partition is 0x7fff, so > +the following three match rules are equivalent: > + > + ipoib : > + ipoib, pkey 0x7fff : > + any, pkey 0x7fff : > + > +.I Note > +: For OFED 1.3, IPoIB partition SL configuration should be done through > +partition configuration file only. > + > +\fBSDP\fP: PR query is matched by Service ID. The Service-ID for SDP is > +0x000000000001PPPP, where PPPP are 4 hex digits holding the remote TCP/IP > +Port Number to connect to. The following two match rules are equivalent: > + > + sdp : > + any, service-id 0x0000000000010000-0x000000000001ffff : > + > +\fBRDS\fP: Similar to SDP, RDS PR query is matched by Service ID. The > +Service ID for RDS is 0x000000000106PPPP, where PPPP are 4 hex digits > +holding the remote TCP/IP Port Number to connect to. Default port number > +for RDS is 0x48CA, which makes a default Service-ID 0x00000000010648CA. > +The following two match rules are equivalent: > + > + rds : > + any, service-id 0x00000000010648CA : > + > +\fBiSER\fP: Similar to RDS, iSER query is matched by Service ID, where the > +Service ID is also 0x000000000106PPPP. Default port number for iSER is 0x035C, > +which makes a default Service-ID 0x000000000106035C. > +The following two match rules are equivalent: > + > + iser : > + any, service-id 0x000000000106035C : > + > +\fBSRP\fP: Service ID for SRP varies from storage vendor to vendor, thus SRP query is > +matched by the target IB port GUID. The following two match rules are > +equivalent: > + > + srp, target-port-guid 0x1234 : > + any, target-port-guid 0x1234 : > + > +Note that any of the above ULPs might contain target port GUID in the PR > +query, so in order for these queries not to be recognized by the QoS manager > +as SRP, the SRP match rule (or any match rule that refers to the target port > +guid only) should be placed at the end of the qos-ulps match rules. > + > +\fBMPI\fP: SL for MPI is manually configured by MPI admin. OpenSM is not > +forcing any SL on the MPI traffic, and that's why it is the only ULP that > +did not appear in the qos-ulps section. > + > + > +.SS SL2VL Mapping and VL Arbitration > +.PP > + > +OpenSM cached options file has a set of QoS related configuration > +parameters, that are used to configure SL2VL mapping and VL arbitration > +on IB ports. These parameters are: > + - Max VLs: the maximum number of VLs that will be on the subnet. > + - High limit: the limit of High Priority component of VL Arbitration > + table (IBA 7.6.9). > + - VLArb low table: Low priority VL Arbitration table (IBA 7.6.9) template. > + - VLArb high table: High priority VL Arbitration table (IBA 7.6.9) template. > + - SL2VL: SL2VL Mapping table (IBA 7.6.6) template. It is a list of VLs > + corresponding to SLs 0-15 (Note that VL15 used here means drop this SL). Why configuration keywords were removed here? OpenSM doesn't know what is "Max VLs", but knows "max_vls". > +There are separate QoS configuration parameters sets for various target > +types: This is optional... > CAs, routers, switch external ports, and switch's enhanced port 0. > +The names of such parameters are prefixed by "qos__" string. > +Here is a full list of the currently supported sets: > + > + qos_ca_ - QoS configuration parameters set for CAs. > + qos_rtr_ - parameters set for routers. > + qos_sw0_ - parameters set for switches' port 0. > + qos_swe_ - parameters set for switches' external ports. > + > +Here's the example of typical default values for all the ports in the > +subnet (hard-coded in OpenSM initialization): > + > + qos_max_vls=15 > + qos_high_limit=0 > + qos_vlarb_high=0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0 > + qos_vlarb_low=0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4 > + qos_sl2vl=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 > + > + > +VL arbitration tables (both high and low) are lists of VL/Weight pairs. > +Each list entry contains a VL number (values from 0-14), and a weighting value > +(values 0-255), indicating the number of 64 byte units (credits) which may be > +transmitted from that VL when its turn in the arbitration occurs. A weight > +of 0 indicates that this entry should be skipped. If a list entry is > +programmed for VL15 or for a VL that is not supported or is not currently > +configured by the port, the port may either skip that entry or send from any > +supported VL for that entry. > + > +Note, that the same VLs may be listed multiple times in the High or Low > +priority arbitration tables, and, further, it can be listed in both tables. Here and below. Do we need to rewrite the IBA spec in the man pages? Those parameters are not invited by OpenSM implementation, it is port parameters as defined in IBA spec. > + > +The limit of high-priority VLArb table (qos__high_limit) indicates the > +number of high-priority packets that can be transmitted without an opportunity > +to send a low-priority packet. Specifically, the number of bytes that can be > +sent is high_limit times 4K bytes. > + > +A high_limit value of 255 indicates that the byte limit is unbounded. > +Note: if the 255 value is used, the low priority VLs may be starved. > +A value of 0 indicates that only a single packet from the high-priority table > +may be sent before an opportunity is given to the low-priority table. > + > +Keep in mind that ports usually transmit packets of size equal to MTU. > +For instance, for 4KB MTU a single packet will require 64 credits, so in order > +to achieve effective VL arbitration for packets of 4KB MTU, the weighting > +values for each VL should be multiples of 64. > + > +Below is an example of SL2VL and VL Arbitration configuration on subnet: > + > + qos_max_vls=15 > + qos_high_limit=6 > + qos_vlarb_high=0:4 > + qos_vlarb_low=0:0,1:64,2:128,3:192,4:0,5:64,6:64,7:64 > + qos_sl2vl=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 > + > +In this example, there are 8 VLs configured on subnet: VL0 to VL7. VL0 is > +defined as a high priority VL, and it is limited to 6 x 4KB = 24KB in a single > +transmission burst. Such configuration would suilt VL that needs low latency > +and uses small MTU when transmitting packets. Rest of VLs are defined as low > +priority VLs with different weights, while VL4 is effectively turned off. > > .SH PREFIX ROUTES > .PP And finally due to a huge size of QoS description, wouldn't it be useful to move it below another (shorter) sections of the man page? Sasha From tziporet at mellanox.co.il Mon Mar 31 03:45:22 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Mon, 31 Mar 2008 13:45:22 +0300 Subject: [ofa-general] RE: [PATCH] ofed/docs: some fixes in OpenSM release notes In-Reply-To: <47E8C0DF.6010203@dev.mellanox.co.il> Message-ID: <6C2C79E72C305246B504CBA17B5500C903A7A1CD@mtlexch01.mtl.com> Done Tziporet -----Original Message----- From: Yevgeny Kliteynik [mailto:kliteyn at dev.mellanox.co.il] Sent: Tuesday, March 25, 2008 11:08 AM To: Tziporet Koren; Tziporet Koren Cc: Hal Rosenstock; Sasha Khapyorsky; OpenIB Subject: [PATCH] ofed/docs: some fixes in OpenSM release notes Hi Tziporet, Added ConnectX to list of supported devices, fixed FW versions to match OFED release notes, and removed device IDs (no reason for them to be there, OFED release notes doesn't have them). Please apply OFED 1.3 docs. Signed-off-by: Yevgeny Kliteynik --- opensm_release_notes.txt | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) diff --git a/opensm_release_notes.txt b/opensm_release_notes.txt index 825d758..bdefc11 100644 --- a/opensm_release_notes.txt +++ b/opensm_release_notes.txt @@ -459,14 +459,15 @@ Table 3 - Qualified Devices and Corresponding Firmware ====================================================== Mellanox -Device | FW versions ---------|----------------------------------------------------------- -MT43132 | InfiniScale - fw-43132 5.2.0 (and later) -MT47396 | InfiniScale III - fw-47396 0.5.0 (and later) -MT23108 | InfiniHost - fw-23108 3.3.2 (and later) -MT25204 | InfiniHost III Lx - fw-25204 1.0.1i (and later) -MT25208 | InfiniHost III Ex (InfiniHost Mode) - fw-25208 4.6.2 (and later) -MT25208 | InfiniHost III Ex (MemFree Mode) - fw-25218 5.0.1 (and later) +Device | FW versions +------------------------------------|------------------------------- +InfiniScale | fw-43132 5.2.000 (and later) +InfiniScale III | fw-47396 0.5.000 (and later) +InfiniHost | fw-23108 3.5.000 (and later) +InfiniHost III Lx | fw-25204 1.2.000 (and later) +InfiniHost III Ex (InfiniHost Mode) | fw-25208 4.8.200 (and later) +InfiniHost III Ex (MemFree Mode) | fw-25218 5.3.000 (and later) +ConnectX IB | fw-25408 2.3.000 (and later) QLogic/PathScale Device | Note -- 1.5.1.4 From tziporet at mellanox.co.il Mon Mar 31 03:47:18 2008 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Mon, 31 Mar 2008 13:47:18 +0300 Subject: [ofa-general] RE: Rename the QoS_in_OFED.txt file QoS_architecture.txt in OFED docs. In-Reply-To: <47E8C1EF.60700@dev.mellanox.co.il> Message-ID: <6C2C79E72C305246B504CBA17B5500C903A7A1D4@mtlexch01.mtl.com> Done Tziporet -----Original Message----- From: Yevgeny Kliteynik [mailto:kliteyn at dev.mellanox.co.il] Sent: Tuesday, March 25, 2008 11:12 AM To: Tziporet Koren; Tziporet Koren Cc: Or Gerlitz; OpenIB Subject: Rename the QoS_in_OFED.txt file QoS_architecture.txt in OFED docs. Tziporet, Please rename the QoS_in_OFED.txt file to QoS_architecture.txt in OFED docs. Thanks -- Yevgeny > Or Gerlitz wrote: >> Yevgeny Kliteynik wrote: >>> This doc describes OFED components (SDP, RDS, SRP, IPoIB, iSER), >>> their support of QoS annex, and a way for a QoS manager (OpenSM, >>> which is part of OFED) >> >> all these components are being worked out by bunch of developers and >> maintainers, ofed is just the framework for temporal distribution of >> them. >> >>> I really don't have any special sentiments for file name, but >>> is QoS_in_OFED.txt wrong? >>> You probably meant something like QoS_architecture.txt? >>> >> yes. >> From koen.segers at vrt.be Mon Mar 31 05:47:00 2008 From: koen.segers at vrt.be (Koen Segers) Date: Mon, 31 Mar 2008 14:47:00 +0200 Subject: [ofa-general] PCI-Express payload In-Reply-To: <47F0D384.9040706@dev.mellanox.co.il> References: <1206963596.8457.17.camel@koenVRT> <47F0D384.9040706@dev.mellanox.co.il> Message-ID: <1206967620.8457.29.camel@koenVRT> Hi Dotan, You are refering to the MTU of InfiniBand. This parameter defines how much data a single packet can contain when transferred over an Infiniband network. I want to know the MTU of PCI-Express. PCI-Express is used to transfer data from memory to HCA and vice versa. As far as I know, this MTU can differ. Our SAS HBA, has a PCI-Express MTU of 512 bytes. Regards, Koen On Mon, 2008-03-31 at 15:05 +0300, Dotan Barak wrote: > Hi. > > You can check what is the maximum supported MTU of your HCA using > ibv_devinfo. > Most of the HCAs supports 2KB MTU and some of them 4KB MTU. > > The MTU is the maximum payload size that can be sent/received > (the space for the IB headers are not part of the MTU) > > > Every connected QP (RC/UC) can define which MTU value he is using > (according to the > used path). > > > I hope that is the info you asked for .. > Dotan > > Koen Segers wrote: > > Hi, > > > > I'm interested in computing the overhead coming from PCI-Express to an > > IB HCA. Therefore I need to know the payload size of different types > > of HCA. > > > > We have InfiniHost III Mellanox cards of 4x SDR and DDR IB. According > > the lspci -vvv command on SLES 10 SP1, the PCI-e payload of these > > cards is maximum 128 bytes. > > > > Can someone give my the payload size for a ConnectX 4x SDR and DDR IB > > card? > > > > Thank you, > > > > Koen > > > > This is my output: > > 41:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > > (rev a0) > > Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > > ParErr+ Stepping- SERR- FastB2B- > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > > SERR- > Latency: 0, Cache Line Size: 64 bytes > > Interrupt: pin A routed to IRQ 209 > > Region 0: Memory at e1700000 (64-bit, non-prefetchable) [size=1M] > > Region 2: Memory at e1800000 (64-bit, prefetchable) [size=8M] > > Capabilities: [40] Power Management version 2 > > Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA > > PME(D0-,D1-,D2-,D3hot-,D3cold-) > > Status: D0 PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [48] Vital Product Data > > Capabilities: [90] Message Signalled Interrupts: Mask- 64bit+ > > Queue=0/5 Enable- > > Address: 0000000000000000 Data: 0000 > > Capabilities: [84] MSI-X: Enable+ Mask- TabSize=32 > > Vector table: BAR=0 offset=00082000 > > PBA: BAR=0 offset=00082200 > > Capabilities: [60] Express Endpoint IRQ 0 > > * Device: Supported: MaxPayload 128 bytes, PhantFunc 0, > > ExtTag+* > > Device: Latency L0s <64ns, L1 unlimited > > Device: AtnBtn- AtnInd- PwrInd- > > Device: Errors: Correctable- Non-Fatal+ Fatal+ > > Unsupported- > > Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- > > Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes > > Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s, Port 8 > > Link: Latency L0s unlimited, L1 unlimited > > Link: ASPM Disabled RCB 128 bytes CommClk- ExtSynch- > > Link: Speed 2.5Gb/s, Width x8 > > > > > > *** Disclaimer *** > > > > Vlaamse Radio- en Televisieomroep > > Auguste Reyerslaan 52, 1043 Brussel > > > > nv van publiek recht > > BTW BE 0244.142.664 > > RPR Brussel > > http://www.vrt.be/disclaimer > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > *** Disclaimer *** Vlaamse Radio- en Televisieomroep Auguste Reyerslaan 52, 1043 Brussel nv van publiek recht BTW BE 0244.142.664 RPR Brussel http://www.vrt.be/disclaimer From PhilpennsylvaniaZimmerman at usip.org Sun Mar 30 21:23:05 2008 From: PhilpennsylvaniaZimmerman at usip.org (Pat Roy) Date: Mon, 31 Mar 2008 09:23:05 +0500 Subject: [ofa-general] Investor Insight Message-ID: <3IX266EJXVWDA077@usip.org> An HTML attachment was scrubbed... URL: From GarlandcameronCochran at familytreedna.com Sun Mar 30 23:54:10 2008 From: GarlandcameronCochran at familytreedna.com (Morgan Cochran) Date: Mon, 31 Mar 2008 10:54:10 +0400 Subject: [ofa-general] Aggressive investors alert Message-ID: <8IX680EJXVWDA136@familytreedna.com> Introducing the new tech giant in the making DNC MultiMedia Corp Symbol: DCNM This is a hugely undervalued Gem with high profile clients such as Microsoft, Samsung, LG, and Napster the huge volume and price spike today on the news release is a good indicator of short and long term movement. See the company's outstanding pr and pull the trigger on DCNM From mkrause at hp.com Mon Mar 31 07:58:17 2008 From: mkrause at hp.com (Michael Krause) Date: Mon, 31 Mar 2008 07:58:17 -0700 Subject: [ofa-general] PCI-Express payload In-Reply-To: <1206967620.8457.29.camel@koenVRT> References: <1206963596.8457.17.camel@koenVRT> <47F0D384.9040706@dev.mellanox.co.il> <1206967620.8457.29.camel@koenVRT> Message-ID: <6.2.0.14.2.20080331074406.0236a148@esmail.cup.hp.com> An HTML attachment was scrubbed... URL: From eli at dev.mellanox.co.il Mon Mar 31 08:24:57 2008 From: eli at dev.mellanox.co.il (Eli Cohen) Date: Mon, 31 Mar 2008 18:24:57 +0300 Subject: [ofa-general] PCI-Express payload In-Reply-To: <1206963596.8457.17.camel@koenVRT> References: <1206963596.8457.17.camel@koenVRT> Message-ID: <1206977097.19448.76.camel@mtls03> This is what I get for ConnectX on my system: Device: MaxPayload 256 bytes, MaxReadReq 512 bytes But I think this also has to do with BIOS configuration and you usually should not change it. It depends on the chipset and other parameters of your system. On Mon, 2008-03-31 at 13:39 +0200, Koen Segers wrote: > Hi, > > I'm interested in computing the overhead coming from PCI-Express to an > IB HCA. Therefore I need to know the payload size of different types > of HCA. > > We have InfiniHost III Mellanox cards of 4x SDR and DDR IB. According > the lspci -vvv command on SLES 10 SP1, the PCI-e payload of these > cards is maximum 128 bytes. > > Can someone give my the payload size for a ConnectX 4x SDR and DDR IB > card? > > Thank you, > > Koen > > This is my output: > 41:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > (rev a0) > Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > ParErr+ Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > SERR- Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 209 > Region 0: Memory at e1700000 (64-bit, non-prefetchable) > [size=1M] > Region 2: Memory at e1800000 (64-bit, prefetchable) [size=8M] > Capabilities: [40] Power Management version 2 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA > PME(D0-,D1-,D2-,D3hot-,D3cold-) > Status: D0 PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [48] Vital Product Data > Capabilities: [90] Message Signalled Interrupts: Mask- 64bit+ > Queue=0/5 Enable- > Address: 0000000000000000 Data: 0000 > Capabilities: [84] MSI-X: Enable+ Mask- TabSize=32 > Vector table: BAR=0 offset=00082000 > PBA: BAR=0 offset=00082200 > Capabilities: [60] Express Endpoint IRQ 0 > Device: Supported: MaxPayload 128 bytes, PhantFunc 0, > ExtTag+ > Device: Latency L0s <64ns, L1 unlimited > Device: AtnBtn- AtnInd- PwrInd- > Device: Errors: Correctable- Non-Fatal+ Fatal+ > Unsupported- > Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- > Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes > Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s, > Port 8 > Link: Latency L0s unlimited, L1 unlimited > Link: ASPM Disabled RCB 128 bytes CommClk- ExtSynch- > Link: Speed 2.5Gb/s, Width x8 > > > *** Disclaimer *** > > Vlaamse Radio- en Televisieomroep > Auguste Reyerslaan 52, 1043 Brussel > > nv van publiek recht > BTW BE 0244.142.664 > RPR Brussel > http://www.vrt.be/disclaimer > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From tziporet at dev.mellanox.co.il Mon Mar 31 07:33:04 2008 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Mon, 31 Mar 2008 17:33:04 +0300 Subject: [ofa-general] [general-bounces@lists.openfabrics.org: Your message to general awaits moderator approval] In-Reply-To: <47F084E2.1000800@dev.mellanox.co.il> References: <20080329122249.GA13708@sashak.voltaire.com> <47F084E2.1000800@dev.mellanox.co.il> Message-ID: <47F0F620.8050404@mellanox.co.il> Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: >> Hi, >> >> Now I get such messages on each post to . > > Same here. > Any news on this? > > -- Yevgeny > I have this problem too Can someone check it? Maybe its related to mails coming from non-US mail servers Tziporet From dotanb at dev.mellanox.co.il Mon Mar 31 07:50:02 2008 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Mon, 31 Mar 2008 17:50:02 +0300 Subject: [ofa-general] [PATCH] core: check optional verbs before using them Message-ID: <200803311750.02916.dotanb@dev.mellanox.co.il> Check that all optional verbs are implemented in the device before using them. Signed-off-by: Dotan Barak --- diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 86ed8af..d0324d5 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -248,7 +248,9 @@ int ib_modify_srq(struct ib_srq *srq, struct ib_srq_attr *srq_attr, enum ib_srq_attr_mask srq_attr_mask) { - return srq->device->modify_srq(srq, srq_attr, srq_attr_mask, NULL); + return srq->device->modify_srq ? + srq->device->modify_srq(srq, srq_attr, srq_attr_mask, NULL) : + -ENOSYS; } EXPORT_SYMBOL(ib_modify_srq); @@ -265,6 +267,9 @@ int ib_destroy_srq(struct ib_srq *srq) struct ib_pd *pd; int ret; + if (!srq->device->destroy_srq) + return -ENOSYS; + if (atomic_read(&srq->usecnt)) return -EBUSY; @@ -672,6 +677,9 @@ struct ib_mr *ib_reg_phys_mr(struct ib_pd *pd, { struct ib_mr *mr; + if (!pd->device->reg_phys_mr) + return ERR_PTR(-ENOSYS); + mr = pd->device->reg_phys_mr(pd, phys_buf_array, num_phys_buf, mr_access_flags, iova_start); @@ -769,6 +777,9 @@ int ib_dealloc_mw(struct ib_mw *mw) struct ib_pd *pd; int ret; + if (!mw->device->dealloc_mw) + return -ENOSYS; + pd = mw->pd; ret = mw->device->dealloc_mw(mw); if (!ret) @@ -817,6 +828,9 @@ int ib_dealloc_fmr(struct ib_fmr *fmr) struct ib_pd *pd; int ret; + if (!fmr->device->dealloc_fmr) + return -ENOSYS; + pd = fmr->pd; ret = fmr->device->dealloc_fmr(fmr); if (!ret) From swise at opengridcomputing.com Mon Mar 31 08:52:50 2008 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 31 Mar 2008 10:52:50 -0500 Subject: [ofa-general] Re: [ewg] OFED March 24 meeting summary on OFED 1.4 plans In-Reply-To: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> Message-ID: <47F108D2.1030606@opengridcomputing.com> An HTML attachment was scrubbed... URL: From Wwalvara2 at mail.usf.edu Mon Mar 31 07:32:59 2008 From: Wwalvara2 at mail.usf.edu (brand geoffrey) Date: Mon, 31 Mar 2008 14:32:59 +0000 Subject: [ofa-general] rolex glow Message-ID: <000701c8934b$02e79abf$8e1e81b9@viyecap> Detailed quality replicas of the most wanted designer watches are here! Genuine solid stainless steel, and 99.9% accurate markings, finish and weight. - The worlds largest online retailer of luxury products, including: Rolex Sports Models Rolex Datejusts Breitling Cartier Porsche Design Dolce & Gabbana Dior Gucci Hermes Watches Patek Philippe Visit - www.ceassus.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From blue_apple at gmx.de Mon Mar 31 09:21:08 2008 From: blue_apple at gmx.de (Gregg Snyder) Date: Tue, 1 Apr 2008 01:21:08 +0900 Subject: [ofa-general] {Viagra_onli2_de} Message-ID: <966906173.61512680818145@gmx.de> Sie leben nur einmal - warum dann nicht was neues ausprobieren? Pr. .. Eise die keine Konk... ..Urrenz kennen - keine versteckte Kos// Ten - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - Kos... Tenlose, arztliche Telefon-Beratung - Bequem und dis kret 0... . N-line! be... .Stellen. - Disk rete Verpackung und Zahlung - Kein peinlicher Arz t besuch erforderlich Originalme/ dikamente Ciia..aa^_^aaalis...... 10 Pack. 21,00 Euro Viia..aa^_^aaagra... 10 Pack. 11,00 Euro Nur fur kurze Zeit - vier Pil. .. len umsonst erhalten Nie mehr zu frueh kommen? (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From BrunorunicSellers at bargaineering.com Mon Mar 31 13:25:10 2008 From: BrunorunicSellers at bargaineering.com (Emery House) Date: Mon, 31 Mar 2008 18:25:10 -0200 Subject: [ofa-general] Ride this winner for easy double or triple bagger Message-ID: <3IX426EJXVWDA159@bargaineering.com> Introducing the new tech giant in the making DNC MultiMedia Corp Symbol: DCNM This is a hugely undervalued Gem with high profile clients such as Microsoft, Samsung, LG, and Napster the huge volume and price spike today on the news release is a good indicator of short and long term movement. See the company's outstanding pr and pull the trigger on DCNM From sean.hefty at intel.com Mon Mar 31 09:20:35 2008 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 31 Mar 2008 09:20:35 -0700 Subject: [ofa-general] RE: the port numbers in some of the rdmacm examples is a fixed value In-Reply-To: <47EF2A80.1020804@dev.mellanox.co.il> References: <47EBBC81.4030501@dev.mellanox.co.il> <000101c89022$ce0b3d30$9c98070a@amr.corp.intel.com> <47EF2A80.1020804@dev.mellanox.co.il> Message-ID: <000101c8934b$265a46e0$37fc070a@amr.corp.intel.com> >I started to work on this patch and for ucmatose everything is fine. >The problem is with udaddy: the parameter "-p" is only being used for >the port space ... >(I really would like to have the same parameter for controlling the port >number for ALL of the examples >of the librdmacm, but i must admit that without doing some changes it >won't happen) ... I'd prefer to use the same parameter as well. If no one objects, I'm okay with modifying the udaddy -p parameter. >what do you think? >(do you want me to send you the changes i made so far?) Sending me the changes that you have would be fine. I can finish them up when I get some time. - Sean From tfelton at iqpc.co.uk Mon Mar 31 09:24:21 2008 From: tfelton at iqpc.co.uk (Jolene Lawrence) Date: Mon, 31 Mar 2008 18:24:21 +0200 Subject: [ofa-general] Smart in bed games Message-ID: <01c8935c$70467880$480b6e55@tfelton> Smart in bed games Please look attached file and know MORE ABOUT THIS! -------------- next part -------------- A non-text attachment was scrubbed... Name: secrets.zip Type: application/zip Size: 634 bytes Desc: not available URL: From michelle at holidays.net Mon Mar 31 09:32:56 2008 From: michelle at holidays.net (michelle at holidays.net) Date: Mon, 31 Mar 2008 19:32:56 +0300 Subject: [ofa-general] This can make you a legend in her eyes! Message-ID: <47F11238.4000807@holidays.net> You wont find high-rank medsat any other place! http://meikyzelpiend.blogspot.com From IsidrobehalfDodson at visitwesthollywood.com Sun Mar 30 23:38:02 2008 From: IsidrobehalfDodson at visitwesthollywood.com (Linwood Estes) Date: Mon, 31 Mar 2008 11:38:02 +0500 Subject: [ofa-general] Volume spike today Message-ID: <3IX547EJXVWDA088@visitwesthollywood.com> An HTML attachment was scrubbed... URL: From mkrause at hp.com Mon Mar 31 09:39:10 2008 From: mkrause at hp.com (Michael Krause) Date: Mon, 31 Mar 2008 09:39:10 -0700 Subject: [ofa-general] PCI-Express payload In-Reply-To: <1206977097.19448.76.camel@mtls03> References: <1206963596.8457.17.camel@koenVRT> <1206977097.19448.76.camel@mtls03> Message-ID: <6.2.0.14.2.20080331093131.0237e540@esmail.cup.hp.com> An HTML attachment was scrubbed... URL: From Jeffrey.C.Becker at nasa.gov Mon Mar 31 10:20:20 2008 From: Jeffrey.C.Becker at nasa.gov (Jeff Becker) Date: Mon, 31 Mar 2008 10:20:20 -0700 Subject: [ofa-general] Spam on mailing list general@openib.org In-Reply-To: References: <47EBB5A0.6030000@isomerica.net> <20080327152720.GB24509@cefeid.wcss.wroc.pl> <47EBBE4B.5090706@isomerica.net> <47EBCF04.6040208@nasa.gov> Message-ID: <47F11D54.601@nasa.gov> Hi. Since valid stuff is being rejected, I reset the SPAM filter for general to its previous setting. As Tom noted, most of the spam comes through the old openib.org lists. Is there any reason to keep these? If not, I can see about turning them off in order to improve the situation. Thanks. -jeff Talpey, Thomas wrote: > At 12:44 PM 3/27/2008, Jeff Becker wrote: > >> Hi all. I tightened the SPAM level trigger on this list. Hopefully >> that'll improve things. >> > > I'm sure you've noticed this didn't work very well. They got raunchier, > in fact. > > I'll note again that they are entering via the old openib.org lists, not the > current openfabrics. Perhaps the filter doesn't apply to the internal > redirection? > > Tom. > > From 02cooley at beeson.co.uk Mon Mar 31 11:41:20 2008 From: 02cooley at beeson.co.uk (jayson teri) Date: Mon, 31 Mar 2008 18:41:20 +0000 Subject: [ofa-general] this is job offer from LUGANO Message-ID: <000501c8936d$07b94975$ee34ecb9@diigcm> Opportunity From LUGANO Best Hotels Hello! Are you looking for an interesting job? Do you want to build perfect career with world leading team? Do you have a few available hours a day? Do you want to try part-time position from world leading tour operator? Bingo! We are glad to say that we are hiring. At this time we have various job openings for US citizens only. You do not need to have any tourism education and experiences, you do not need to relocate, you do not need to invest anything and we will cover all possible charges and fees. All you have to do at this time is just to read requirements and send us your CV or RESUME or just a few words about yourself for qualification. Our HR department will inform you with results very soon. Requirements: Strong computer skills (MS Office(MS Word 97/2003/2007) Any POP3 Mail Client) Good communication skills (Verbal and Written) Mobile phone (Just for Urgent Calls) High School, College, University (Any Education) Must Love People Send your resume/cv/profile to this address only: patrisia.balcker at gmail.com Best Places Best Guides SALARY: $3,500.00/month + % JOB TYPE: Part-Time/Employee PLACE OF WORK: Virtual Office ÷¿ Copyright 2004-2008 | Lugano Turismo -------------- next part -------------- An HTML attachment was scrubbed... URL: From EulaameliaSeals at criminallawyergroup.com Mon Mar 31 18:31:23 2008 From: EulaameliaSeals at criminallawyergroup.com (Leigh Castle) Date: Tue, 1 Apr 2008 00:31:23 -0100 Subject: [ofa-general] SuperNova Stocks Message-ID: <0IX368EJXVWDA756@criminallawyergroup.com> An HTML attachment was scrubbed... URL: From ChristaferryJoiner at almaz.com Mon Mar 31 06:43:48 2008 From: ChristaferryJoiner at almaz.com (Lorna Driscoll) Date: Mon, 31 Mar 2008 18:43:48 +0500 Subject: [ofa-general] Hot Stock Talk Message-ID: <5IX173EJXVWDA467@almaz.com> An HTML attachment was scrubbed... URL: From revadap at hellerdraper.com Mon Mar 31 17:00:18 2008 From: revadap at hellerdraper.com (revadap at hellerdraper.com) Date: Mon, 31 Mar 2008 19:00:18 -0500 Subject: [ofa-general] Doh! April's Fool. Message-ID: <002701c8938b$5f1ecb50$4abddabe@fhyc> Happy All Fool's Day. http://24.128.211.65 From LuisacantoneseWolff at prisonplanet.tv Mon Mar 31 08:52:59 2008 From: LuisacantoneseWolff at prisonplanet.tv (Jeri Kraft) Date: Mon, 31 Mar 2008 20:52:59 +0500 Subject: [ofa-general] Wall Street Alert Message-ID: <5IX676EJXVWDA135@prisonplanet.tv> DnC Multimedia Corporation Today came firing out of the gate today Symbol:DCNM 600% Volume Spike and Over 20% gains on a a huge news release The hot PR DnC Multimedia Announces Distribution Agreement and $445,000 Purchase Order, read more about it. The trick with penny stocs is to hit it while its hot, and today's activity clearly backs our beliefs of DCNM being in the zone. Investors are discovering this hidden gem Grab this gem while its in cents it wont last there long. Ride the gains with DCNM DnC Multimedia Corporation Today From LethasecrecyEngle at reiki.nu Mon Mar 31 09:01:14 2008 From: LethasecrecyEngle at reiki.nu (Brianna Darby) Date: Mon, 31 Mar 2008 21:01:14 +0500 Subject: [ofa-general] Investor Insight Message-ID: <9IX720EJXVWDA559@reiki.nu> An HTML attachment was scrubbed... URL: From woessner at seismo.ifg.ethz.ch Mon Mar 31 18:19:33 2008 From: woessner at seismo.ifg.ethz.ch (Lilly Pack) Date: Tue, 1 Apr 2008 10:19:33 +0900 Subject: [ofa-general] Ficken wie ein Weltmeister ? Message-ID: <01c893e1$e0e3a880$f61e5edc@woessner> Versuchen Sie unser Produkt und Sie werden fuhlen was unsere Kunden bestatigen Pr. .. Eise die keine Konk... ..Urrenz kennen - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - Kos... Tenlose, arztliche Telefon-Beratung - Kein peinlicher Arz t besuch erforderlich - Bequem und dis kret 0... . N-line! be... .Stellen. - keine versteckte Kos// Ten - Disk rete Verpackung und Zahlung Originalme/ dikamente Ciia..aa^_^aaalis...... 10 Pack. 21,00 Euro Viia..aa^_^aaagra... 10 Pack. 11,00 Euro Vier Dosen gibt's bei jeder Bestellung umsonst (bitte warten Sie einen Moment bis die Seite vollstandig geladen ist) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ogerlitz at voltaire.com Mon Mar 31 23:23:34 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 01 Apr 2008 09:23:34 +0300 Subject: [ofa-general] Re: summary on OFED 1.4 plans - re adding new kernel features/verbs through ofed In-Reply-To: <47F108D2.1030606@opengridcomputing.com> References: <6C2C79E72C305246B504CBA17B5500C90282E5BB@mtlexch01.mtl.com> <47F108D2.1030606@opengridcomputing.com> Message-ID: <47F1D4E6.2010108@voltaire.com> Steve Wise wrote: > Tziporet Koren wrote: >> >> * OFED 1.4: * >> 1. Kernel base: since we target 1.4 release to Sep we target the >> kernel base to be 2.6.27 >> This is a good target, but we may need to stay with 2.6.26 if the >> kernel progress will not be aligned. >> 2. Suggestions for new features: >> >> * Verbs: Reliable Multicast (to be presented at Sonoma) >> * IPoIB - continue with performance enhancements >> > Sorry I missed these meetings. For iWARP, here is my plan: > New iWARP Verbs: > - stag_alloc/dealloc > - nsmr_fastreg > - read-with-inv-local-stag > - inv-local-stag > Note the above verbs might be transport-independent. I believe the > IBTA has defined a fastreg verb too? > - peer-2-peer support in IWCM/Drivers Steve, Tziporet, So you are talking about adding new verbs/features to the Linux RDMA stack. Are you intending to do this through the mainline kernel cycles, eg the general list, the maintainer (Roland), etc? and if not, why? Same for for the ipoib performance enhancements, with all the troubles we experienced with the ipoib features merged to ofed 1.3 between rc3 and rc4 (selective TX signaling, 4K MTU), I think such patch must be first reviewed in the general list. For example, the merge of the selective TX signaling patch was suspended by Eli since he doesn't see any performance improvement on top of the CQ separation etc patch, so if it would have been reviewed for mainline in the first place and the question "how much does this improves" would have been asked, etc we could avoid much of the troubles that were associated with this patch. Or. Or. Or. From ogerlitz at voltaire.com Mon Mar 31 23:34:39 2008 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 01 Apr 2008 09:34:39 +0300 Subject: [ofa-general] Spam on mailing list general@openib.org In-Reply-To: <47F11D54.601@nasa.gov> References: <47EBB5A0.6030000@isomerica.net> <20080327152720.GB24509@cefeid.wcss.wroc.pl> <47EBBE4B.5090706@isomerica.net> <47EBCF04.6040208@nasa.gov> <47F11D54.601@nasa.gov> Message-ID: <47F1D77F.7030104@voltaire.com> Jeff Becker wrote: > Hi. Since valid stuff is being rejected, I reset the SPAM filter for > general to its previous setting. As Tom noted, most of the spam comes > through the old openib.org lists. Is there any reason to keep these? > If not, I can see about turning them off in order to improve the > situation. Thanks. I vote for turning off the old openib.org lists, best if you can set some automatic reply on them redirecting the sender to the openfabrics.org lists, etc. Or. From or.gerlitz at gmail.com Mon Mar 31 23:53:30 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 1 Apr 2008 09:53:30 +0300 Subject: [ofa-general] [PATCH 6/10 v1] IB/mlx4: Add LSO support In-Reply-To: <1206452112.25950.360.camel@mtls03> References: <1206452112.25950.360.camel@mtls03> Message-ID: <15ddcffd0803312353l57b0cfaft273c9f809387fb68@mail.gmail.com> On Tue, Mar 25, 2008 at 4:35 PM, Eli Cohen wrote: > Add LSO support to mlx4 driver such that it will be able > to send SKBs passed from the driver which publish NETIF_TSO. > > Signed-off-by: Eli Cohen > --- > Changes since last post: > 1. Verify that header length does not exceed 60 bytes. > 2. Remove unnecessary printk calls > > OK, so this patch would complete the LSO merging! Eli - what is the status here, does some editing is needed to comply to the way the rest of the patches were merged (qp creation flags, etc), or its applicable to review/merge as was posted? Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From or.gerlitz at gmail.com Mon Mar 31 23:53:30 2008 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Tue, 1 Apr 2008 09:53:30 +0300 Subject: [ofa-general] [PATCH 6/10 v1] IB/mlx4: Add LSO support In-Reply-To: <1206452112.25950.360.camel@mtls03> References: <1206452112.25950.360.camel@mtls03> Message-ID: <15ddcffd0803312353l57b0cfaft273c9f809387fb68@mail.gmail.com> On Tue, Mar 25, 2008 at 4:35 PM, Eli Cohen wrote: > Add LSO support to mlx4 driver such that it will be able > to send SKBs passed from the driver which publish NETIF_TSO. > > Signed-off-by: Eli Cohen > --- > Changes since last post: > 1. Verify that header length does not exceed 60 bytes. > 2. Remove unnecessary printk calls > > OK, so this patch would complete the LSO merging! Eli - what is the status here, does some editing is needed to comply to the way the rest of the patches were merged (qp creation flags, etc), or its applicable to review/merge as was posted? Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From joalvo at solar.com.br Mon Mar 31 23:10:22 2008 From: joalvo at solar.com.br (lonnie kakogawa) Date: Tue, 01 Apr 2008 06:10:22 +0000 Subject: [ofa-general] Your neighbour naked!! watch Message-ID: <000501c893ce$07936bd7$ad1f9b83@kqsfbc> QBYDRtqqry Download and WatchyjKNQQBYDRt -------------- next part -------------- An HTML attachment was scrubbed... URL: