[openib-general] [PATCH 0/7 v2] for 2.6.20 rdma/cma: add userspace support

[Jack Morgenstein]

On Wednesday 25 October 2006 00:25, Sean Hefty wrote:
> The following set of patches expand the rdma_cm support to include
> UD and multicast, and expose the rdma_cm to userspace.  I would like to
> target the 2.6.20 kernel, but at least getting them into one or more
> branches would be helpful for other developers to test against these
> changes.
> 

I have incorporated your rdma patches for 2.6.20 (1-7 v2) into
our driver, and am experiencing problems with multicast.c.

When I unload the Infiniband driver I am getting a 
kernel Oops (consistently at the same location, with the same
stack output).

I am doing the driver unload immediately after reboot
(the boot process loads the infiniband driver).
I am not running opensm.

Looks to me like a reference counting problem.

Below is the relevant data.

Jack
================================================
Console output:
# /etc/init.d/openibd stop
Shutting down interface ib0:                               [  OK  ]
Shutting down interface ib1:                               [  OK  ]

Message from syslogd at Mon Nov  6 12:00:06 2006 ...
kernel: BUG: spinlock bad magic on CPU#1, ib_mad2/1570

Message from syslogd at Mon Nov  6 12:00:06 2006 ...
kernel: general protection fault: 0000 [1] SMP

================================================
lsmod output (infiniband modules only):

ib_mthca              123972  0
ib_umad                18736  0
ib_sa                  25920  0
ib_mad                 39864  3 ib_mthca,ib_umad,ib_sa
ib_core                56448  4 ib_mthca,ib_umad,ib_sa,ib_mad

=================================
ps -ef shows that the following command has hung:
/sbin/modprobe -r ib_ipoib

===========================================
/var/log/messages:

BUG: spinlock bad magic on CPU#1, ib_mad2/1570
general protection fault: 0000 [1] SMP
CPU 1
Modules linked in: nfsd exportfs ipv6 parport_pc lp parport autofs4 nfs lockd nfs_acl sunrpc vfat fat dm_mirr
or dm_mod button battery ac ohci_hcd ehci_hcd i2c_nforce2 i2c_core ib_mthca ib_umad ib_sa ib_mad ib_core tg3 ext3 jbd sata_nv libata mptsas
 scsi_transport_sas sd_mod
Pid: 1570, comm: ib_mad2 Not tainted 2.6.17.7 #3
RIP: 0010:[<ffffffff802ddc40>] <ffffffff802ddc40>{spin_bug+116}
RSP: 0018:ffff81013bb95ca8  EFLAGS: 00010002
RAX: 000000006b6b6b6b RBX: 6b6b6b6b6b6b6b6b RCX: ffffffff8044c057
RDX: ffffffff804a7f18 RSI: 0000000000000046 RDI: ffffffff804a7f00
RBP: ffff81013ba36668 R08: 00000000ffffffff R09: 0000000000000003
R10: 0000000100000000 R11: 0000000000000000 R12: ffff81013ba36668
R13: 0000000000000283 R14: 0000000000000000 R15: ffffffff8808f2cf
FS:  00002b86142cdb00(0000) GS:ffff81013fc616d0(0000) knlGS:00000000f7f038e0
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000003b5a131eb0 CR3: 000000013aa5d000 CR4: 00000000000006e0
Process ib_mad2 (pid: 1570, threadinfo ffff81013bb94000, task ffff81013f2768b0)
Stack: 0000000000000003 ffff81013ba36668 ffff81013ba36660 ffffffff802ddc8d
       ffff81013c88e8b8 ffff81013ba36660 ffff81013ba36668 ffffffff80428b2b
       0000000000000246 ffffffff88097efe
Call Trace: <ffffffff802ddc8d>{_raw_spin_lock+28} <ffffffff80428b2b>{_spin_lock_irqsave+11}
       <ffffffff88097efe>{:ib_sa:release_group+26} <ffffffff88098904>{:ib_sa:mcast_work_handler+1280}
       <ffffffff80428bd7>{_spin_unlock_irq+7} <ffffffff8808f2cf>{:ib_mad:timeout_sends+0}
       <ffffffff880978c3>{:ib_sa:ib_sa_mcmember_rec_callback+64}
       <ffffffff80428bd7>{_spin_unlock_irq+7} <ffffffff88097ac4>{:ib_sa:send_handler+74}
       <ffffffff8808f45c>{:ib_mad:timeout_sends+397} <ffffffff80238450>{run_workqueue+161}
       <ffffffff8023849a>{worker_thread+0} <ffffffff8023b444>{keventd_create_kthread+0}
       <ffffffff8023859f>{worker_thread+261} <ffffffff80223ddd>{default_wake_function+0}
       <ffffffff8023b444>{keventd_create_kthread+0} <ffffffff80223ddd>{default_wake_function+0}
       <ffffffff8023b444>{keventd_create_kthread+0} <ffffffff8023b41b>{kthread+200}
       <ffffffff8020a6a6>{child_rip+8} <ffffffff8023b444>{keventd_create_kthread+0}

Code: 44 8b 83 04 01 00 00 48 8d 8b a0 02 00 00 8b 55 04 41 89 c1