[openib-general] nfsrdma server stop responding,

Mon Dec 11 17:32:10 PST 2006

Hit *send* too soon - here is the objdump of swap.o

-vu


> James Lentini wrote:
>> A couple of questions Vu:
>>
>> What NFS-RDMA release are you using? This looks like release 7.
>>
> 
> Yes. I'm using release 7
> 
>> Is this reproducible?
> 
> I ran into it twice - I think that it may co-relate to 
> openSM restart incident. I'll double check it and confirm
> 
> 
>> What kernel version are you using?
> 
> 2.6.18.5
> 
>> What hardware is this on? It looks like x86-64 to me, which is fine. I 
>> just want to be sure I know what I'm looking at. As many specifics as 
>> possible is good (number of CPUs, hyperthreading, etc.)
>>
> 
> Dual woodcrest xeon based CPUs
> 
>> Could you send the output of 
>>
>> objdump -Slr /path/to/kernel/mm/swap.o
>>
> 
> I attached the objdump output here
> 
>> Actually, just the put_page disassembly is all I want to see.
>>
>> Is there any more text available? Usually there is an explanation 
>> given for an oops message (e.g. "Unable to handle kernel paging 
>> request..").
>>
> 
> I did not see any oops text message. System was still 
> responsive with ipoib ping or login
> 
> 
>> I opened a bug at the NFS-RDMA SourceForge project to track this:
>>
>> http://sourceforge.net/tracker/index.php?func=detail&aid=1613201&group_id=97628&atid=618583
> 
> thanks for your help,
> 
> -vu
> 
>> Thanks for reporting this.
>> james
>>
>> On Fri, 8 Dec 2006, Vu Pham wrote:
>>
>>> Hi James,
>>>   I got these errors in server's /var/log/messages and then the server stop
>>> responding to login, I/O...; however, the server is still up, ipoib is still
>>> working
>>>
>>>
>>> Dec  8 06:38:21 ibd201 kernel: RIP: 0010:[<ffffffff8025dff7>]
>>> [<ffffffff8025dff7>] put_page+0x17/0x40
>>> Dec  8 06:38:21 ibd201 kernel: RSP: 0018:ffff810219ddfb08  EFLAGS: 00010246
>>> Dec  8 06:38:21 ibd201 kernel: RAX: 0000000000000000 RBX: 0000000000000001
>>> RCX: 000000000003ffff
>>> Dec  8 06:38:21 ibd201 kernel: RDX: 0000000000000000 RSI: 0000000000000001
>>> RDI: ffff8102274e92f8
>>> Dec  8 06:38:21 ibd201 kernel: RBP: ffff8101ab785000 R08: 0000000000000034
>>> R09: 0000000000000000
>>> Dec  8 06:38:21 ibd201 kernel: R10: 0000000000000000 R11: 0000000000000000
>>> R12: ffff81020ef96800
>>> Dec  8 06:38:21 ibd201 kernel: R13: ffff8101ab785000 R14: 0000000000000000
>>> R15: ffff8102053ee890
>>> Dec  8 06:38:21 ibd201 kernel: FS:  00002ad76b8acb00(0000)
>>> GS:ffff81022066eb40(0000) knlGS:0000000000000000
>>> Dec  8 06:38:21 ibd201 kernel: CS:  0010 DS: 0000 ES: 0000 CR0:
>>> 000000008005003b
>>> Dec  8 06:38:21 ibd201 kernel: CR2: 00002aaaaabf1000 CR3: 000000021c22b000
>>> CR4: 00000000000006e0
>>> Dec  8 06:38:21 ibd201 kernel: Process nfsd (pid: 15038, threadinfo
>>> ffff810219dde000, task ffff81020d87f0c0)
>>> Dec  8 06:38:21 ibd201 kernel: Stack:  ffffffff8835e547 ffff81020ef96968
>>> ffff81020ef96800 ffff81020ef96958
>>> Dec  8 06:38:21 ibd201 kernel:  ffffffff88360c72 000000010395dc90
>>> ffffffff80424e05 0000000000000000
>>> Dec  8 06:38:21 ibd201 kernel:  0000000000200200 000000010395dc90
>>> ffffffff80239b90 ffff81020d87f0c0
>>> Dec  8 06:38:21 ibd201 kernel: Call Trace:
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff8835e547>]
>>> :sunrpc:svc_rdma_put_context+0x37/0xd0
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff88360c72>]
>>> :sunrpc:svc_rdma_recvfrom+0x5a2/0x11e0
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80424e05>]
>>> schedule_timeout+0x95/0xb0
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80239b90>] process_timeout+0x0/0x10
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80423c2d>]
>>> wait_for_completion_timeout+0xcd/0x150
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80228db0>]
>>> default_wake_function+0x0/0x10
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff881c1402>]
>>> :ib_mthca:mthca_cmd_post+0x232/0x260
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80228db0>]
>>> default_wake_function+0x0/0x10
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff802fac39>] __next_cpu+0x19/0x30
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80227dae>]
>>> find_busiest_group+0x24e/0x6d0
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80424772>] thread_return+0x0/0xde
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff804263f8>]
>>> _spin_unlock_irqrestore+0x8/0x10
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff8023a331>]
>>> try_to_del_timer_sync+0x51/0x60
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff8023a34c>] del_timer_sync+0xc/0x20
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80424e05>]
>>> schedule_timeout+0x95/0xb0
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff883559e6>]
>>> :sunrpc:svc_recv+0x416/0x510
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80228db0>]
>>> default_wake_function+0x0/0x10
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff80228db0>]
>>> default_wake_function+0x0/0x10
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff883a9540>] :nfsd:nfsd+0x0/0x380
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff883a9651>] :nfsd:nfsd+0x111/0x380
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff8020ab9c>] child_rip+0xa/0x12
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff883a9540>] :nfsd:nfsd+0x0/0x380
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff883a9540>] :nfsd:nfsd+0x0/0x380
>>> Dec  8 06:38:21 ibd201 kernel:  [<ffffffff8020ab92>] child_rip+0x0/0x12
>>> Dec  8 06:38:21 ibd201 kernel:
>>> Dec  8 06:38:21 ibd201 kernel:
>>> Dec  8 06:38:21 ibd201 kernel: Code: 0f 0b 68 8c 41 45 80 c2 2c 01 f0 ff 4f 08
>>> 0f 94 c0 84 c0 74
>>> Dec  8 06:38:21 ibd201 kernel: RIP  [<ffffffff8025dff7>] put_page+0x17/0x40
>>> Dec  8 06:38:21 ibd201 kernel:  RSP <ffff810219ddfb08>
>>>
>>> -vu
>>>
> 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
> 

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: swap.objdump
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20061211/a4d86fae/attachment.ksh>