[openib-general] [oops] recent opensm crash

Tom Duffy tduffy at sun.com
Thu Jun 30 15:10:07 PDT 2005


#0  stack_dump () at src/stack.c:72
72              if (!__builtin_frame_address(2))
(gdb) bt
#0  stack_dump () at src/stack.c:72
#1  0x00002aaaaacbd1a6 in handler (x=11) at src/stack.c:151
#2  <signal handler called>
#3  __osm_sm_mad_ctrl_send_err_cb (bind_context=0x550dd8, p_madw=0x567820)
    at osm_sm_mad_ctrl.c:832
#4  0x00002aaaaaaaeeed in osm_vendor_send (h_bind=0x586920, p_madw=0x567820,
    resp_expected=1) at osm_vendor_ibumad.c:889
#5  0x000000000042ef72 in __osm_vl15_poller (p_ptr=0x552620) at osm_madw.h:933
#6  0x00002aaaaadc911e in __cl_thread_wrapper (arg=0x0) at cl_thread.c:61
#7  0x00000036d28060aa in start_thread () from /lib64/tls/libpthread.so.0
#8  0x00000036d19c53d3 in clone () from /lib64/tls/libc.so.6
#9  0x0000000000000000 in ?? ()

I was bringing up and down an node when this happened.

Attached are the last 500 lines from osm.log.

-tduffy
-------------- next part --------------
				base_ver................0x1
				mgmt_class..............0x81
				class_ver...............0x1
				method..................0x1 (SubnGet)
				status..................0x0
				hop_ptr.................0x0
				hop_count...............0x1
				trans_id................0x16c6a
				attr_id.................0x16 (P_KeyTable)
				resv....................0x0
				attr_mod................0x50000
				m_key...................0x0000000000000000
				dr_slid.................0xFFFF
				dr_dlid.................0xFFFF

				Initial path: [0][1]
				Return path:  [0][0]
				Reserved:     [0][0][0][0][0][0][0]

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

Jun 30 14:59:00 [43806960] -> osm_vendor_send: [
Jun 30 14:59:00 [43005960] -> PortInfo dump:
				port number.............0x8
				node_guid...............0x000617000000000d
				port_guid...............0x000617000000000d
				m_key...................0x0000000000000000
				subnet_prefix...........0x0000000000000000
				base_lid................0x0
				master_sm_base_lid......0x0
				capability_mask.........0x0
				diag_code...............0x0
				m_key_lease_period......0x0
				local_port_num..........0x7
				link_width_enabled......0x3
				link_width_supported....0x3
				link_width_active.......0x0
				link_speed_supported....0x1
				port_state..............DOWN
				state_info2.............0x22
				m_key_protect_bits......0x0
				lmc.....................0x0
				link_speed..............0x11
				mtu_smsl................0x10
				vl_cap..................0x40
				vl_high_limit...........0x0
				vl_arb_high_cap.........0x20
				vl_arb_low_cap..........0x20
				mtu_cap.................0x5
				vl_stall_life...........0x8
				vl_enforce..............0x10
				m_key_violations........0x0
				p_key_violations........0x0
				q_key_violations........0x0
				guid_cap................0x0
				subnet_timeout..........0x0
				resp_time_value.........0x0
				error_threshold.........0xFF
Jun 30 14:59:00 [43005960] -> Capabilities Mask:
Jun 30 14:59:00 [43005960] -> __osm_pi_rcv_process_switch_port: [
Jun 30 14:59:00 [43005960] -> __osm_pi_rcv_process_switch_port: ]
Jun 30 14:59:00 [43005960] -> __osm_pi_rcv_get_pkey_slvl_vla_tables: [
Jun 30 14:59:00 [43005960] -> osm_physp_has_pkey: [
Jun 30 14:59:00 [43005960] -> osm_req_get: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: [
Jun 30 14:59:00 [43806960] -> __osm_mtl_send_callback: Completed Sending Request MADW: 0x5b0ac0.
Jun 30 14:59:00 [43806960] -> osm_vendor_send: ]
Jun 30 14:59:00 [43005960] -> osm_vendor_get: [
Jun 30 14:59:00 [44808960] -> osm_vendor_put: [
Jun 30 14:59:00 [44808960] -> osm_vendor_put: Retiring UMAD 0x591a50.
Jun 30 14:59:00 [44808960] -> osm_vendor_put: ]
Jun 30 14:59:00 [44808960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43806960] -> __osm_vl15_poller: 1 on wire, 11 outstanding, 10 unicasts sent, 88641 sent total.
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_process_get_resp: Posting Dispatcher message OSM_MSG_MAD_NODE_INFO.
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_process_get_resp: ]
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_rcv_callback: ]
Jun 30 14:59:00 [43005960] -> osm_vendor_get: Acquiring UMAD for p_madw = 0x567428, size = 256.
Jun 30 14:59:00 [44808960] -> osm_mad_pool_get: [
Jun 30 14:59:00 [44808960] -> osm_vendor_get: [
Jun 30 14:59:00 [43005960] -> osm_vendor_get: Acquired UMAD 0x592620, size = 256.
Jun 30 14:59:00 [43005960] -> osm_vendor_get: ]
Jun 30 14:59:00 [44808960] -> osm_vendor_get: Acquiring UMAD for p_madw = 0x560268, size = 256.
Jun 30 14:59:00 [44808960] -> osm_vendor_get: Acquired UMAD 0x591a50, size = 256.
Jun 30 14:59:00 [44808960] -> osm_vendor_get: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: Acquired p_madw = 0x567410, p_mad = 0x592654, size = 256.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: ]
Jun 30 14:59:00 [44808960] -> osm_mad_pool_get: Acquired p_madw = 0x560250, p_mad = 0x591a84, size = 256.
Jun 30 14:59:00 [44808960] -> osm_mad_pool_get: ]
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_rcv_callback: [
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_rcv_callback: 88641 QP0 MADs received.
Jun 30 14:59:00 [43005960] -> osm_req_get: Getting P_KeyTable (0x16), modifier = 0x80000, TID = 0x16c6d.
Jun 30 14:59:00 [43005960] -> osm_vl15_post: [
Jun 30 14:59:00 [43005960] -> osm_vl15_post: Servicing p_madw = 0x567410 (mad 0x592654 req 1)
Jun 30 14:59:00 [44808960] -> SMP dump:
				base_ver................0x1
				mgmt_class..............0x81
				class_ver...............0x1
				method..................0x81 (SubnGetResp)
				status..................0x8000
				hop_ptr.................0x0
				hop_count...............0x1
				trans_id................0x16c6a
				attr_id.................0x16 (P_KeyTable)
				resv....................0x0
				attr_mod................0x50000
				m_key...................0x0000000000000000
				dr_slid.................0xFFFF
				dr_dlid.................0xFFFF

				Initial path: [0][1]
				Return path:  [0][7]
				Reserved:     [0][0][0][0][0][0][0]

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_process_get_resp: [
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_update_wire_stats: [
Jun 30 14:59:00 [43005960] -> osm_vl15_post: 1 MADs on wire, 12 MADs outstanding.
Jun 30 14:59:00 [43005960] -> osm_vl15_poll: [
Jun 30 14:59:00 [43005960] -> osm_vl15_poll: Signalling poller thread.
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_update_wire_stats: 0 SMPs on the wire, 12 outstanding.
Jun 30 14:59:00 [44808960] -> osm_vl15_poll: [
Jun 30 14:59:00 [44808960] -> osm_vl15_poll: Signalling poller thread.
Jun 30 14:59:00 [44808960] -> osm_vl15_poll: ]
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_update_wire_stats: ]
Jun 30 14:59:00 [44808960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [44808960] -> osm_mad_pool_put: Releasing p_madw = 0x5b0ac0, p_mad = 0x5930e4.
Jun 30 14:59:00 [44808960] -> osm_vendor_put: [
Jun 30 14:59:00 [44808960] -> osm_vendor_put: Retiring UMAD 0x5930b0.
Jun 30 14:59:00 [44808960] -> osm_vendor_put: ]
Jun 30 14:59:00 [44808960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43806960] -> __osm_vl15_poller: Servicing p_madw = 0x567820 (mad 0x591bc4 req 1)
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_process_get_resp: Posting Dispatcher message OSM_MSG_MAD_PKEY.
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_process_get_resp: ]
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_rcv_callback: ]
Jun 30 14:59:00 [43806960] -> SMP dump:
				base_ver................0x1
				mgmt_class..............0x81
				class_ver...............0x1
				method..................0x1 (SubnGet)
				status..................0x0
				hop_ptr.................0x0
				hop_count...............0x1
				trans_id................0x16c6b
				attr_id.................0x16 (P_KeyTable)
				resv....................0x0
				attr_mod................0x60000
				m_key...................0x0000000000000000
				dr_slid.................0xFFFF
				dr_dlid.................0xFFFF

				Initial path: [0][1]
				Return path:  [0][0]
				Reserved:     [0][0][0][0][0][0][0]

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

Jun 30 14:59:00 [43005960] -> osm_vl15_poll: ]
Jun 30 14:59:00 [43806960] -> osm_vendor_send: [
Jun 30 14:59:00 [43005960] -> osm_vl15_post: ]
Jun 30 14:59:00 [43005960] -> osm_req_get: ]
Jun 30 14:59:00 [43005960] -> osm_physp_has_pkey: ]
Jun 30 14:59:00 [43005960] -> __osm_pi_rcv_get_pkey_slvl_vla_tables: ]
Jun 30 14:59:00 [43005960] -> osm_pi_rcv_process: ]
Jun 30 14:59:00 [44808960] -> osm_mad_pool_get: [
Jun 30 14:59:00 [44808960] -> osm_vendor_get: [
Jun 30 14:59:00 [44808960] -> osm_vendor_get: Acquiring UMAD for p_madw = 0x5b0ad8, size = 256.
Jun 30 14:59:00 [44808960] -> osm_vendor_get: Acquired UMAD 0x5930b0, size = 256.
Jun 30 14:59:00 [44808960] -> osm_vendor_get: ]
Jun 30 14:59:00 [44808960] -> osm_mad_pool_get: Acquired p_madw = 0x5b0ac0, p_mad = 0x5930e4, size = 256.
Jun 30 14:59:00 [44808960] -> osm_mad_pool_get: ]
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_rcv_callback: [
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_rcv_callback: 88642 QP0 MADs received.
Jun 30 14:59:00 [43806960] -> osm_vendor_send: Send failed -5 (Success).
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43806960] -> __osm_sm_mad_ctrl_send_err_cb: [
Jun 30 14:59:00 [44808960] -> SMP dump:
				base_ver................0x1
				mgmt_class..............0x81
				class_ver...............0x1
				method..................0x81 (SubnGetResp)
				status..................0x8000
				hop_ptr.................0x0
				hop_count...............0x1
				trans_id................0x16c6b
				attr_id.................0x16 (P_KeyTable)
				resv....................0x0
				attr_mod................0x60000
				m_key...................0x0000000000000000
				dr_slid.................0xFFFF
				dr_dlid.................0xFFFF

				Initial path: [0][1]
				Return path:  [0][7]
				Reserved:     [0][0][0][0][0][0][0]

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

				00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00

Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_process_get_resp: [
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_update_wire_stats: [
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_update_wire_stats: 0 SMPs on the wire, 12 outstanding.
Jun 30 14:59:00 [44808960] -> osm_vl15_poll: [
Jun 30 14:59:00 [44808960] -> osm_vl15_poll: Signalling poller thread.
Jun 30 14:59:00 [44808960] -> osm_vl15_poll: ]
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_update_wire_stats: ]
Jun 30 14:59:00 [44808960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [44808960] -> osm_mad_pool_put: Releasing p_madw = 0x567820, p_mad = 0x591bc4.
Jun 30 14:59:00 [44808960] -> osm_vendor_put: [
Jun 30 14:59:00 [44808960] -> osm_vendor_put: Retiring UMAD 0x591b90.
Jun 30 14:59:00 [44808960] -> osm_vendor_put: ]
Jun 30 14:59:00 [44808960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_process_get_resp: Posting Dispatcher message OSM_MSG_MAD_PKEY.
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_process_get_resp: ]
Jun 30 14:59:00 [44808960] -> __osm_sm_mad_ctrl_rcv_callback: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c62.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x5b0370, p_mad = 0x5b1244.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x5b1210.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 11 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: [
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 port_num 0 with GUID = 0x617000000000d for parent node GUID = 0x617000000000d, TID = 0x16c63.
Jun 30 14:59:00 [43005960] -> P_Key table dump:
			port_guid...........0x000617000000000d
			block_num...........0x0
			port_num............0x0
	P_Key Table:  0XFFFF | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 |
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c63.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x55df60, p_mad = 0x5b0c74.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x5b0c40.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 10 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: [
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 port_num 1 with GUID = 0x617000000000d for parent node GUID = 0x617000000000d, TID = 0x16c64.
Jun 30 14:59:00 [43005960] -> P_Key table dump:
			port_guid...........0x000617000000000d
			block_num...........0x0
			port_num............0x1
	P_Key Table:  0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 |
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c64.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x5b0920, p_mad = 0x5916c4.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x591690.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 9 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: [
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 port_num 2 with GUID = 0x617000000000d for parent node GUID = 0x617000000000d, TID = 0x16c65.
Jun 30 14:59:00 [43005960] -> P_Key table dump:
			port_guid...........0x000617000000000d
			block_num...........0x0
			port_num............0x2
	P_Key Table:  0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 |
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c65.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x55add0, p_mad = 0x591d04.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x591cd0.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 8 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_ni_rcv_process: [
Jun 30 14:59:00 [43005960] -> NodeInfo dump:
				base_version............0x1
				class_version...........0x1
				node_type...............Channel Adapter
				num_ports...............0x2
				sys_guid................0x0002c9000100d050
				node_guid...............0x0002c901097624c0
				port_guid...............0x0002c901097624c1
				partition_cap...........0x40
				device_id...............0x5A44
				revision................0xA1
				port_num................0x1
				vendor_id...............0x2C9
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing: [
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing: Rediscovered Channel Adapter node 0x2c901097624c0
				TID = 0x16c66, discovered 0 times already.
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing_ca: [
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_ca_port: [
Jun 30 14:59:00 [43005960] -> osm_req_get: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: [
Jun 30 14:59:00 [43005960] -> osm_vendor_get: [
Jun 30 14:59:00 [43005960] -> osm_vendor_get: Acquiring UMAD for p_madw = 0x55ade8, size = 256.
Jun 30 14:59:00 [43005960] -> osm_vendor_get: Acquired UMAD 0x5b1210, size = 256.
Jun 30 14:59:00 [43005960] -> osm_vendor_get: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: Acquired p_madw = 0x55add0, p_mad = 0x5b1244, size = 256.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: ]
Jun 30 14:59:00 [43005960] -> osm_req_get: Getting PortInfo (0x15), modifier = 0x1, TID = 0x16c6e.
Jun 30 14:59:00 [43005960] -> osm_vl15_post: [
Jun 30 14:59:00 [43005960] -> osm_vl15_post: Servicing p_madw = 0x55add0 (mad 0x5b1244 req 1)
Jun 30 14:59:00 [43005960] -> osm_vl15_post: 0 MADs on wire, 9 MADs outstanding.
Jun 30 14:59:00 [43005960] -> osm_vl15_poll: [
Jun 30 14:59:00 [43005960] -> osm_vl15_poll: Signalling poller thread.
Jun 30 14:59:00 [43005960] -> osm_vl15_poll: ]
Jun 30 14:59:00 [43005960] -> osm_vl15_post: ]
Jun 30 14:59:00 [43005960] -> osm_req_get: ]
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_ca_port: ]
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing_ca: ]
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_set_links: [
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_set_links: Link already exists.
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_set_links: ]
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing: ]
Jun 30 14:59:00 [43005960] -> osm_ni_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c66.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x5b09f0, p_mad = 0x592fa4.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x592f70.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 8 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: [
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 port_num 3 with GUID = 0x617000000000d for parent node GUID = 0x617000000000d, TID = 0x16c67.
Jun 30 14:59:00 [43005960] -> P_Key table dump:
			port_guid...........0x000617000000000d
			block_num...........0x0
			port_num............0x3
	P_Key Table:  0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 |
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c67.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x561f90, p_mad = 0x592e64.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x592e30.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 7 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: [
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 port_num 4 with GUID = 0x617000000000d for parent node GUID = 0x617000000000d, TID = 0x16c68.
Jun 30 14:59:00 [43005960] -> P_Key table dump:
			port_guid...........0x000617000000000d
			block_num...........0x0
			port_num............0x4
	P_Key Table:  0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 |
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c68.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x5b0850, p_mad = 0x591804.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x5917d0.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 6 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_ni_rcv_process: [
Jun 30 14:59:00 [43005960] -> NodeInfo dump:
				base_version............0x1
				class_version...........0x1
				node_type...............Channel Adapter
				num_ports...............0x2
				sys_guid................0x0002c90109765633
				node_guid...............0x0002c90109765630
				port_guid...............0x0002c90109765631
				partition_cap...........0x20
				device_id...............0x5A44
				revision................0xA1
				port_num................0x1
				vendor_id...............0x2C9
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing: [
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing: Rediscovered Channel Adapter node 0x2c90109765630
				TID = 0x16c69, discovered 0 times already.
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing_ca: [
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_ca_port: [
Jun 30 14:59:00 [43005960] -> osm_req_get: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: [
Jun 30 14:59:00 [43005960] -> osm_vendor_get: [
Jun 30 14:59:00 [43005960] -> osm_vendor_get: Acquiring UMAD for p_madw = 0x5b0868, size = 256.
Jun 30 14:59:00 [43005960] -> osm_vendor_get: Acquired UMAD 0x5b0c40, size = 256.
Jun 30 14:59:00 [43005960] -> osm_vendor_get: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: Acquired p_madw = 0x5b0850, p_mad = 0x5b0c74, size = 256.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_get: ]
Jun 30 14:59:00 [43005960] -> osm_req_get: Getting PortInfo (0x15), modifier = 0x1, TID = 0x16c6f.
Jun 30 14:59:00 [43005960] -> osm_vl15_post: [
Jun 30 14:59:00 [43005960] -> osm_vl15_post: Servicing p_madw = 0x5b0850 (mad 0x5b0c74 req 1)
Jun 30 14:59:00 [43005960] -> osm_vl15_post: 0 MADs on wire, 7 MADs outstanding.
Jun 30 14:59:00 [43005960] -> osm_vl15_poll: [
Jun 30 14:59:00 [43005960] -> osm_vl15_poll: Signalling poller thread.
Jun 30 14:59:00 [43005960] -> osm_vl15_poll: ]
Jun 30 14:59:00 [43005960] -> osm_vl15_post: ]
Jun 30 14:59:00 [43005960] -> osm_req_get: ]
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_ca_port: ]
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing_ca: ]
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_set_links: [
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_set_links: Link already exists.
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_set_links: ]
Jun 30 14:59:00 [43005960] -> __osm_ni_rcv_process_existing: ]
Jun 30 14:59:00 [43005960] -> osm_ni_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c69.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x5b02a0, p_mad = 0x592144.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x592110.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 6 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: [
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 port_num 5 with GUID = 0x617000000000d for parent node GUID = 0x617000000000d, TID = 0x16c6a.
Jun 30 14:59:00 [43005960] -> P_Key table dump:
			port_guid...........0x000617000000000d
			block_num...........0x0
			port_num............0x5
	P_Key Table:  0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 |
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c6a.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x560250, p_mad = 0x592514.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x5924e0.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 5 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: [
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: Got GetResp(PKey) block:0 port_num 6 with GUID = 0x617000000000d for parent node GUID = 0x617000000000d, TID = 0x16c6b.
Jun 30 14:59:00 [43005960] -> P_Key table dump:
			port_guid...........0x000617000000000d
			block_num...........0x0
			port_num............0x6
	P_Key Table:  0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 | 0X0 |
Jun 30 14:59:00 [43005960] -> osm_pkey_rcv_process: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: [
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: Retiring MAD with TID = 0x16c6b.
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: [
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: Releasing p_madw = 0x5b0ac0, p_mad = 0x591a84.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: [
Jun 30 14:59:00 [43005960] -> osm_vendor_put: Retiring UMAD 0x591a50.
Jun 30 14:59:00 [43005960] -> osm_vendor_put: ]
Jun 30 14:59:00 [43005960] -> osm_mad_pool_put: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: 4 QP0 MADs outstanding.
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_retire_trans_mad: ]
Jun 30 14:59:00 [43005960] -> __osm_sm_mad_ctrl_disp_done_callback: ]
Jun 30 14:59:00 [43806960] -> __osm_sm_mad_ctrl_send_err_cb: ERR 3113: MAD completed in error (IB_SUCCESS).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050630/84efbbd0/attachment.sig>


More information about the general mailing list