Hi Roland,<br>Thanks for your reply!<br>The previous problem disappeared after I modified my code as you told. But another one while connecting hold me up:<br><br>Feb 28 23:05:02 linux3 kernel: VIPKL(1): var/tmp/IBGD/tmp/openib/infiniband/ib_verbs/hw/mellanox-hca/vip/qpm.c[1108]: VAPI_RESET=>VAPI_RESET, must_mask: 0x0, allowed_mask: 0x1
<br>Feb 28 23:05:02 linux3 kernel: VIPKL(1): var/tmp/IBGD/tmp/openib/infiniband/ib_verbs/hw/mellanox-hca/vip/qpm.c[1111]: The following bits are not supported or allowed in this transition: 0x30<br>Feb 28 23:05:02 linux3 kernel:
<br>Feb 28 23:05:02 linux3 kernel: [KERNEL_IB][tsIbTavorQpModify][/var/tmp/IBGD/tmp/openib/infiniband/ib_verbs/hw/provider/tavor_qp.c:457]InfiniHost_III_Lx0: VAPI_modify_qp failed, return code = -233 (Unsupported attribute)
<br>Feb 28 23:05:02 linux3 kernel: [IB_CM][ib_cm_connect][/var/tmp/IBGD/tmp/openib/infiniband/ib_cm/cm_api.c:168]ib_qp_modify to INIT failed<br><br>Was this caused by my improper QP creation?<br>Any suggestion is appreciated. Here are the relative codes of my own:
<br><br>create QP<br>=======<br> qp_param.limit.max_outstanding_send_request =<br> params_p->ib_res.cm_send_cq_size;<br> qp_param.limit.max_outstanding_receive_request =<br> params_p->ib_res.cm_recv_cq_size;
<br> qp_param.limit.max_send_gather_element = 1;<br> qp_param.limit.max_receive_scatter_element = 1;<br> qp_param.pd = params_p->ib_res.cm_pd_p;<br> qp_param.send_queue = params_p->ib_res.cm_send_cq_p;<br>
qp_param.receive_queue = params_p->ib_res.cm_recv_cq_p;<br> qp_param.send_policy = IB_WQ_SIGNAL_SELECTABLE;<br> qp_param.receive_policy = IB_WQ_SIGNAL_ALL;<br> qp_param.transport = IB_TRANSPORT_RC;
<br> qp_param.device_specific = NULL;<br><br> res = ib_qp_create(&qp_param,<br> ¶ms_p->ib_res.cm_qp_p,<br> ¶ms_p->ib_res.cm_qpn);<br> if (res) {<br> PRINT_ERR("ib_qp_create failed\n");
<br> cm_ib_clean(¶ms_p->ib_res);<br> return -1;<br> }<br> PRINT_TRACE("QP created: 0x%p, Num: 0x%x\n", params_p->ib_res.cm_qp_p, params_p->ib_res.cm_qpn);<br><br><br><br>connect
<br>======<br> active_param.qp = params_p->ib_res.cm_qp_p;<br> active_param.req_private_data = msg_p;<br> active_param.req_private_data_len = msg_len;<br> active_param.responder_resources = 4;
<br> active_param.initiator_depth = 4;<br> active_param.retry_count = 7;<br> active_param.rnr_retry_count = 7;<br> active_param.cm_response_timeout = 20; /* 4 seconds */<br> active_param.max_cm_retries = 15;
<br> active_param.flow_control = 1;<br><br> params_p->ib_res.cm_path.mtu =<br> (params_p->ib_res.cm_hca_infor.device_id == 23108) ?<br> MTU1024 : MTU2048;<br> memcpy(¶ms_p->ib_res.cm_path.sgid, params_p->ib_res.cm_port_gid, sizeof(tTS_IB_GID));
<br> memcpy(¶ms_p->ib_res.cm_path.dgid, params_p->ib_res.cm_port_gid, sizeof(tTS_IB_GID));<br> params_p->ib_res.cm_path.packet_life = 13; /* FIXME */<br> PRINT_TRACE("connect...\n");<br>
res = ib_cm_connect(&active_param,<br> ¶ms_p->ib_res.cm_path,<br> NULL,<br> params_p->ib_res.dst_service_id,<br> 0,<br> params_p->ib_res.call_back_func,
<br> (void *)params_p,<br> ¶ms_p->ib_res.cm_comm_id_conn);<br> if (res) {<br> PRINT_ERR("ib_cm_connect failed\n");<br> return -1;<br> }<br><br><br clear="all">
<br>-- <br>Ian Jiang<br><a href="mailto:ianjiang.ict@gmail.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
ianjiang.ict@gmail.com</a><br><br>Laboratory of Spatial Information Technology<br>Division of System Architecture<br>Institute of Computing Technology<br>Chinese Academy of Sciences