[libfabric-users] rdma fail with verbs

[Biddiscombe, John A.]

Arun

I just dropped back to tag v1.4.1 for libfabric and tag v1.4.1 for fabtests. Here’s some output 

(NB. I see that using 1.4.1 the provider does not report endpoint type RDM for verbs (which it does for master))

greina7:~/src/libfabric ((tags/v1.4.1^0))$ ~/apps/libfabric/bin/fi_info -p verbs
provider: verbs
    fabric: IB-0x80fe
    domain: mlx5_0
    version: 1.0
    type: FI_EP_MSG
    protocol: FI_PROTO_RDMA_CM_IB_RC


greina7:~/src/libfabric ((tags/v1.4.1^0))$ ~/apps/fabtests/bin/fi_msg_rma -p verbs greina8
libfabric:core:core:fi_param_define_():221<info> registered var provider
libfabric:core:core:fi_param_define_():221<info> registered var fork_unsafe
libfabric:core:core:fi_param_get_():270<info> variable provider=<not set>
libfabric:core:core:fi_param_define_():221<info> registered var provider_path
libfabric:core:core:fi_param_get_():270<info> variable provider_path=<not set>
libfabric:core:core:fi_register_provider():115<info> registering provider: usnic (1.0)
libfabric:verbs:core:fi_param_define_():221<info> registered var iface
libfabric:verbs:core:fi_param_define_():221<info> registered var rdm_buffer_num
libfabric:verbs:core:fi_param_define_():221<info> registered var rdm_buffer_size
libfabric:verbs:core:fi_param_define_():221<info> registered var rdm_rndv_seg_size
libfabric:verbs:core:fi_param_define_():221<info> registered var rdm_cqread_bunch_size
libfabric:verbs:core:fi_param_define_():221<info> registered var rdm_thread_timeout
libfabric:verbs:core:fi_param_define_():221<info> registered var rdm_eager_send_opcode
libfabric:core:core:fi_register_provider():115<info> registering provider: verbs (1.0)
libfabric:core:core:fi_register_provider():115<info> registering provider: UDP (1.0)
libfabric:sockets:core:fi_param_define_():221<info> registered var pe_waittime
libfabric:sockets:core:fi_param_define_():221<info> registered var max_conn_retry
libfabric:sockets:core:fi_param_define_():221<info> registered var def_conn_map_sz
libfabric:sockets:core:fi_param_define_():221<info> registered var def_av_sz
libfabric:sockets:core:fi_param_define_():221<info> registered var def_cq_sz
libfabric:sockets:core:fi_param_define_():221<info> registered var def_eq_sz
libfabric:sockets:core:fi_param_define_():221<info> registered var pe_affinity
libfabric:sockets:core:fi_param_define_():221<info> registered var dgram_drop_rate
libfabric:core:core:fi_register_provider():115<info> registering provider: sockets (2.0)
libfabric:core:core:fi_param_get_():270<info> variable fork_unsafe=<not set>
libfabric:verbs:core:fi_ibv_init_info():946<info> Enabling IB fork support
libfabric:verbs:core:fi_param_get_():270<info> variable rdm_buffer_size=<not set>
libfabric:verbs:fabric:fi_ibv_create_ep():184<info> rdma_create_ep: No such device(19)
libfabric:core:core:fi_getinfo_():553<warn> fi_getinfo: provider verbs returned -19 (No such device)
fi_getinfo(): common/shared.c:454, ret=-61 (No data available)
libfabric:usnic:fabric:usdf_fini():964<trace>
libfabric:verbs:core:fi_param_undefine():158<debug> Removing param: iface
libfabric:verbs:core:fi_param_undefine():158<debug> Removing param: rdm_buffer_num
libfabric:verbs:core:fi_param_undefine():158<debug> Removing param: rdm_buffer_size
libfabric:verbs:core:fi_param_undefine():158<debug> Removing param: rdm_rndv_seg_size
libfabric:verbs:core:fi_param_undefine():158<debug> Removing param: rdm_cqread_bunch_size
libfabric:verbs:core:fi_param_undefine():158<debug> Removing param: rdm_thread_timeout
libfabric:verbs:core:fi_param_undefine():158<debug> Removing param: rdm_eager_send_opcode
libfabric:sockets:core:fi_param_undefine():158<debug> Removing param: pe_waittime
libfabric:sockets:core:fi_param_undefine():158<debug> Removing param: max_conn_retry
libfabric:sockets:core:fi_param_undefine():158<debug> Removing param: def_conn_map_sz
libfabric:sockets:core:fi_param_undefine():158<debug> Removing param: def_av_sz
libfabric:sockets:core:fi_param_undefine():158<debug> Removing param: def_cq_sz
libfabric:sockets:core:fi_param_undefine():158<debug> Removing param: def_eq_sz
libfabric:sockets:core:fi_param_undefine():158<debug> Removing param: pe_affinity
libfabric:sockets:core:fi_param_undefine():158<debug> Removing param: dgram_drop_rate