[libfabric-users] How to retrieve the source address of an incoming datagram with prov/verbs?

[Ilango, Arun]

The verbs provider doesn’t support FI_SOURCE for the FI_EP_DGRAM endpoint type today but it can be added. Btw, you’re using the verbs interface of hfi1. Is there a reason to use verbs instead of psm2? What’s the use case here?

Thanks,
Arun.

From: Libfabric-users [mailto:libfabric-users-bounces at lists.openfabrics.org] On Behalf Of Yilong Li
Sent: Friday, August 16, 2019 12:08 AM
To: libfabric-users at lists.openfabrics.org
Subject: [libfabric-users] How to retrieve the source address of an incoming datagram with prov/verbs?

Hi,

I notice that the verbs provider doesn't advertise the FI_SOURCE capability. So how can I retrieve the source address of an incoming datagram? My application has a client-server architecture: the server cannot know the address of a client until the client tries to contact the server.

I am pasting the output of `fi_info -t FI_EP_DGRAM -p verbs -v` on my machine below.

Thanks,
Yilong

---------------------------------------------------------------------
fi_info:
    caps: [ FI_MSG, FI_RECV, FI_SEND, FI_LOCAL_COMM, FI_REMOTE_COMM ]
    mode: [ FI_MSG_PREFIX, FI_RX_CQ_DATA ]
    addr_format: FI_ADDR_IB_UD
    src_addrlen: 32
    dest_addrlen: 0
    src_addr: fi_addr_ib_ud://:::0/0/0/0
    dest_addr: (null)
    handle: (nil)
    fi_tx_attr:
        caps: [ FI_MSG, FI_RECV, FI_SEND, FI_LOCAL_COMM, FI_REMOTE_COMM ]
        mode: [  ]
        op_flags: [  ]
        msg_order: [ FI_ORDER_RAR, FI_ORDER_RAW, FI_ORDER_RAS, FI_ORDER_WAW, FI_ORDER_WAS, FI_ORDER_SAW, FI_ORDER_SAS, FI_ORDER_RMA_RAR, FI_ORDER_RMA_RAW, FI_ORDER_RMA_WAW, FI_ORDER_ATOMIC_RAR, FI_ORDER_ATOMIC_RAW, FI_ORDER_ATOMIC_WAW ]
        comp_order: [ FI_ORDER_STRICT ]
        inject_size: 0
        size: 384
        iov_limit: 4
        rma_iov_limit: 1
    fi_rx_attr:
        caps: [ FI_MSG, FI_RECV, FI_SEND, FI_LOCAL_COMM, FI_REMOTE_COMM ]
        mode: [ FI_MSG_PREFIX, FI_RX_CQ_DATA ]
        op_flags: [  ]
        msg_order: [ FI_ORDER_RAR, FI_ORDER_RAW, FI_ORDER_RAS, FI_ORDER_WAW, FI_ORDER_WAS, FI_ORDER_SAW, FI_ORDER_SAS, FI_ORDER_RMA_RAR, FI_ORDER_RMA_RAW, FI_ORDER_RMA_WAW, FI_ORDER_ATOMIC_RAR, FI_ORDER_ATOMIC_RAW, FI_ORDER_ATOMIC_WAW ]
        comp_order: [ FI_ORDER_STRICT, FI_ORDER_DATA ]
        total_buffered_recv: 0
        size: 384
        iov_limit: 4
    fi_ep_attr:
        type: FI_EP_DGRAM
        protocol: FI_PROTO_IB_UD
        protocol_version: 1
        max_msg_size: 4096
        msg_prefix_size: 40
        max_order_raw_size: 4096
        max_order_war_size: 0
        max_order_waw_size: 4096
        mem_tag_format: 0x0000000000000000
        tx_ctx_cnt: 1
        rx_ctx_cnt: 1
        auth_key_size: 0
    fi_domain_attr:
        domain: 0x0
        name: hfi1_0-dgram
        threading: FI_THREAD_SAFE
        control_progress: FI_PROGRESS_AUTO
        data_progress: FI_PROGRESS_AUTO
        resource_mgmt: FI_RM_ENABLED
        av_type: FI_AV_UNSPEC
        mr_mode: [ FI_MR_LOCAL, FI_MR_ALLOCATED ]
        mr_key_size: 4
        cq_data_size: 4
        cq_cnt: 131071
        ep_cnt: 32768
        tx_ctx_cnt: 1024
        rx_ctx_cnt: 1024
        max_ep_tx_ctx: 1024
        max_ep_rx_ctx: 1024
        max_ep_stx_ctx: 0
        max_ep_srx_ctx: 1024
        cntr_cnt: 0
        mr_iov_limit: 1
    caps: [ FI_LOCAL_COMM, FI_REMOTE_COMM ]
    mode: [  ]
        auth_key_size: 0
        max_err_data: 255
        mr_cnt: 0
    fi_fabric_attr:
        name: IB-0xfe80000000000000
        prov_name: verbs
        prov_version: 1.0
        api_version: 1.8
    fid_nic:
        fi_device_attr:
            name: hfi1_0
            device_id: 9456
            device_version: 17
            vendor_id: 4469
            driver: (null)
            firmware: 1.26.1
        fi_bus_attr:
            fi_bus_type: FI_BUS_UNKNOWN
        fi_link_attr:
            address: (null)
            mtu: 4096
            speed: 104857600
            state: FI_LINK_UP
            network_type: InfiniBand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20190816/3651cbcf/attachment-0001.html>