[libfabric-users] Fwd:

[Anders Dam Jensen]

Hi Libfabric community,

I have been "playing" with libfabric for a while wishing to write a
provider agnostic communication interface for our application - to the
extend possible. In doing tests for how to best do this, I have stumble on
an issue that is either due to a misunderstanding on my side or a possible
bug when using udp;ofi_rxd. I have been able to reproduce the issue with
the code below. Basically with the tcp and the udp provider, I am able to
fi_av_insert an ip address and fi_av_lookup it up afterwards. However if I
select the upd;ofi_rxd provider, I can fi_av_insert the address with
success, but fi_av_lookup fails to find the address afterward, error -61 ->
No data available. Please see code and output below.

Any feedback is most welcomed, even if it is that I have simply
misunderstood something.

Thanks & Best Regards, Anders

#include <iostream>

#include <rdma/fabric.h>
#include <rdma/fi_domain.h>
#include <cstring>


void result( int err, const char *name )
{
    printf( "%s: %i -> %s\n", name, err, fi_strerror( err ) );
}

void av_test( fi_info *hints )
{
    fi_info *fi;
    fid_fabric *fabric;
    fid_domain *domain;
    fid_av *av;
    fi_addr_t peer_addr {};


    int err = fi_getinfo( FI_VERSION(1, 21), nullptr, "5000",
FI_SOURCE, hints, &fi);

    result( err, "fi_getinfo" );

    printf("Provider: %s\n", fi->fabric_attr->prov_name);

    err = fi_fabric(fi->fabric_attr, &fabric, nullptr);
    result( err, "fi_fabric");

    err = fi_domain(fabric, fi, &domain, nullptr );
    result( err, "fi_domain" );

    fi_av_attr av_attr {};

    av_attr.type = FI_AV_UNSPEC;

    err = fi_av_open(domain, &av_attr, &av, nullptr );
    result( err, "fi_av_open" );

    char dst_addr[] = {2, 0, 19, -120, 10, 0, -1, 2, 0, 0, 0, 0, 0, 0,
0, 0, 0 };

    err = fi_av_insert(av, dst_addr, 1, &peer_addr, 0, NULL);
    if( err == 1 ) err = 0; // fi_av_insert return number of
successful insert upon success, i.e. 1
    result( err, "fi_av_insert" );

    char lookupAddr[ 64 ]; size_t len = 64;
    err = fi_av_lookup(av, peer_addr, lookupAddr, &len);
    result( err, "fi_av_lookup" );

    printf( "\n" );

    fi_close((fid_t)av);
    fi_close((fid_t)domain);
    fi_close((fid_t)fabric);
}

void test_tcp()
{
    fi_info *hints;
    hints = fi_allocinfo();

    hints->ep_attr->type = FI_EP_RDM;
    hints->caps = FI_MSG;
    hints->addr_format = FI_SOCKADDR;
    hints->fabric_attr->prov_name = strdup("tcp");

    av_test( hints );

    fi_freeinfo( hints );
}

void test_dgram_udp()
{
    fi_info *hints;
    hints = fi_allocinfo();

    hints->ep_attr->type = FI_EP_DGRAM;
    hints->caps = FI_MSG;
    hints->addr_format = FI_SOCKADDR;
    hints->fabric_attr->prov_name = strdup("udp");

    av_test( hints );

    fi_freeinfo( hints );
}

void test_rdm_udp()
{
    fi_info *hints;
    hints = fi_allocinfo();

    hints->ep_attr->type = FI_EP_RDM;
    hints->caps = FI_MSG;
    hints->addr_format = FI_SOCKADDR;
    hints->fabric_attr->prov_name = strdup("udp");

    av_test( hints );

    fi_freeinfo( hints );
}


int main()
{
    test_tcp();
    test_dgram_udp();
    test_rdm_udp();
}


Output:

fi_getinfo: 0 -> Success
Provider: tcp
fi_fabric: 0 -> Success
fi_domain: 0 -> Success
fi_av_open: 0 -> Success
fi_av_insert: 0 -> Success
fi_av_lookup: 0 -> Success

fi_getinfo: 0 -> Success
Provider: udp
fi_fabric: 0 -> Success
fi_domain: 0 -> Success
fi_av_open: 0 -> Success
fi_av_insert: 0 -> Success
fi_av_lookup: 0 -> Success

fi_getinfo: 0 -> Success
Provider: udp;ofi_rxd
fi_fabric: 0 -> Success
fi_domain: 0 -> Success
fi_av_open: 0 -> Success
fi_av_insert: 0 -> Success
fi_av_lookup: -61 -> No data available


Process finished with exit code 0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20240710/1d1fe662/attachment.htm>