[ofa-general] uDAPL private data size issue

[Tang, Changqing]

How to switch to scm ?

Here is my /etc/dat.conf and libdapl*:

mpixbl09:/usr/lib64:cat /etc/dat.conf
#
# DAT 1.2 and 2.0 configuration file
#
# Each entry should have the following fields:
#
# <ia_name> <api_version> <threadsafety> <default> <lib_path> \
#           <provider_version> <ia_params> <platform_params>
#
# For the uDAPL cma provder, specify <ia_params> as one of the following:
#       network address, network hostname, or netdev name and 0 for port
#
# Simple (OpenIB-cma) default with netdev name provided first on list
# to enable use of same dat.conf version on all nodes
#
# 1.2 and 2.0 examples for multiple interfaces, IPoIB HA failover, bonding:
#
OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib0 0" ""
OpenIB-cma-1 u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib1 0"
""
OpenIB-cma-2 u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib2 0"
""
OpenIB-cma-3 u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib3 0"
""
OpenIB-bond u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "bond0 0"
""
ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "ib0 0" ""
ofa-v2-ib1 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "ib1 0" ""
ofa-v2-ib2 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "ib2 0" ""
ofa-v2-ib3 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "ib3 0" ""
ofa-v2-bond u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "bond0 0"
""

mpixbl09:/usr/lib64:
mpixbl09:/usr/lib64:ls libdapl*
libdaplcma.so    libdaplcma.so.1.0.2  libdaplofa.so    libdaplofa.so.2.0.0
libdaplcma.so.1  libdaplofa.a         libdaplofa.so.2
mpixbl09:/usr/lib64:



> -----Original Message-----
> From: Davis, Arlin R [mailto:arlin.r.davis at intel.com]
> Sent: Wednesday, August 20, 2008 1:31 PM
> To: Tang, Changqing; general at lists.openfabrics.org
> Cc: Sean Hefty
> Subject: RE: [ofa-general] uDAPL private data size issue
>
>
> >Hi, uDAPL Developers:
> >
> >        I have a system running uDAPL 2.0, the dat_ia_query() return
> >the provider attributes, the 'max_private_data_size' is only 48.
> >
> >        The standard 2.0 says that private data size is at least 64
> >bytes:
> >
> >        Is there any way to tune private data size to 128
> bytes, or at
> >least 64 bytes ?
>
> It actually should report 56 bytes..
>
> IB cm supports 92 bytes on requests but rdma_cm steals 36 bytes as
> follow:
>
> union cma_ip_addr {
>         struct in6_addr ip6;
>         struct {
>                 __u32 pad[3];
>                 __u32 addr;
>         } ip4;
> };
>
> struct cma_hdr {
>         u8 cma_version;
>         u8 ip_version;  /* IP version: 7:4 */
>         __u16 port;
>         union cma_ip_addr src_addr;
>         union cma_ip_addr dst_addr;
> };
>
> I guess that makes the uDAPL cma provider non-compliant
> unless there is a way for rdma_cm to give back some of IB CM
> request private data area.
>
> Sean, is there anything that can be done here?
>
> CQ Tang, can you use the uDAPL scm provider instead of cma?
>
> -arlin
>
>