[openib-general] DAPL and local_iov in RDMA RR/RW mode

[Ryszard Jurga]

Hi Arlin,

Thank you for your quick reply. Both dat_ep_post_rdma_read nad 
dat_ep_post_rdma_write return DAT_SUCCESS. When I read a field 
'transfered_length' from DAT_DTO_COMPLETION_EVENT_DATA after calling a post 
function I receive the correct value which equals num_segs*seg_size. 
Unfortunately, when I read a content of a local buffer, only first segment 
is filled by appropriete data. I have tried to set up debug switch (by 
export DAPL_DBG_TYPE=0xffff before running my application) but unfortunately 
this does not produce any additional output for post functions. Do you have 
any other ideas? I did not mention before, but the case with num_segments>1 
works fine with a send/recv type of transmision.

Best regards,
Ryszard.





----- Original Message ----- 
From: "Arlin Davis" <ardavis at ichips.intel.com>
To: "Ryszard Jurga" <ryszard.jurga at cern.ch>
Cc: "openib" <openib-general at openib.org>
Sent: Friday, August 11, 2006 10:14 PM
Subject: Re: [openib-general] DAPL and local_iov in RDMA RR/RW mode


> Ryszard Jurga wrote:
>
>> Hi everybody,
>>  I have one question about a number of segments in local_iov when using 
>> RDMA Write and Read mode. Is it possible to have num_segments>1? I am 
>> asking, because  when I try to set up num_segments to a value > 1, then I 
>> can still only read/write one segment, even though I have an appropriate 
>> remote buffer already reserved. The size of transfered buffer is 10bytes, 
>> num_segs=2. The information, which is printed below, was obrained from 
>> network devices with one remark - I have set up manualy 
>> max_rdma_read_iov=10 and max_rdma_write_iov=10. Thank you in advance for 
>> your help.
>
> Yes, uDAPL will support num_segments up to the max counts returned on the 
> ep_attr.  Can you be more specific? Does the post return immediate errors 
> or are you simply missing data on the remote node?  Can you turn up the 
> uDAPL debug switch (export DAPL_DBG_TYPE=0xffff) and send output of the 
> post call?
>
> -arlin
>
>> Best regards,
>> Ryszard.
>>  EP_ATTR: the same for both nodes:
>> ----------------------------------
>>  max_message_size=2147483648
>>  max_rdma_size=2147483648
>>  max_recv_dtos=16
>>  max_request_dtos=16
>>  max_recv_iov=4
>>  max_request_iov=4
>>  max_rdma_read_in=4
>>  max_rdma_read_out=4
>>  srq_soft_hw=0
>>  max_rdma_read_iov=10
>>  max_rdma_write_iov=10
>>  ep_transport_specific_count=0
>>  ep_provider_specific_count=0
>> ----------------------------------
>>  IA_ATTR: different for nodes
>> ----------------------------------
>> IA Info:
>>  max_eps=64512
>>  max_dto_per_ep=65535
>>  max_rdma_read_per_ep_in=4
>>  max_rdma_read_per_ep_out=1610616831
>>  max_evds=65408
>>  max_evd_qlen=131071
>>  max_iov_segments_per_dto=28
>>  max_lmrs=131056
>>  max_lmr_block_size=18446744073709551615
>>  max_pzs=32768
>>  max_message_size=2147483648
>>  max_rdma_size=2147483648
>>  max_rmrs=0
>>  max_srqs=0
>>   max_ep_per_srq=0
>>  max_recv_per_srq=143263
>>  max_iov_segments_per_rdma_read=1073741824
>>  max_iov_segments_per_rdma_write=0
>>  max_rdma_read_in=0
>>  max_rdma_read_out=65535
>>  max_rdma_read_per_ep_in_guaranteed=7286
>>  max_rdma_read_per_ep_out_guaranteed=0
>>  IA Info:
>>  max_eps=64512
>>  max_dto_per_ep=65535
>>  max_rdma_read_per_ep_in=4
>>  max_rdma_read_per_ep_out=0
>>  max_evds=65408
>>  max_evd_qlen=131071
>>  max_iov_segments_per_dto=28
>>  max_lmrs=131056
>>  max_lmr_block_size=18446744073709551615
>>  max_pzs=32768
>>  max_message_size=2147483648
>>  max_rdma_size=2147483648
>>  max_rmrs=0
>>  max_srqs=0
>>  max_ep_per_srq=0
>>  max_recv_per_srq=142247
>>  max_iov_segments_per_rdma_read=1073741824
>>  max_iov_segments_per_rdma_write=0
>>  max_rdma_read_in=0
>>  max_rdma_read_out=65535
>>  max_rdma_read_per_ep_in_guaranteed=7286
>>  max_rdma_read_per_ep_out_guaranteed=28
>>
>>------------------------------------------------------------------------
>>
>>_______________________________________________
>>openib-general mailing list
>>openib-general at openib.org
>>http://openib.org/mailman/listinfo/openib-general
>>
>>To unsubscribe, please visit 
>>http://openib.org/mailman/listinfo/openib-general
>>