<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.0.6618.4">
<TITLE>How to increase the waiting period/timeout for the completion of an RDMA operation?</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P DIR=LTR><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">Yesterday I</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> complained</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> about a gen2 x86</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> client RDMA read not working</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> on a gen1 PowerPC server.</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">Today I got my hardware guy measuring the PCI-Express interface on the PowerPC</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> gen1 server.</SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">It seems as if the RDMA read is at least started.</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> So all</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> ideas</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> about endianess problems</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> </SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> </SPAN></P>
<P DIR=LTR><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">To explain the problem I have to give a</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> little more</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> details on our application:</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">We are using the InfiniBand interface between</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> a PowerPC gen1 server</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> and</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> a</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> gen1/</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">gen 2</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> x86</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> client</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> for</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> high speed</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> film image transport from a scanner</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> (server)</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> to a workstation</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> (client)</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">.</SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">In order to get the fastest response and performance the x86 client reads the images out of the scanner by RDMA</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> on a hardware</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> FIFO</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> which is registered as physical memory</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">.</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">Unfortunately the scanner does not always have an image ready to</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> deliver.</SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">So to avoid</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> time consuming</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> connected, rdma and</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> disconnect</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> for each and every image I</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">’</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">m holding the connection up</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> and the scanner</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> FIFO</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> hardware delays the responses to the memory reads up to 500 msec.</SPAN></P>
<P DIR=LTR><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN></P>
<P DIR=LTR><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">A</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">nd here is now the problem:</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">It seems as if the gen2 stack does not like that long delay on the rdma.</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> </SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">Since a gen1 stack</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> client</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> can live with that long delay I think</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> the gen2 stack should be able to do the same on the same hardware?</SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">The question</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> now is:</SPAN></P>
<P DIR=LTR><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"> How to increase the waiting period/timeout for the completion of an RDMA operation?</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">Thin</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb">gs are getting clearer.</SPAN></P>
<P DIR=LTR><SPAN LANG="en-gb">Thomas</SPAN><SPAN LANG="de"></SPAN><SPAN LANG="en-gb"></SPAN></P>
</BODY>
</HTML>