<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.45">
<TITLE>RE: ptrace peektext failure for Mellanox IBGD 1.7.0 based cluster</TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=2>David,</FONT>
<BR><FONT SIZE=2>IBGD 1.7 does not support kernel 2.6.11 so I assume you have made changes to IBGD to make it compile.</FONT>
<BR><FONT SIZE=2>In the files you sent I can't see a call to ptrace with PTRACE_PEEKTEXT but I can see a call to PTRACE_PEEKDATA. Note that in the IBGD stack, registered buffers are not inherited by a child process when a the parent forks. This is accomplished by setting the VM_DONTCOPY flag on the vma. It is so done to retain the virtual to physical translation of a page at the parent by disabling COW on the pages. So the child may not even have these buffers in its address space and this could be the reason why ptrace fails.</FONT></P>
<P><FONT SIZE=2>Note also that IBGD 1.8 is the latest release and it does support kernel 2.6.11 so you may consider using it, though the description above holds also for IBGD 1.8</FONT></P>
<P><FONT SIZE=2>Eli</FONT>
</P>
<P><FONT SIZE=2>-----Original Message-----</FONT>
<BR><FONT SIZE=2>From: David Lecomber [<A HREF="mailto:david@allinea.com">mailto:david@allinea.com</A>]</FONT>
<BR><FONT SIZE=2>Sent: Monday, October 10, 2005 11:23 AM</FONT>
<BR><FONT SIZE=2>To: openib-general@openib.org</FONT>
<BR><FONT SIZE=2>Subject: ptrace peektext failure for Mellanox IBGD 1.7.0 based cluster</FONT>
</P>
<BR>
<P><FONT SIZE=2>Dear all,</FONT>
</P>
<P><FONT SIZE=2>I'm having a kernel problem which I believe to be caused by the</FONT>
<BR><FONT SIZE=2>infiniband drivers on the system I am using.</FONT>
</P>
<P><FONT SIZE=2>Kernel 2.6.11, Mellanox software stack IBGD 1.7.0.</FONT>
</P>
<P><FONT SIZE=2>Essentially, once an MPI code is started, the kernel refuses to allow</FONT>
<BR><FONT SIZE=2>ptrace() access to the text segment (ie. where the program instructions</FONT>
<BR><FONT SIZE=2>lie), although it is possible to access the data segment.</FONT>
</P>
<P><FONT SIZE=2>This means debugging is impossible (gdb, idb, ddt, etc.).</FONT>
</P>
<P><FONT SIZE=2>The attached code demonstrates the problem.</FONT>
</P>
<P><FONT SIZE=2>Untar, and then make. Run the 'mpi' program, and pick a line of it's</FONT>
<BR><FONT SIZE=2>output, paste into another shell. On the standard, non MPI test code,</FONT>
<BR><FONT SIZE=2>the ptrace reads are all successful. On the MPI test, it gives an error</FONT>
<BR><FONT SIZE=2>for the text segment reads..</FONT>
</P>
<P><FONT SIZE=2>Is this a known issue - are there any upgrades/fixes which should have</FONT>
<BR><FONT SIZE=2>been applied? I would appreciate if someone could run the test</FONT>
<BR><FONT SIZE=2>suggested on a really new setup, and see if the error happens.</FONT>
</P>
<BR>
<P><FONT SIZE=2>Regards</FONT>
<BR><FONT SIZE=2>David</FONT>
<BR><FONT SIZE=2>-- </FONT>
<BR><FONT SIZE=2>David Lecomber, CTO, Allinea Software</FONT>
<BR><FONT SIZE=2>tel: +44 1926 623231 fax: +44 1926 623232</FONT>
</P>
</BODY>
</HTML>