[openib-general] problem with 2.6.19?

[Steve Wise]

Also, if I drop the memory down to 1G (via mem=1G on the boot line),
then things work.  But I think I'm just disabling the IOMMU.  Note that
__pa() and dma_map_single() return the same thing:


c2_alloc_mqsp_chunk mqsp_chunk va ffff81003c4d2000 dma_addr 3c4d2000 __pa 3c4d2000
c2_alloc_mqsp addr ffff81003c4d201a dma_addr 3c4d201a
c2_alloc_mqsp addr ffff81003c4d201c dma_addr 3c4d201c
c2_alloc_mqsp addr ffff81003c4d201e dma_addr 3c4d201e
c2_alloc_mqsp addr ffff81003c4d2020 dma_addr 3c4d2020
c2_rnic_init rep_vq va ffff81003cb08000 dma 3cb08000 __pa 3cb08000
c2_rnic_init aeq va ffff81003d4e0000 dma 3d4e0000 __pa 3d4e0000



On Fri, 2006-10-27 at 09:43 -0500, Steve Wise wrote:
> On Thu, 2006-10-26 at 15:26 -0700, Roland Dreier wrote:
> >     Steve> The adapter seems to be dma'ing into the wrong memory.  The
> >     Steve> patch below backs the usage of dma_map_single() back to
> >     Steve> using __pa() for converting kernel virtual addresses (from
> >     Steve> kmalloc) into bus addresses, and things work ok.
> > 
> > Hmm.  It might be interesting to hack the driver to print the result
> > of both dma_map_single() and __pa() and see if they're different.
> > 
> 
> Here's a dump.  They are different.  The __pa()'s are what I expect.  I
> dunno if the dma_addr's returned from dma_map_single() look good or not
> (they certainly don't work :)
> 
> c2_alloc_mqsp_chunk mqsp_chunk va ffff810148b1b000 dma_addr 3a56000 __pa 148b1b000
> c2_alloc_mqsp addr ffff810148b1b01a dma_addr 3a5601a
> c2_alloc_mqsp addr ffff810148b1b01c dma_addr 3a5601c
> c2_alloc_mqsp addr ffff810148b1b01e dma_addr 3a5601e
> c2_alloc_mqsp addr ffff810148b1b020 dma_addr 3a56020
> c2_rnic_init rep_vq va ffff810147e78000 dma 3a57000 __pa 147e78000
> c2_rnic_init aeq va ffff810147d48000 dma 3a5f000 __pa 147d48000
> 
> > Are you running on a 32-bit (i386) or 64-bit (x86_64) kernel?  How
> > much RAM do you have?  
> 
> 64b/X86_64.  4GB RAM.  The CPUs are Dempsey class XEONs - Dual CPU, Dual
> core.  So with HT on linux sees 8 CPUs.
> 
> > Is the kernel using swiotlb?  If so then you
> > need to make sure your DMA_{TO,FROM} directions and dma_unmap calls
> > are right, since otherwise the DMAed data won't be copied to/from the
> > bounce buffer at the right time.
> 
> All these mappings are for the device to DMA into the host memory, and
> I'm using DMA_FROM_DEVICE in my calls to dma_map_single().
> 
> How do I know if the kernel is using swiotlb?
> 
> 
> > Another thing to do if you're patient would be to use git-bisect and
> > figure out exactly which patch made amso1100 stop working.
> > 
> 
> I added these calls as part of the review for submission into the
> kernel, and I originally tested them on dual CPU opteron systems with
> 1GB of memory.  But maybe they weren't using the IOMMU?  Dunno.
> 
> 
> 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>