[openib-general] problem with 2.6.19?

[Steve Wise]

And I do have CONFIG_SWIOTLB set:

CONFIG_SWIOTLB=y

On Fri, 2006-10-27 at 10:48 -0500, Steve Wise wrote:
> Also, if I drop the memory down to 1G (via mem=1G on the boot line),
> then things work.  But I think I'm just disabling the IOMMU.  Note that
> __pa() and dma_map_single() return the same thing:
> 
> 
> c2_alloc_mqsp_chunk mqsp_chunk va ffff81003c4d2000 dma_addr 3c4d2000 __pa 3c4d2000
> c2_alloc_mqsp addr ffff81003c4d201a dma_addr 3c4d201a
> c2_alloc_mqsp addr ffff81003c4d201c dma_addr 3c4d201c
> c2_alloc_mqsp addr ffff81003c4d201e dma_addr 3c4d201e
> c2_alloc_mqsp addr ffff81003c4d2020 dma_addr 3c4d2020
> c2_rnic_init rep_vq va ffff81003cb08000 dma 3cb08000 __pa 3cb08000
> c2_rnic_init aeq va ffff81003d4e0000 dma 3d4e0000 __pa 3d4e0000
> 
> 
> 
> On Fri, 2006-10-27 at 09:43 -0500, Steve Wise wrote:
> > On Thu, 2006-10-26 at 15:26 -0700, Roland Dreier wrote:
> > >     Steve> The adapter seems to be dma'ing into the wrong memory.  The
> > >     Steve> patch below backs the usage of dma_map_single() back to
> > >     Steve> using __pa() for converting kernel virtual addresses (from
> > >     Steve> kmalloc) into bus addresses, and things work ok.
> > > 
> > > Hmm.  It might be interesting to hack the driver to print the result
> > > of both dma_map_single() and __pa() and see if they're different.
> > > 
> > 
> > Here's a dump.  They are different.  The __pa()'s are what I expect.  I
> > dunno if the dma_addr's returned from dma_map_single() look good or not
> > (they certainly don't work :)
> > 
> > c2_alloc_mqsp_chunk mqsp_chunk va ffff810148b1b000 dma_addr 3a56000 __pa 148b1b000
> > c2_alloc_mqsp addr ffff810148b1b01a dma_addr 3a5601a
> > c2_alloc_mqsp addr ffff810148b1b01c dma_addr 3a5601c
> > c2_alloc_mqsp addr ffff810148b1b01e dma_addr 3a5601e
> > c2_alloc_mqsp addr ffff810148b1b020 dma_addr 3a56020
> > c2_rnic_init rep_vq va ffff810147e78000 dma 3a57000 __pa 147e78000
> > c2_rnic_init aeq va ffff810147d48000 dma 3a5f000 __pa 147d48000
> > 
> > > Are you running on a 32-bit (i386) or 64-bit (x86_64) kernel?  How
> > > much RAM do you have?  
> > 
> > 64b/X86_64.  4GB RAM.  The CPUs are Dempsey class XEONs - Dual CPU, Dual
> > core.  So with HT on linux sees 8 CPUs.
> > 
> > > Is the kernel using swiotlb?  If so then you
> > > need to make sure your DMA_{TO,FROM} directions and dma_unmap calls
> > > are right, since otherwise the DMAed data won't be copied to/from the
> > > bounce buffer at the right time.
> > 
> > All these mappings are for the device to DMA into the host memory, and
> > I'm using DMA_FROM_DEVICE in my calls to dma_map_single().
> > 
> > How do I know if the kernel is using swiotlb?
> > 
> > 
> > > Another thing to do if you're patient would be to use git-bisect and
> > > figure out exactly which patch made amso1100 stop working.
> > > 
> > 
> > I added these calls as part of the review for submission into the
> > kernel, and I originally tested them on dual CPU opteron systems with
> > 1GB of memory.  But maybe they weren't using the IOMMU?  Dunno.
> > 
> > 
> > 
> > 
> > _______________________________________________
> > openib-general mailing list
> > openib-general at openib.org
> > http://openib.org/mailman/listinfo/openib-general
> > 
> > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
> > 
> 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>