<html>
<body>
<font size=3><br>
MPS is the size of transactions on the wire. Max_Read_Request has
nothing to do with what is on the wire. Many devices and chipsets
support up to 4KB for Max_Read_Requests since it enables optimizations to
occur at the chipset / memory controller level. Read
completions are a different matter and as I noted will often come back as
a small number of cache lines up to the MPS value. Some
devices may choose to set the Max_Read_Request to smaller than 4KB in
order to interleave requests in an attempt to improve average latency per
I/O operation at the potential expense of bandwidth. Whether
this has any real impact is unclear since the robustness of chipsets
vary. One might be better off allowing a large
Max_Read_Request and then putting the intelligence to manage request
injection rates and size in the device since chipsets are often designed
/ optimized for high-volume, often simple devices rather than
sophisticated devices such as a HCA or RNIC. Mileage will
vary by where the PCIe Root Complex exists within the system and the
capabilities it provides. <br><br>
Mike<br><br>
At 08:24 AM 3/31/2008, Eli Cohen wrote:<br>
<blockquote type=cite class=cite cite="">This is what I get for ConnectX
on my system:<br>
Device: MaxPayload 256 bytes, MaxReadReq 512 bytes<br><br>
But I think this also has to do with BIOS configuration and you
usually<br>
should not change it. It depends on the chipset and other parameters
of<br>
your system.<br><br>
<br>
On Mon, 2008-03-31 at 13:39 +0200, Koen Segers wrote:<br>
> Hi,<br>
> <br>
> I'm interested in computing the overhead coming from PCI-Express to
an<br>
> IB HCA. Therefore I need to know the payload size of different
types<br>
> of HCA.<br>
> <br>
> We have InfiniHost III Mellanox cards of 4x SDR and DDR IB.
According<br>
> the lspci -vvv command on SLES 10 SP1, the PCI-e payload of
these<br>
> cards is maximum 128 bytes.<br>
> <br>
> Can someone give my the payload size for a ConnectX 4x SDR and DDR
IB<br>
> card? <br>
> <br>
> Thank you,<br>
> <br>
> Koen<br>
> <br>
> This is my output:<br>
> 41:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III
Ex<br>
> (rev a0)<br>
> Subsystem: Mellanox
Technologies MT25208 InfiniHost III Ex<br>
> Control: I/O- Mem+
BusMaster+ SpecCycle- MemWINV- VGASnoop-<br>
> ParErr+ Stepping- SERR- FastB2B-<br>
> Status: Cap+ 66MHz-
UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-<br>
> <TAbort- <MAbort- >SERR- <PERR-<br>
> Latency: 0, Cache
Line Size: 64 bytes<br>
> Interrupt: pin A
routed to IRQ 209<br>
> Region 0: Memory at
e1700000 (64-bit, non-prefetchable)<br>
> [size=1M]<br>
> Region 2: Memory at
e1800000 (64-bit, prefetchable) [size=8M]<br>
> Capabilities: [40]
Power Management version 2<br>
>
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA<br>
> PME(D0-,D1-,D2-,D3hot-,D3cold-)<br>
>
Status: D0 PME-Enable- DSel=0 DScale=0 PME-<br>
> Capabilities: [48]
Vital Product Data<br>
> Capabilities: [90]
Message Signalled Interrupts: Mask- 64bit+<br>
> Queue=0/5 Enable-<br>
>
Address: 0000000000000000 Data: 0000<br>
> Capabilities: [84]
MSI-X: Enable+ Mask- TabSize=32<br>
>
Vector table: BAR=0 offset=00082000<br>
>
PBA: BAR=0 offset=00082200<br>
> Capabilities: [60]
Express Endpoint IRQ 0<br>
>
Device: Supported: MaxPayload 128 bytes, PhantFunc 0,<br>
> ExtTag+<br>
>
Device: Latency L0s <64ns, L1 unlimited<br>
>
Device: AtnBtn- AtnInd- PwrInd-<br>
>
Device: Errors: Correctable- Non-Fatal+ Fatal+<br>
> Unsupported-<br>
>
Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-<br>
>
Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes<br>
>
Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s,<br>
> Port 8<br>
>
Link: Latency L0s unlimited, L1 unlimited<br>
>
Link: ASPM Disabled RCB 128 bytes CommClk- ExtSynch-<br>
>
Link: Speed 2.5Gb/s, Width x8<br>
> <br>
> <br>
> *** Disclaimer ***<br>
> <br>
> Vlaamse Radio- en Televisieomroep<br>
> Auguste Reyerslaan 52, 1043 Brussel<br>
> <br>
> nv van publiek recht<br>
> BTW BE 0244.142.664<br>
> RPR Brussel<br>
>
<a href="http://www.vrt.be/disclaimer" eudora="autourl">
http://www.vrt.be/disclaimer</a> <br>
> _______________________________________________<br>
> general mailing list<br>
> general@lists.openfabrics.org<br>
>
<a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general" eudora="autourl">
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</a><br>
> <br>
> To unsubscribe, please visit
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a><br><br>
_______________________________________________<br>
general mailing list<br>
general@lists.openfabrics.org<br>
<a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general" eudora="autourl">
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</a><br><br>
To unsubscribe, please visit
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a>
</font></blockquote></body>
</html>