[ofw] [Patch][ipoib][ipoib_NDIS6_CM] Fixing a bug when OID_GEN_NETWORK_LAYER_ADDRESSES contains bad data

Mon Aug 30 01:31:22 PDT 2010

Hi Fab,

After some more testing, we have found that the problem happens on windows 2008 R2, and we have a simple repro on our machines.

What you have to do is go to ipoib adapter , and add 2 ip addresses to the same interface.

Here is a dump of the raw data that we get from NDIS 2008 R2:

4: kd> db 0xfffffa80`0c39f240

fffffa80`0c39f240  02 00 00 00 02 00 10 00-02 00 00 00 10 00 02 00  ................

fffffa80`0c39f250  0c 7f 00 00 10 04 0c 7f-00 00 00 00 00 00 00 00  ................

fffffa80`0c39f260  00 00 00 00 00 00 00 00-00 00 00 00 00 00 70 9f  ..............p.

fffffa80`0c39f270  04 00 08 02 45 76 65 ee-b4 05 00 00 b8 0b 00 00  ....Eve.........

fffffa80`0c39f280  00 00 00 00 70 00 00 00-00 00 00 00 00 00 00 00  ....p...........

fffffa80`0c39f290  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00  ................

fffffa80`0c39f2a0  01 00 00 00 00 00 00 00-01 00 00 00 00 00 00 00  ................

fffffa80`0c39f2b0  00 00 00 00 00 00 00 00-0c 00 08 00 00 00 00 00  ................

As you can see, the first struct starts at:

typedef struct _NETWORK_ADDRESS_LIST {

  LONG  AddressCount;

  USHORT  AddressType;

  NETWORK_ADDRESS  Address[1];

} NETWORK_ADDRESS_LIST, *PNETWORK_ADDRESS_LIST;

Which means that AddressCount==2 AddressType =2

The first Addresses is marked with yellow. The second addresses seems like garbage.

Here is a similar Address struct on 2008

fffffa80`0c005ae0  02 00 00 00 02 00 10 00-02 00 00 00 00 00 0f 04  ................

fffffa80`0c005af0  0c 80 00 00 00 00 00 00-00 00 10 00 02 00 00 00  ................

fffffa80`0c005b00  00 00 10 04 0c 80 00 00-00 00 00 00 00 00 00 00  ................

fffffa80`0c005b10  04 00 08 02 4e 74 66 72-00 00 00 00 00 00 00 00  ....Ntfr........

fffffa80`0c005b20  70 11 a4 0c 80 fa ff ff-b0 7b a3 0c 80 fa ff ff  p........{......

fffffa80`0c005b30  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00  ................

fffffa80`0c005b40  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00  ................

fffffa80

The call stack on windows 2008 R2 is:

4: kd> k 30

Child-SP          RetAddr           Call Site

fffff880`021a6fb0 fffff880`0510b0d6 ipoib!__ipoib_set_net_addr+0xc2 [b:\users\tzachid\mlnx_winof-2_1_2\ulp\ipoib_ndis6_cm\kernel\ipoib_driver.cpp @ 3461]

fffff880`021a7080 fffff880`0510c0af ipoib!ipoib_set_info+0xa46 [b:\users\tzachid\mlnx_winof-2_1_2\ulp\ipoib_ndis6_cm\kernel\ipoib_driver.cpp @ 2951]

fffff880`021a7160 fffff880`012fc92c ipoib!ipoib_oid_handler+0xef [b:\users\tzachid\mlnx_winof-2_1_2\ulp\ipoib_ndis6_cm\kernel\ipoib_driver.cpp @ 3105]

fffff880`021a71a0 fffff880`01278a19 NDIS!ndisMDoOidRequest+0x26c

fffff880`021a7250 fffff880`01278629 NDIS!ndisFQueueRequestOnNext+0x389

fffff880`021a72c0 fffff880`02d1411b NDIS!NdisFOidRequest+0xc9

fffff880`021a73a0 fffff880`012fcd0b pacer!PcFilterRequest+0x5b

fffff880`021a73d0 fffff880`012788bd NDIS!ndisFDoOidRequest+0x21b

fffff880`021a74a0 fffff880`01278629 NDIS!ndisFQueueRequestOnNext+0x22d

fffff880`021a7510 fffff880`02d0b625 NDIS!NdisFOidRequest+0xc9

fffff880`021a75f0 fffff880`012fcd0b wfplwf!FilterOidRequest+0x61

fffff880`021a7620 fffff880`0127822f NDIS!ndisFDoOidRequest+0x21b

fffff880`021a76f0 fffff880`0133328e NDIS!ndisQueueRequestOnTop+0x21f

fffff880`021a7780 fffff880`018590ef NDIS!ndisMOidRequest+0xde

fffff880`021a7870 fffff880`01817108 tcpip!FlpNdisRequestUnderReference+0x9f

fffff880`021a79e0 fffff880`0188a54a tcpip!FlpNdisRequest+0x58

fffff880`021a7a20 fffff880`01811a9a tcpip! ?? ::FNODOBFM::`string'+0x8e50

fffff880`021a7aa0 fffff880`0180ee02 tcpip!IppUpdateFlAddressList+0x5a

fffff880`021a7b10 fffff880`018233c5 tcpip!IppNotifyAddressChangeAtPassive+0x3d2

fffff880`021a7c00 fffff880`0120599b tcpip!IppInterfaceDelayedWorker+0x15

fffff880`021a7c30 fffff800`0197a541 NETIO!NetiopIoWorkItemRoutine+0x3b

fffff880`021a7c80 fffff800`0168e161 nt!IopProcessWorkItem+0x3d

fffff880`021a7cb0 fffff800`01924166 nt!ExpWorkerThread+0x111

fffff880`021a7d40 fffff800`0165f486 nt!PspSystemThreadStartup+0x5a

fffff880`021a7d80 00000000`00000000 nt!KxStartSystemThread+0x16

Please let us know if you need any more information on reproducing the issue.

On a more practical approach (assuming there is no fix in the short term) I suggest that on windows 2008 R2 we will not try to parse this structure if it contains more than one element. Please note that this means that ND will not work on such machines (with more than one ip address per interface).

Thanks

Tzachi and xAlex

> -----Original Message-----

> From: ofw-bounces at lists.openfabrics.org [mailto:ofw-

> bounces at lists.openfabrics.org] On Behalf Of Fab Tillier

> Sent: Wednesday, August 25, 2010 7:26 PM

> To: Alex Naslednikov; Hefty, Sean; ofw at lists.openfabrics.org

> Subject: Re: [ofw] [Patch][ipoib][ipoib_NDIS6_CM] Fixing a bug when

> OID_GEN_NETWORK_LAYER_ADDRESSES contains bad data

>

> What are the repro steps for this?

>

> Thanks,

> -Fab

>

> Alex Naslednikov wrote on Tue, 24 Aug 2010 at 23:06:15

>

> > Yes, we got to the same decision - the NETWORK_ADDRESS array

> contained 5

> > 6-bytes chunks instead of 5 14-bytes chunks. But anyway, this bug at

> > NDIS caused BSOD at ipoib (because of memory violation)

> >

> > -----Original Message-----

> > From: Hefty, Sean [mailto:sean.hefty at intel.com]

> > Sent: Monday, August 23, 2010 7:16 PM

> > To: Alex Naslednikov; ofw at lists.openfabrics.org

> > Subject: RE: [ofw] [Patch][ipoib][ipoib_NDIS6_CM] Fixing a bug when

> > OID_GEN_NETWORK_LAYER_ADDRESSES contains bad data

> >

> >> Fixing the bug when NDIS sends OID_GEN_NETWORK_LAYER_ADDRESSES with

> >> the list of new addresses with invalid formatting (happened when

> >> AddressCount

> >> =5)

> >>

> >> NDIS sends NETWORK_ADDRESS_LIST structure, which contains an array

> of

> >> NETWORK_ADDRESS structures of variable size.

> >> The calculation of the next address offset is based on

> AddressLength;

> >> in a case when this field contains wrong data, one can get access

> >> violation error

> >

> > This sounds like a bug in NDIS

> >

> > _______________________________________________

> > ofw mailing list

> > ofw at lists.openfabrics.org

> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw

> _______________________________________________

> ofw mailing list

> ofw at lists.openfabrics.org

> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20100830/29fa8db3/attachment.html>