[ofw] WDK build environment migration thoughts

Alex Naslednikov alexn at mellanox.co.il
Thu Apr 10 07:09:29 PDT 2008


Hi all,
It's a good idea to clarify some points before announcing Mellanox patch
for WDK porting and __ptr64 problems.
Hope, these explanations will be informative enough and not so long.

1. __ptr64 problem
Briefly speaking, this problem arises when copying 32bit len pointer
into 64bit len pointer. In this case, signed pointer extension will take
place.
How it's applicable to WinOF ?  A lot of pointer were declared to be
__ptr64 (i.e., to be always "long", even in 32bit kernel systems),
that's to preserve on unique size of structs used in IOCTL calls.  The
main problem it will cause is between 32bit user applications and 64bit
kernel application.
When user code do operation like 
s_ptr = &my_struct;
my_type* __ptr64 ptr = s_ptr;
Than kernel will receive ptr with invalid upper bits data (4 bytes FF).
To avoid signed pointer extension, PtrToPtr64() function should be used.
Also, I found some other places where dangerous signed pointer extension
took place, even on 32bit kernel.
Yet another problem that arises with __ptr64 attribute is internal
compiler error (C1001)  in WDK when using __ptr64 pointer to function
(callback)
This problem was described in ofw discussion, you can see also :
http://blogs.msdn.com/texblog/archive/2005/10/31/487436.aspx
http://lists.openfabrics.org/pipermail/ofw/2007-July/001613.html (posted
by Jan from OFW)

Our solution:
1. Initially, we decided to remove all __ptr64 attributes except those
ones inside IOCTL structures. After, put PtrToPtr64() conversion on
every assignment to long pointer.
(like my_type* __ptr64 ptr = PtrToPtr64(s_ptr);  )
During this solution, we changed a huge amount of code, so patch became
unreadable. And it was difficult to validate that all long pointer (with
__ptr64 attribute) were used in a proper manner

2. So, we decided about another solution:
 All __ptr64 occurrences were replaced by either:
 i) TO_LONG_PTR(type, field) macro, when occurred inside structure
ii) VOID_PTR64 macro otherwise (defined as void macro)

#define CONCAT(str1, str2) str1##str2

#define TO_LONG_PTR(type,member_name) \
    union { type member_name;  uint64_t CONCAT(member_name,_padding) ; }

Thus, we can both preserve on a uniform shapes of structs in user and
kernel and to avoid unsafe pointer arithmetic !
The patch now is much more readable, but it sill consist of thousands
lines.

2. Migration to WDK
Main issue here was to preserve on backward compatibility with DDK
We were able to compile our stack with WDK, while the main problems we
found were :

1. WDK uses newer version of SDK (SDK Vista). So, when using 2 or more
versions of SDK on the same build machine, one has to update 
PLATFORM_SDK_PATH variable to point on the proper version of SDK (for
example, PLATFORM_SDK_PATH=%sysdrive%:\PROGRA~1\MI2578~1\windows\v6.1)

2.verify.src script in WDK (new add-on) checks if your SOURCES file is
in appropriate format.
For example, you can't set implicitly path to system .dll in TARGETLIBS,
but to use USE_<MODULE_NAME> =1 macro
Example:
Old code : 
 ....
TARGETLIBS= \
   $(CRT_LIB_PATH)\msvcprt.lib\
   $(SDK_LIB_PATH)\Ws2_32.lib\
   $(TARGETPATH)\*\mtcr.lib
 
New code :
USE_MSVCRT=1
USE_NTDLL=1
 
TARGETLIBS= \
   $(SDK_LIB_PATH)\Ws2_32.lib\
   $(TARGETPATH)\*\mtcr.lib

3. Some other problems, like mulitple includes error in .rc files, or
problem with substituing more than one symbol constant into string in
Makefiles (some version of WDK)


Currently, we continue testing and will advertise these patches right
after the testing will finish

Naslednikov Alexander (a.k.a XaleX)
Windows Team
Mellanox Technologies 


-----Original Message-----
From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Smith, Stan
Sent: Tuesday, April 08, 2008 4:10 PM
To: Ishai Rabinovitz
Cc: ofw at lists.openfabrics.org
Subject: [ofw] WDK build environment migration thoughts

Hello,
  I strongly believe it would help the WinOF community in transitioning
to the WDK build environment if the connectX branch
(svn:gen1\branches\ConnectX) was used as a WDK build environment staging
grounds prior to merging the WDK modifications into the mainline trunk.
This has been talked about before although it still (as of last Friday)
does not build using the latest WDK version.
 
One week prior to merging the WDK fixes into the mainline trunk, if you
were to push all the WDK fixes into the ConnectX branch and then
advertise on the ofw mailing list the availability of a WDK build branch
along with

  1) how to build in the WDK environment,
     which version of the WDK is required + a URL link where to get the
WDK.

  2) An explanation of why and how the __ptr64 attributes were removed
along with how
     others should correct their codes containing __ptr64 attributes.

  3) updates to the WinOF wiki page describing how to build in the WDK
env.

Let this branch exist for one week, receiving feedback from the list and
then merge into the mainline trunk.

Using this approach is certainly community friendly and may prevent
developer surprises.

ConnectX branch availability dates plus when the actual WDK fixes would
be merged into the mainline trunk would be published beforehand.


Thanks for your consideration,

Stan.



_______________________________________________
ofw mailing list
ofw at lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20080410/019b30ce/attachment.html>


More information about the ofw mailing list