[ofa-general] Re: [ewg] [PATCH] ofed_scripts: Add location code fix for older ppc64 kernels

[Vladimir Sokolovsky]

Joachim Fenkes wrote:
> Kernels prior to 2.6.24 have problems with multiple devices sharing the same
> location code on ppc64 systems -- only one of these devices would be usable
> by ibmebus. This will be a problem on systems with multiple eHCA chips on a
> single hardware location.
> 
> For older kernels, this problem can be circumvented by, prior to loading the
> eHCA driver, changing the location codes of the offending devices so that
> they're not the same anymore.
> 
> This patch adds an openibd patch file which, if applied, will make openibd
> change the location codes of eHCA adapters with the same location code.
> ofed_patch.sh is changed so that it applies that patch if, and only if, it
> is run on a ppc64 architecture and the kernel version implies that the
> kernel has the ibmebus bug.
> 
> Signed-off-by: Joachim Fenkes <fenkes at de.ibm.com>
> ---
>  ofed_scripts/ofed_patch.sh          |   49 +++++++++++++++++++++++++++++++++++
>  ofed_scripts/openibd-loc_code.patch |   43 ++++++++++++++++++++++++++++++
>  2 files changed, 92 insertions(+), 0 deletions(-)
>  create mode 100644 ofed_scripts/openibd-loc_code.patch
> 
> diff --git a/ofed_scripts/ofed_patch.sh b/ofed_scripts/ofed_patch.sh
> index e1f039d..b254000 100755
> --- a/ofed_scripts/ofed_patch.sh
> +++ b/ofed_scripts/ofed_patch.sh
> @@ -200,6 +200,44 @@ get_backport_dir()
>  
>  }
>  
> +need_openibd_loc_code_patch()
> +{
> +	local sub
> +
> +	if [ "$ARCH" != "ppc64" ]; then
> +		return 1;
> +	fi
> +
> +	case $KVERSION in
> +	2.6.9-*.EL*)
> +		sub=$(echo $KVERSION | cut -d"-" -f2 | cut -d"." -f1)
> +		if [ $sub -lt 62 ]; then
> +			return 0;
> +		fi
> +	;;
> +	2.6.16.*-*-*)
> +		sub=$(echo $KVERSION | cut -d"." -f4 | cut -d"-" -f1)
> +		if [ $sub -lt 53 ]; then
> +			return 0;
> +		fi
> +	;;
> +	2.6.18-*.el5*)
> +		sub=$(echo $KVERSION | cut -d"-" -f2 | cut -d"." -f1)
> +		if [ $sub -lt 52 ]; then
> +			return 0;
> +		fi
> +	;;
> +	2.6.*)
> +		sub=$(echo $KVERSION | cut -d"." -f3 | cut -d"-" -f1 | tr -d [:alpha:][:punct:])
> +		if [ $sub -lt 24 ]; then
> +			return 0;
> +		fi
> +	;;
> +	esac
> +
> +	return 1;
> +}
> +
>  # Apply patch
>  apply_patch()
>  {
> @@ -253,6 +291,13 @@ apply_backport_patches()
>          fi
>  }
>  
> +apply_openibd_patches()
> +{
> +	if need_openibd_loc_code_patch; then
> +		apply_patch ${CWD}/ofed_scripts/openibd-loc_code.patch
> +	fi
> +}
> +
>  # Apply patches
>  patches_handle()
>  {
> @@ -288,6 +333,9 @@ EOF
>                          fi
>                          BACKPORT_INCLUDES='-I${CWD}/kernel_addons/backport/'${BACKPORT_DIR}/include/
>                  fi
> +                
> +                # Apply openibd patches
> +                apply_openibd_patches $KVERSION
>  
>  
>  #FIXME: why are these applied here? Move them to before backports?
> @@ -399,6 +447,7 @@ main()
>  
>  #Set default values
>  KVERSION=${KVERSION:-$(uname -r)}
> +ARCH=${ARCH:-$(uname -m)}
>  WITH_QUILT=${WITH_QUILT:-"yes"}
>  WITH_PATCH=${WITH_PATCH:-"yes"}
>  WITH_KERNEL_FIXES=${WITH_KERNEL_FIXES:-"yes"}
> diff --git a/ofed_scripts/openibd-loc_code.patch b/ofed_scripts/openibd-loc_code.patch
> new file mode 100644
> index 0000000..43d70b4
> --- /dev/null
> +++ b/ofed_scripts/openibd-loc_code.patch
> @@ -0,0 +1,43 @@
> +--- a/ofed_scripts/openibd	2007-10-25 08:01:51.000000000 -0500
> ++++ b/ofed_scripts/openibd	2007-10-27 09:58:56.000000000 -0500
> +@@ -538,6 +538,32 @@ if test -x /sbin/lspci && test -x /sbin/
> + fi
> + }
> + 
> ++fix_location_codes()
> ++{
> ++	# ppc64 only:
> ++	# Fix duplicate location codes on kernels where ibmebus can't handle them
> ++	if [ -d /proc/device-tree -a -f /proc/ppc64/ofdt ]; then
> ++		local i=1 phandle lcode len
> ++		# output all duplicate location codes and their devices
> ++		for attr in $(find /proc/device-tree -wholename "*lhca\@*/ibm,loc-code"); do
> ++			echo -e $(dirname $attr)"\t"$(cat $attr)
> ++		done | sort -k2 | uniq -f1 --all-repeated=separate | cut -f1 | while read dev; do
> ++			if [ -n "$dev" ]; then
> ++				# append an instance counter to the location code
> ++				phandle=$(hexdump -e '8 "%u"' $dev/ibm,phandle)
> ++				lcode=$(cat $dev/ibm,loc-code)-I$i
> ++				len=$(echo -n "$lcode" | wc -c)
> ++				# echo "$dev -> $lcode"
> ++				echo -n "update_property $phandle ibm,loc-code $len $lcode" > /proc/ppc64/ofdt
> ++				i=$(($i + 1))
> ++			else
> ++				# empty line means new group -- reset i
> ++				i=1
> ++			fi
> ++		done
> ++	fi
> ++}
> ++
> + rotate_log()
> + {
> +         local log=$1
> +@@ -694,6 +720,7 @@ start()
> + 
> +     # Load eHCA driver
> +     if [ "X${EHCA_LOAD}" == "Xyes" ]; then
> ++        fix_location_codes
> +         /sbin/modprobe ib_ehca > /dev/null 2>&1
> +         my_rc=$?
> +         if [ $my_rc -ne 0 ]; then


Hi Joachim,
I think that it will be easier to maintain the openibd script if you will change it directly (not via patch applied by ofed_patch.sh).

You can add 'need_openibd_loc_code_patch' function to the openibd with:

KVERSION=`uname -r`
if need_openibd_loc_code_patch; then
	fix_location_codes	
fi


Regards,
Vladimir