[openib-general] FW: SDP problems with 64K page size

[Roland Dreier]

Hi, Jerome asked me to forward this on, since for some reason his
email didn't appear when he sent it.

In any case there seem to be some PAGE_SIZE dependencies in SDP.
Libor provided a patch that fixed this up a while ago, but I don't
know if this is the right way to handle this.

 - R.

From: Jerome Pioux 
To: openib-general at openib.org 
Sent: Wednesday, September 21, 2005 5:36 PM
Subject: SDP problem running SDP on 64k kernel page size


Hi

I tried to run SDP on 2.6.12 kernel but configured with a 64k page size.
I have the Mellanox 3.3.3 rc firmware release (rc7) that allows for this.

It does not work - the error returned to the application is:
105 = No buffer space available

We had similar results with Topspin stack in the past and it was fixed by the 2 patches below.
I am not sure how this patch could be reconstructed in the openIB source and if even possible, but I was hoping that showing it here could be proved useful to someone?

I stand by to provide you with more info (?) if needed...

- Jerome


PS: 
Patches to allow SDP to work on 64k page size kernel (3.3.3 firmware required)

diff -Naur topspin-src-3.0.0-178/host/ulp/sdp/inet/include/sdp_buff_p.h topspin-src-3.0.0-178_bas4v2/host/ulp/sdp/inet/include/sdp_buff_p.h
--- topspin-src-3.0.0-178/host/ulp/sdp/inet/include/sdp_buff_p.h        2005-03-17 21:13:14.000000000 +0100
+++ topspin-src-3.0.0-178_bas4v2/host/ulp/sdp/inet/include/sdp_buff_p.h 2005-07-21 11:55:46.895428699 +0200
@@ -45,6 +45,7 @@
 #define TS_SDP_BUFFER_COUNT_MAX 1048576
 #define TS_SDP_BUFFER_COUNT_INC 128
 #define TS_SDP_BUFFER_FREE_MARK 1024
+#define TS_SDP_BUFFER_SIZE      16384

diff -Naur topspin-src-3.0.0-178/host/ulp/sdp/inet/sdp_buff.c topspin-src-3.0.0-178_bas4v2/host/ulp/sdp/inet/sdp_buff.c
--- topspin-src-3.0.0-178/host/ulp/sdp/inet/sdp_buff.c  2005-03-17 21:13:14.000000000 +0100
+++ topspin-src-3.0.0-178_bas4v2/host/ulp/sdp/inet/sdp_buff.c   2005-07-21 11:55:46.896405262 +0200
@@ -585,7 +585,7 @@
      * buffer descriptor.
      */
     m_pool->buff_cur--;
-    free_page((unsigned long)buff->head);
+       kfree(buff->head);
     kmem_cache_free(m_pool->buff_cache, buff);
   }
   
@@ -652,7 +652,7 @@
       break;
     } /* if */

-    buff->head = (tPTR)__get_free_page(GFP_ATOMIC);
+       buff->head = kmalloc(TS_SDP_BUFFER_SIZE, GFP_ATOMIC);
     if (NULL == buff->head) {

       TS_TRACE(MOD_LNX_SDP, T_TERSE, TRACE_FLOW_FATAL,
@@ -663,7 +663,7 @@
       break;
     } /* if */

-    buff->end     = buff->head + PAGE_SIZE;
+    buff->end     = buff->head + TS_SDP_BUFFER_SIZE;
     buff->data    = buff->head;
     buff->tail    = buff->head;
     buff->lkey    = 0;
@@ -679,7 +679,7 @@
       TS_TRACE(MOD_LNX_SDP, T_TERSE, TRACE_FLOW_FATAL,
               "BUFFER: Failed to insert buffer into pool. <%d>", result);

-      free_page((unsigned long)buff->head);
+      kfree(buff->head);
       kmem_cache_free(m_pool->buff_cache, buff);
       break;
     } /* if */
@@ -742,7 +742,7 @@
 
   memset(main_pool, 0, sizeof(tSDP_MAIN_POOL_STRUCT));

-  main_pool->buff_size = PAGE_SIZE;
+  main_pool->buff_size = TS_SDP_BUFFER_SIZE;
   main_pool->buff_min  = buff_min;
   main_pool->buff_max  = buff_max;
   main_pool->alloc_inc = alloc_inc;