[openib-general] Some Missing Features from mthca/user MAD access

[shaharf]

> > After consideration, I think the proper way is add a reference count
> > and clean is_sm when it falls to 0.
> > This way runnning two opensms on the same HCA and two different
> > HCAs in the same subnet behaves the same.
> 
> I don't understand this.  IsSM is per-port, so what does it mean for
the
> reference count of IsSM for a port to be 2?  Two SMs are running on
that
> port?

I totally agree. I see no point running two SMs on the same port. Not
even failover - you have to many shared singe points of failure in such
case.

> 
> > To do this, I think either and ioctl on umad, or
> > a flag to open to set this bit is not too bad though.
> 
> I don't like either the ioctl() or a magic flag for open().  I'm
> trying to decide
> between a special issm file, or changing the umad file interface to
put a
> command code in the structure that is passed via write() (this would
also
> let us add a mechanism for canceling MADs).
> 

When did ioctl go out of fashion? What is the big difference between
ioctl and an embedded field in the umad structure? Both are binary
parameters.
Personally I think that non binary format is better when applicable -
this mean issm file (you can say it is Plan9 style...). It would not
necessarily use the open method. You may want to use the write method
and to enable everyone read the file. This read may return the pid of
the owner (== the first to write something into the file). 

The cancel mad operation is less important than I thought. The OpenSM
manage a send queue and let only a specific window of packets to reach
the kernel. Unless you allow the SM to use very large window, most
canceled mads will be in the internal SM queue. The default window is 1
(should be changed). In short, you don't have to take the cancel mad
into account. Canceling any pending mads when the file is closed should
be enough.
 
Shahar