[Ofvwg] Further thoughts on uAPI

[Hefty, Sean]

> The kernel common code side is pretty straightforward, just a bunch of
> tables of function pointers, templates and idrs for each object_type.

This is the part where the intent and implementation are not clear to me.  I see value in generic user space handle validation, but I prefer they map to driver specific resources.  For example, I don't want a user space allocation of a QP to result in this:

1403 struct ib_qp {
1404         struct ib_device       *device;
1405         struct ib_pd           *pd;
1406         struct ib_cq           *send_cq;
1407         struct ib_cq           *recv_cq;
1408         struct ib_srq          *srq;
1409         struct ib_xrcd         *xrcd; /* XRC TGT QPs only */
1410         struct list_head        xrcd_list;
1411         /* count times opened, mcast attaches, flow attaches */
1412         atomic_t                usecnt;
1413         struct list_head        open_list;
1414         struct ib_qp           *real_qp;
1415         struct ib_uobject      *uobject;
1416         void                  (*event_handler)(struct ib_event *, void *);
1417         void                   *qp_context;
1418         u32                     qp_num;
1419         enum ib_qp_type         qp_type;
1420 };

monstrosity being allocated in the kernel.  The size and layout of this sort of structure impacts scalability and performance.  This is where I want to make sure there's alignment on the overall architecture.

More specifically, I would switch the existing uABI commands to ioctl's, leaving the code paths mostly unchanged, but marking them as legacy.  Then add the new ioctls that you suggested, with future drivers hooking directly into the ioctl framework.  Honestly, I'm not sure the rdma core should define anything beyond the common ioctl structure.  Anything else can just be placed into the driver specific portion, or maybe rdma core sub-subsystem (e.g. ib, iwarp, roce, opa, mlx, ...)  Maybe we would define some common ioctl's across all devices, but I don't think they would look anything like the current commands.  They would need to be much higher level.

- Sean