[openib-general] Re: [PATCH 2/6] [RFC] mthca kernel changes for resizeCQ

[Roland Dreier]

  > static inline int ib_poll_cq(struct ib_cq *cq, int num_entries,
  >                              struct ib_wc *wc)
  > {
  > 	unsigned long flags;
  > 	int ret;
  > 	spin_lock_irqsave(cq->lock, flags);
  >         ret = cq->poll_cq(cq, num_entries, wc);
  > 	spin_unlock_irqrestore(cq->lock, flags);
  > 	return ret;

    Caitlin> This is not an issue of a nebulous future change. This is
    Caitlin> a matter of one device trying to force future devices to
    Caitlin> have the same limitation.

Well, all four devices for which we have drivers (mthca, ipath, ehca,
amso1100) all take a lock while polling CQs.  And that's all the
devices for which we have driver code, as far as I know.  So yes, as
far as I can tell planning for lockless CQ polling is purely
hypothetical.

    Caitlin> Devices that do not require a host-wide spinlock to poll
    Caitlin> the CQ are not hypothetical, they exist even if drivers
    Caitlin> under OpenIB for them do not yet exist. Not planning for
    Caitlin> the needs of such drivers will not help the process of
    Caitlin> making those drivers exist.

The code above is not taking a host-wide lock -- it is taking the
per-CQ lock "cq->lock".  Do you know of a device that permits lockless
CQ polling?  If so a pointer to the driver code or at least a
description of how consistency is maintained without locks would be
interesting.

 - R.