Re: issue with gininsert under very high load

Heikki Linnakangas <hlinnakangas@vmware.com> writes:
> Perhaps we should use a lock to enforce that only one process tries to
> clean up the pending list at a time.

Something like the attached?  Can somebody who's seen this problem confirm
this improves matters?

(ginInsertCleanup's header comment also needs to be rewritten, but for
testing purposes, this is fine.)

            regards, tom lane

diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 4a65046..38cda14 100644
*** a/src/backend/access/gin/ginfast.c
--- b/src/backend/access/gin/ginfast.c
***************
*** 21,26 ****
--- 21,27 ----
  #include "access/gin_private.h"
  #include "commands/vacuum.h"
  #include "miscadmin.h"
+ #include "storage/lmgr.h"
  #include "utils/memutils.h"
  #include "utils/rel.h"

*************** ginInsertCleanup(GinState *ginstate,
*** 739,744 ****
--- 740,755 ----
      KeyArray    datums;
      BlockNumber blkno;

+     /*
+      * We use a heavyweight lock on the metapage to ensure that only one
+      * backend at a time tries to clean up the pending list.  While it does
+      * actually work for multiple backends to run this code concurrently, that
+      * turns out to be a bad idea because there's lots of locking conflicts.
+      * So if someone else is already running cleanup, we just do nothing.
+      */
+     if (!ConditionalLockPage(index, GIN_METAPAGE_BLKNO, ExclusiveLock))
+         return;
+
      metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
      LockBuffer(metabuffer, GIN_SHARE);
      metapage = BufferGetPage(metabuffer);
*************** ginInsertCleanup(GinState *ginstate,
*** 748,753 ****
--- 759,765 ----
      {
          /* Nothing to do */
          UnlockReleaseBuffer(metabuffer);
+         UnlockPage(index, GIN_METAPAGE_BLKNO, ExclusiveLock);
          return;
      }

*************** ginInsertCleanup(GinState *ginstate,
*** 925,930 ****
--- 937,944 ----

      ReleaseBuffer(metabuffer);

+     UnlockPage(index, GIN_METAPAGE_BLKNO, ExclusiveLock);
+
      /* Clean up temporary space */
      MemoryContextSwitchTo(oldCtx);
      MemoryContextDelete(opCtx);