Re: Memory leak during delete with sequential scan

On Fri, Sep 12, 2014 at 5:17 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Roman Konoval <rkonoval@gmail.com> writes:
>
>
> > This problem can be reliably reproducible only after restart of postgres.
>
> That sounds suspiciously like what you are counting is a process's
> accesses to shared memory.  You did not say what shared_buffers setting
> you're using, but if the "leak" tops out at something close to your
> shared_buffers setting then that's almost certainly what it is.
> In Linux systems you should generally be looking at RES minus SHR
> not just RES to determine a process' private memory.
>
>

Thanks for your reply, Tom.

Your guess is correct I think.
I'm using default shared_buffers settings. Different version of postgres
have different default shared_buffers setting that's why I get different
results. And the amount of memory I see is very much correlated with
shared_buffers size:

version  shared_buffers  max private_memory
9.1.13     320Mb               260Mb
9.1.14     24Mb                 26Mb
9.3.5       128Mb               128Mb
9.4beta2 128Mb               128Mb

By private memory here I mean the sum of Private_Dirty and Private_Clean
values for every memory segment in /proc/<pid>/smaps. The main portion of
the memory is mapped to segment associated with file /SYSV0052e2c1. I
suppose this is file used by postgres to share between processes.

This is how it looks for one process:
7fa461c56000-7fa46a8e2000 rw-s 00000000 00:04 32768
 /SYSV0052e2c1 (deleted)
Size:             143920 kB
Rss:              131876 kB
Pss:              130077 kB
Shared_Clean:          0 kB
Shared_Dirty:       3188 kB
Private_Clean:         0 kB
Private_Dirty:    128688 kB
Referenced:       131876 kB
Anonymous:             0 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB

When I run several processed executing the same query the memory is moved
to shared:
7fa461c56000-7fa46a8e2000 rw-s 00000000 00:04 32768
 /SYSV0052e2c1 (deleted)
Size:             143920 kB
Rss:              131876 kB
Pss:               65270 kB
Shared_Clean:          0 kB
Shared_Dirty:     131872 kB
Private_Clean:         0 kB
Private_Dirty:         4 kB
Referenced:       130880 kB
Anonymous:             0 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB

So I was confusing private memory of the process with the portion of shared
memory modified by this process alone.

My understanding now is that private memory of the process is only the one
associated with heap and stack mappings:

7fa470bbf000-7fa470c93000 rw-p 00000000 00:00 0
 [heap]
Size:                848 kB
Rss:                 648 kB
Pss:                 648 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:       648 kB
Referenced:          648 kB
Anonymous:           648 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
VmFlags: rd wr mr mw me ac
7fffae0f3000-7fffae122000 rw-p 00000000 00:00 0
 [stack]
Size:                192 kB
Rss:                 148 kB
Pss:                  38 kB
Shared_Clean:          0 kB
Shared_Dirty:        128 kB
Private_Clean:         0 kB
Private_Dirty:        20 kB
Referenced:           20 kB
Anonymous:           148 kB
AnonHugePages:         0 kB
Swap:                  0 kB


KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB


                        regards, tom lane
>