Обсуждение: vacuum and table locking

Hi all,

I have a question regarding vacuum and locking, but since I currently don't have access to my servers (maintenance
activities),I can't verify on my own. I know that VACUUM has at 
least four variants. OK, actually two with ability to add ANALYZE after the operation. Here we go:

VACUUM (VACUUM ANALYZE) - shouldn't need to lock the table(?). I'm looking at the Douglas' book "PostgreSQL" on page
804where it claims the operation does not require exclusive 
access.

VACUUM FULL (VACUUM FULL ANALYZE) - will need to lock the table, but I'm thinking this may only take a couple of
millisecondsas I think this builds a temporary table from the one 
being vacuumed and then when complete, drops the old table and renames the temporary. So the lock should only be during
thismove, or does it lock the table for the entirety? 

Is this correct? And how does it then resolve say another deletion in the table that occurs while the VACUUM is in
operation?I hope that is clearly expressed. 
--
Jay

On 04/03/2015 08:59 AM, John Scalia wrote:
> Hi all,
>
> I have a question regarding vacuum and locking, but since I currently don't have access to my servers (maintenance
activities),I can't verify on my own. I know that VACUUM has at 
> least four variants. OK, actually two with ability to add ANALYZE after the operation. Here we go:
Start here: http://www.postgresql.org/docs/9.4/static/sql-vacuum.html
>
> VACUUM (VACUUM ANALYZE) - shouldn't need to lock the table(?). I'm looking at the Douglas' book "PostgreSQL" on page
804where it claims the operation does not require exclusive 
> access.
True, but the increase in I/O *can* impact overall server operation.
Note that the typical default install includes the autovacuum which will
automatically run the vacuum process when necessary. Vacuuming is a
critical part of maintenance for a number of reasons and not running
vacuum is definitely discouraged. Vacuum essentially marks released
space within the files containing the data as reusable. Vacuum does not
free up disk space.
>
> VACUUM FULL (VACUUM FULL ANALYZE) - will need to lock the table, but I'm thinking this may only take a couple of
millisecondsas I think this builds a temporary table from the one 
> being vacuumed and then when complete, drops the old table and renames the temporary. So the lock should only be
duringthis move, or does it lock the table for the entirety? 
No, it locks for the duration which can potentially be a very long time
as it is physically reshuffling the data to shrink the on-disk space
usage. This is typically reserved for situations that cause excessive
table bloat such as full-table updates or deletes of large portions of
the table *and* when there is insufficient disk-space to use an
alternative method of compacting.

Look also at CLUSTER
(http://www.postgresql.org/docs/9.4/static/sql-cluster.html). Unlike
VACUUM FULL, CLUSTER *does* write the data to a new disk files but is
far faster than VACUUM and can reorder the table data to match an index
which can improve the performance of certain queries. CLUSTER requires
sufficient available disk-space to write the new copy of the table and
associate indexes which can be anywhere from the size of your original
table/indexes if it is already packed to far less if the table is
heavily bloated.

N.B. The operation of CLUSTER and VACUUM have changed from version to
version so read the docs for your version. For example, CLUSTER used to
require an index and would reorder using that index. This requirement
has been dropped in the latest version.

CLUSTER also requires and exclusive lock so...

Check out the pg_reorg extension. This extension does create a new table
on the fly while tracking updates. It then applies any updates that have
occurred while creating the new copy and finally swaps the new version
in place of the old. It creates I/O but doesn't require the
long-duration locks of the other methods.
>
> Is this correct? And how does it then resolve say another deletion in the table that occurs while the VACUUM is in
operation?I hope that is clearly expressed. 
>
Vacuum is just a maintenance operation. Changes to the table will be
dealt with in the next VACUUM.

Note, while you can run VACUUM ANALYZE together, VACUUM and ANALYZE do
different things and can be run independently. While VACUUM deals with
space usage and a few other issues, ANALYZE updates the table statistics
used by the planner.

Cheers,
Steve

On 04/03/2015 09:34 AM, Steve Crawford wrote:
>
> Check out the pg_reorg extension. This extension does create a new
> table on the fly while tracking updates. It then applies any updates
> that have occurred while creating the new copy and finally swaps the
> new version in place of the old. It creates I/O but doesn't require
> the long-duration locks of the other methods.
Actually I should have also said check out pg_repack, the fork of
pg_reorg. In either case also do some reading on potential pitfalls:
http://adpgtech.blogspot.com/2014/10/pgrepack-pitfalls.html

Cheers,
Steve

John Scalia <jayknowsunix@gmail.com> wrote:

> VACUUM (VACUUM ANALYZE) - shouldn't need to lock the table(?).
> I'm looking at the Douglas' book "PostgreSQL" on page 804 where
> it claims the operation does not require exclusive access.

Without the FULL keyword, it only requires a SHARE UPDATE EXCLUSIVE
lock, which does not conflict with ordinary DML.

http://www.postgresql.org/docs/9.2/interactive/explicit-locking.html#LOCKING-TABLES

> VACUUM FULL (VACUUM FULL ANALYZE) - will need to lock the table,

Yes, in this case with an ACCESS EXCLUSIVE lock, which conflicts
with all other locks.

> but I'm thinking this may only take a couple of milliseconds as I
> think this builds a temporary table from the one
> being vacuumed and then when complete, drops the old table and
> renames the temporary. So the lock should only be during this
> move, or does it lock the table for the entirety?

If it only locked while after the data was scanned, how could it
deal with DML affecting portions it had already scanned?  If that
were the only problem it could take a SHARE lock, which would allow
concurrent reads, but then we would have the issue of escalating
the lock to swap in the new heap and indexes -- and lock escalation
carries a risk of deadlock.

> And how does it then resolve say another deletion in the table
> that occurs while the VACUUM is in operation?

A VACUUM FULL could not cope with that, which is why it needs the
ACCESS EXCLUSIVE lock.  A non-FULL VACUUM is OK because of
visibility checking and/or (depending on relation type) a special
form of page locking used just for cleanups.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Apr 3, 2015, at 9:59 AM, John Scalia <jayknowsunix@gmail.com> wrote:
>
> So the lock should only be during this move, or does it lock the table for the entirety?

The entirety. It doesn't know how to track changes that occur during the copying to the new table and apply them after,
soit locks it the whole time. (Just went through this myself...) 

--
Scott Ribe
scott_ribe@elevated-dev.com
http://www.elevated-dev.com/
https://www.linkedin.com/in/scottribe/
(303) 722-0567 voice

<p dir="ltr">If you use the --no-table-lock flag from the contrib stuff, you can alleviate that...<br /><div
class="cm_quote"style=" color: #787878">On Fri, Apr 03, 2015 at 3:27 PM, Scott Ribe <<a
href="mailto:scott_ribe@elevated-dev.com">scott_ribe@elevated-dev.com</a>>wrote:</div><br /><div id="oldcontent"
style="background-color:rgb(255, 255, 255); background-position: initial initial; background-repeat: initial
initial;"><blockquotestyle=""><p dir="ltr">On Apr 3, 2015, at 9:59 AM, John Scalia <jayknowsunix@gmail.com>
wrote:<br /> > <br /> > So the lock should only be during this move, or does it lock the table for the entirety?
<br/><br /> The entirety. It doesn't know how to track changes that occur during the copying to the new table and apply
themafter, so it locks it the whole time. (Just went through this myself...) <br /><br /> -- <br /> Scott Ribe <br />
scott_ribe@elevated-dev.com<br /> http://www.elevated-dev.com/ <br /> https://www.linkedin.com/in/scottribe/ <br />
(303)722-0567 voice <br /><br /><br /><br /><br /><br /><br /><br /> -- <br /> Sent via pgsql-admin mailing list
(pgsql-admin@postgresql.org)<br /> To make changes to your subscription: <br />
http://www.postgresql.org/mailpref/pgsql-admin<br /></blockquote></div>