== PostgreSQL Weekly News - April 12, 2020 ==
| От | David Fetter |
|---|---|
| Тема | == PostgreSQL Weekly News - April 12, 2020 == |
| Дата | |
| Msg-id | 20200412204846.GA17695@fetter.org обсуждение исходный текст |
| Список | pgsql-announce |
== PostgreSQL Weekly News - April 12, 2020 == There is now a "common" yum repository for those things which do not depend directly on specific versions of PostgreSQL to operate. This reduces build times for those that do so depend by factoring out the ones that don't. == PostgreSQL Product News == psycopg2 2.8.5, a Python connector for PostgreSQL, released. http://initd.org/psycopg/articles/2020/04/06/psycopg-285-released/ == PostgreSQL Local == PGCon 2020 will take place online on May 26-29, 2020. https://www.pgcon.org/2020/ PostgresLondon 2020 will be July 7-8, 2020 with an optional training day on July 6. http://postgreslondon.org PG Day Russia will be in Saint Petersburg on July 10, 2020. https://pgday.ru/en/2020/ FOSS4G 2020, will take place in Calgary, Alberta, Canada August 24-29 2020. the Call for Papers is currently open at https://2020.foss4g.org/speakers/ https://2020.foss4g.org/ PGDay Ukraine will take place September 5th, 2020 in Lviv at the Bank Hotel. https://pgday.org.ua/ pgDay Israel 2020 will take place on September 10, 2020 in Tel Aviv. http://pgday.org.il/ PGDay Austria will take place September 18, 2020 at Schloss Schoenbrunn (Apothekertrakt) in Vienna. The CfP is open until April 19, 2020 at https://pgday.at/en/talk-commitee/ https://pgday.at/en/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org. == Applied Patches == Andres Freund pushed: - Fix recently introduced typo. Reported-By: David Rowley https://git.postgresql.org/pg/commitdiff/549a3e23c3d618103487161b19dbbf8fd2206a5c - Use TransactionXmin instead of RecentGlobalXmin in heap_abort_speculative(). There's a very low risk that RecentGlobalXmin could be far enough in the past to be older than relfrozenxid, or even wrapped around. Luckily the consequences of that having happened wouldn't be too bad - the page wouldn't be pruned for a while. Avoid that risk by using TransactionXmin instead. As that's announced via MyPgXact->xmin, it is protected against wrapping around (see code comments for details around relfrozenxid). Author: Andres Freund Discussion: https://postgr.es/m/20200328213023.s4eyijhdosuc4vcj@alap3.anarazel.de Backpatch: 9.5- https://git.postgresql.org/pg/commitdiff/f946069e6827e729857b9f2db06bf27a1c0563ee - Recompute stack base in forked postmaster children. This is for the benefit of running postgres under the rr debugger. When using rr signal handlers running while a syscall is active use an alternative stack. As e.g. bgworkers are started from within signal handlers, the forked backend then has a different stack base than postmaster. Previously that subsequently lead to those processes triggering spurious "stack depth limit exceeded" errors. Discussion: https://postgr.es/m/20200327182217.ubrrl32lyfhxfwk5@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/fc3f4453a2bc95549682e23600b22e658cb2d6d7 - Fix XLogReader FD leak that makes backends unusable after 2PC usage. Before the fix every 2PC commit/abort leaked a file descriptor. As the files are opened using BasicOpenFile(), that quickly leads to the backend running out of file descriptors. Once enough 2PC abort/commit have caused enough FDs to leak, any IO in the backend will fail with "Too many open files", as BasicOpenFilePerm() will have triggered all open files known to fd.c to be closed. The leak causing the problem at hand is a consequence of 0dc8ead46, but is only exascerbated by it. Previously most XLogPageReadCB callbacks used static variables to cache one open file, but after the commit the cache is private to each XLogReader instance. There never was infrastructure to close FDs at the time of XLogReaderFree, but the way XLogReader was used limited the leak to one FD. This commit just closes the during XLogReaderFree() if the FD is stored in XLogReaderState.seg.ws_segno. This may not be the way to solve this medium/long term, but at least unbreaks 2PC. Discussion: https://postgr.es/m/20200406025651.fpzdb5yyb7qyhqko@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/91c40548d5f7a9241d9fa344fae8069cfdb92bf2 - snapshot scalability: Move delayChkpt from PGXACT to PGPROC. The goal of separating hotly accessed per-backend data from PGPROC into PGXACT is to make accesses fast (GetSnapshotData() in particular). But delayChkpt is not actually accessed frequently; only when starting a checkpoint. As it is frequently modified (multiple times in the course of a single transaction), storing it in the same cacheline as hotly accessed data unnecessarily dirties a contended cacheline. Therefore move delayChkpt to PGPROC. This is part of a larger series of patches intending to improve GetSnapshotData() scalability. It is committed and pushed separately, as it is independently beneficial (small but measurable win, limited by the other frequent modifications of PGXACT). Author: Andres Freund Reviewed-By: Robert Haas, Thomas Munro, David Rowley Discussion: https://postgr.es/m/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/75848bc74411130ede23995d0ab1aefb12c4c4b0 Michaël Paquier pushed: - Preserve clustered index after rewrites with ALTER TABLE. A table rewritten by ALTER TABLE would lose tracking of an index usable for CLUSTER. This setting is tracked by pg_index.indisclustered and is controlled by ALTER TABLE, so some extra work was needed to restore it properly. Note that ALTER TABLE only marks the index that can be used for clustering, and does not do the actual operation. Author: Amit Langote, Justin Pryzby Reviewed-by: Ibrar Ahmed, Michael Paquier Discussion: https://postgr.es/m/20200202161718.GI13621@telsasoft.com Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/a40caf5f862ca8b7e927b2ab2567e934868e9376 - Refactor cluster.c to use new routine get_index_isclustered(). This new cache lookup routine has been introduced in a40caf5, and more code paths can directly use it. Note that in cluster_rel(), the code was returning immediately if the tuple's entry in pg_index for the clustered index was not valid. This commit changes the code so as a lookup error is raised instead, something that could not happen from the start as we check for the existence of the index beforehand, while holding an exclusive lock on the parent table. Author: Justin Pryzby Reviewed-by: Álvaro Herrera, Michael Paquier Discussion: https://postgr.es/m/20200202161718.GI13621@telsasoft.com https://git.postgresql.org/pg/commitdiff/8ef9451f58ee92d1cdb910e72010dbe75e76f9b8 - Fix crash when using COLLATE in partition bound expressions. Attempting to use a COLLATE clause with a type that it not collatable in a partition bound expression could crash the server. This commit fixes the code by adding more checks similar to what is done when computing index or partition attributes by making sure that there is a collation iff the type is collatable. Backpatch down to 12, as 7c079d7 introduced this problem. Reported-by: Alexander Lakhin Author: Dmitry Dolgov Discussion: https://postgr.es/m/16325-809194cf742313ab@postgresql.org Backpatch-through: 12 https://git.postgresql.org/pg/commitdiff/c0187869a0f6eb05135d388462522a593ced1b88 - Fix collection of typos and grammar mistakes in the tree. This fixes some comments and documentation new as of Postgres 13. Author: Justin Pryzby Discussion: https://postgr.es/m/20200408165653.GF2228@telsasoft.com https://git.postgresql.org/pg/commitdiff/dd0f37eccecc2db5c0ffafefbf697a2c916e8bc3 Amit Kapila pushed: - Add the option to report WAL usage in EXPLAIN and auto_explain. This commit adds a new option WAL similar to existing option BUFFERS in the EXPLAIN command. This option allows to include information on WAL record generation added by commit df3b181499 in EXPLAIN output. This also allows the WAL usage information to be displayed via the auto_explain module. A new parameter auto_explain.log_wal controls whether WAL usage statistics are printed when an execution plan is logged. This parameter has no effect unless auto_explain.log_analyze is enabled. Author: Julien Rouhaud Reviewed-by: Dilip Kumar and Amit Kapila Discussion: https://postgr.es/m/CAB-hujrP8ZfUkvL5OYETipQwA=e3n7oqHFU=4ZLxWS_Cza3kQQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/33e05f89c53e5a1533d624046bb6fb0da7bb7141 - Allow autovacuum to log WAL usage statistics. This commit allows autovacuum to log WAL usage statistics added by commit df3b181499. Author: Julien Rouhaud Reviewed-by: Dilip Kumar and Amit Kapila Discussion: https://postgr.es/m/CAB-hujrP8ZfUkvL5OYETipQwA=e3n7oqHFU=4ZLxWS_Cza3kQQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/b7ce6de93b59852c55d09acdaeebbf5aaf89114e - Allow parallel create index to accumulate buffer usage stats. Currently, we don't account for buffer usage incurred by parallel workers for parallel create index. This commit allows each worker to record the buffer usage stats and leader backend to accumulate that stats at the end of the operation. This will allow pg_stat_statements to display correct buffer usage stats for (parallel) create index command. Reported-by: Julien Rouhaud Author: Sawada Masahiko Reviewed-by: Dilip Kumar, Julien Rouhaud and Amit Kapila Backpatch-through: 11, where this was introduced Discussion: https://postgr.es/m/20200328151721.GB12854@nol https://git.postgresql.org/pg/commitdiff/5c71362174eb56676f8b91c73ec066dd5513fd4b Peter Eisentraut pushed: - Add logical replication support to replicate into partitioned tables. Mainly, this adds support code in logical/worker.c for applying replicated operations whose target is a partitioned table to its relevant partitions. Author: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Rafia Sabih <rafia.pghackers@gmail.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com> Reviewed-by: Petr Jelinek <petr@2ndquadrant.com> Discussion: https://www.postgresql.org/message-id/flat/CA+HiwqH=Y85vRK3mOdjEkqFK+E=ST=eQiHdpj43L=_eJMOOznQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/f1ac27bfda6ce8a399d8001843e9aefff5814f9b - Allow publishing partition changes via ancestors. To control whether partition changes are replicated using their own identity and schema or an ancestor's, add a new parameter that can be set per publication named 'publish_via_partition_root'. This allows replicating a partitioned table into a different partition structure on the subscriber. Author: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Rafia Sabih <rafia.pghackers@gmail.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com> Reviewed-by: Petr Jelinek <petr@2ndquadrant.com> Discussion: https://www.postgresql.org/message-id/flat/CA+HiwqH=Y85vRK3mOdjEkqFK+E=ST=eQiHdpj43L=_eJMOOznQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/83fd4532a72179c370e318075a10e0e2aa832024 - createuser: Change a fprintf to pg_log_error. https://git.postgresql.org/pg/commitdiff/f45b8e51b6838ab820df13983c194f737be48778 - Fix CREATE TABLE LIKE INCLUDING GENERATED column order issue. CREATE TABLE LIKE INCLUDING GENERATED would fail if a generated column referred to a column with a higher attribute number. This is because the column mapping mechanism created the mapping incrementally as columns are added. This was sufficient for previous uses of that mechanism (omitting dropped columns), and it also happened to work if generated columns only referred to columns with lower attribute numbers, but here it failed. This fix is to build the attribute mapping in a separate loop before processing the columns in detail. Bug: #16342 Reported-by: Ethan Waldo <ewaldo@healthetechs.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> https://git.postgresql.org/pg/commitdiff/e92e4a2b68fe877677278c1300db1780956c984e - Fix relcache reference leak. Introduced by 83fd4532a72179c370e318075a10e0e2aa832024 https://git.postgresql.org/pg/commitdiff/5a1d0c9925fbda9ec434061dee74b1b5d9a16038 - Fix RELCACHE_FORCE_RELEASE issue. Introduced by 83fd4532a72179c370e318075a10e0e2aa832024. To fix, the tuple descriptors need to be copied into the current memory context. Discussion: https://www.postgresql.org/message-id/04d78603-edae-9243-9dde-fe3037176a7d@2ndquadrant.com https://git.postgresql.org/pg/commitdiff/12fb189bfeaf03faea5b6f15544c1f71e9ef4677 Tom Lane pushed: - Re-stabilize infinite_recurse() test case. Since commit 8f59f6b9c0, CLOBBER_CACHE_ALWAYS buildfarm members have been failing this test case because the error message now sometimes includes an error cursor position. It seems largely just luck that that never happened before, and there are likely to be more ways it could happen in future. Hence, rather than trying to prevent it, adjust the test script to suppress that component of the report. At some point we might need to back-patch this, but refrain until there's a demonstrated need. (We'd need a different fix before v12, anyway, since VERBOSITY=sqlstate is a recent thing.) Tom Lane and Andres Freund Discussion: https://postgr.es/m/30675.1586111599@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/3c8553547b1493c4afdb80393f4a47dbfa019a79 - Fix representation of SORT_TYPE_STILL_IN_PROGRESS. It turns out that the code did indeed rely on a zeroed TuplesortInstrumentation.sortMethod field to indicate "this worker never did anything", although it seems the issue only comes up during certain race-condition-y cases. Hence, rearrange the TuplesortMethod enum to restore SORT_TYPE_STILL_IN_PROGRESS to having the value zero, and add some comments reinforcing that that isn't optional. Also future-proof a loop over the possible values of the enum. sizeof(bits32) happened to be the correct limit value, but only by purest coincidence. Per buildfarm and local investigation. Discussion: https://postgr.es/m/12222.1586223974@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/c7654f6a37792ab9525ff98b710c23b27c7868a5 - Adjust bytea get_bit/set_bit to use int8 not int4 for bit numbering. Since the existing bit number argument can't exceed INT32_MAX, it's not possible for these functions to manipulate bits beyond the first 256MB of a bytea value. Lift that restriction by redeclaring the bit number arguments as int8 (which requires a catversion bump, hence is not back-patchable). The similarly-named functions for bit/varbit don't really have a problem because we restrict those types to at most VARBITMAXLEN bits; hence leave them alone. While here, extend the encode/decode functions in utils/adt/encode.c to allow dealing with values wider than 1GB. This is not a live bug or restriction in current usage, because no input could be more than 1GB, and since none of the encoders can expand a string more than 4X, the result size couldn't overflow uint32. But it might be desirable to support more in future, so make the input length values size_t and the potential-output-length values uint64. Also add some test cases to improve the miserable code coverage of these functions. Movead Li, editorialized some by me; also reviewed by Ashutosh Bapat Discussion: https://postgr.es/m/20200312115135445367128@highgo.ca https://git.postgresql.org/pg/commitdiff/26a944cf29ba67bb49f42656dd2be98fe2485f5f - Allow psql's \g and \gx commands to transiently change \pset options. We invented \gx to allow the "\pset expanded" flag to be forced on for the duration of one command output, but that turns out to not be nearly enough to satisfy the demand for variant output formats. Hence, make it possible to change any pset option(s) for the duration of a single command output, by writing "option=value ..." inside parentheses, for example \g (format=csv csv_fieldsep='\t') somefile \gx can now be understood as a shorthand for including expanded=on inside the parentheses. Patch by me, expanding on a proposal by Pavel Stehule Discussion: https://postgr.es/m/CAFj8pRBx9OnBPRJVtfA5ycUpySge-XootAXAsv_4rrkHxJ8eRg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/b63c293bcbd7439f883cd4cf748f6755df0fbb3c - Fix circle_in to accept "(x,y),r" as it's advertised to do. Our documentation describes four allowed input syntaxes for circles, but the regression tests tried only three ... with predictable consequences. Remarkably, this has been wrong since the circle datatype was added in 1997, but nobody noticed till now. David Zhang, with some help from me Discussion: https://postgr.es/m/332c47fa-d951-7574-b5cc-a8f7f7201202@highgo.ca https://git.postgresql.org/pg/commitdiff/41a194f49177daf9348bfde2c42e85b806dcee31 - Allow partitionwise join to handle nested FULL JOIN USING cases. This case didn't work because columns merged by FULL JOIN USING are represented in the parse tree by COALESCE expressions, and the logic for recognizing a partitionable join failed to match upper-level join clauses to such expressions. To fix, synthesize suitable COALESCE expressions and add them to the nullable_partexprs lists. This is pretty ugly and brute-force, but it gets the job done. (I have ambitions of rethinking the way outer-join output Vars are represented, so maybe that will provide a cleaner solution someday. For now, do this.) Amit Langote, reviewed by Justin Pryzby, Richard Guo, and myself Discussion: https://postgr.es/m/CA+HiwqG2WVUGmLJqtR0tPFhniO=H=9qQ+Z3L_ZC+Y3-EVQHFGg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/981643dcdb70b6ce70d8a08417f71f465f236cb5 - Put back mistakenly removed #include. In commit 4dbcb3f84 I removed some code from parse_coerce.c, and also removed some apparently-no-longer-needed #includes. But removing datum.h broke some not-compiled-by-default code. Discussion: https://postgr.es/m/20200407205436.pyjhddw5bn5upvsu@development https://git.postgresql.org/pg/commitdiff/7a5d74b7dd4b1324e8a8fff4086d51d4f43b1721 - Fix pg_dump/pg_restore to restore event trigger comments later. Repair an oversight in commit 8728b2c70: if we're postponing restore of event triggers to the end, we must also postpone restoring any comments on them, since of course we cannot create the comments first. (This opens yet another opportunity for an event trigger to bollix the restore, but there's no help for that.) Per bug #16346 from Alexander Lakhin. Like the previous commit, back-patch to all supported branches. Hamid Akhtar and Tom Lane Discussion: https://postgr.es/m/16346-6210ad7a0ea81be1@postgresql.org https://git.postgresql.org/pg/commitdiff/a9d70c108786712a1023c65e360602edf7bafbf4 - Cosmetic improvements for default text search parser's ts_headline code. This code was woefully unreadable and under-commented. Try to improve matters by adding comments, using some macros to make complicated if-tests more readable, using boolean type where appropriate, etc. There are a couple of tiny coding improvements too, but this commit includes (I hope) no behavioral change. Nonetheless, back-patch as far as 9.6, because a followup bug-fixing commit depends on this. Discussion: https://postgr.es/m/16345-2e0cf5cddbdcd3b4@postgresql.org https://git.postgresql.org/pg/commitdiff/b10f8bb9fd39da44efd411aec5c643b92bd23676 - Fix default text search parser's ts_headline code for phrase queries. This code could produce very poor results when asked to highlight a string based on a query using phrase-match operators. The root cause is that hlCover(), which is supposed to find a minimal substring that matches the query, was written assuming that word position is not significant. I'm only 95% convinced that its algorithm was correct even for plain AND/OR queries; but it definitely fails completely for phrase matches, causing it to possibly not identify a cover string at all. Hence, rewrite hlCover() with a less-tense algorithm that just tries all the possible substrings, earlier and shorter ones first. (This is not as bad as it sounds performance-wise, because all of the string matching has been done already: the repeated tsquery match checks boil down to pointer comparisons.) Unfortunately, since that approach produces more candidate cover strings than before, it also exposes that there were bugs in the heuristics in mark_hl_words() for selecting a best cover string. Fixes there include: * Do not apply the ShortWord filter to words that appear in the query. * Remove a misguided optimization for quickly rejecting a cover. * Fix order-of-operation bug that could cause computation of a wrong figure of merit (poslen) when shortening a cover. * Change the preference rule so that candidate headlines that do not include their whole cover string (after MaxWords trimming) are lowest priority, since they may not actually satisfy the user's query. This results in some changes in existing regression test cases, but they all seem reasonable. Note in particular that the tests involving strings like "1 2 3" were previously being affected by the ShortWord filter, masking the normal matching behavior. Per bug #16345 from Augustinas Jokubauskas; the new test cases are based on that example. Back-patch to 9.6 where phrase search was added to tsquery. Discussion: https://postgr.es/m/16345-2e0cf5cddbdcd3b4@postgresql.org https://git.postgresql.org/pg/commitdiff/c9b0c678d30aa3f6bbf996ea15af8acbfcfb2ac8 - Doc: improve documentation about ts_headline() function. Now that I've had my nose in that code, I thought the docs about it left something to be desired. https://git.postgresql.org/pg/commitdiff/a4d4f59196ea8745fe4c048085d6d2bd66e8e7d8 - Further cleanup of ts_headline code. Suppress a probably-meaningless uninitialized-variable warning (induced by my previous patch, I'm sorry to say). Improve mark_hl_fragments()'s test for overlapping cover strings: it failed to consider the possibility that the current string is strictly within another one. That's unlikely given the preceding splitting into MaxWords fragments, but I don't think it's impossible. Discussion: https://postgr.es/m/16345-2e0cf5cddbdcd3b4@postgresql.org https://git.postgresql.org/pg/commitdiff/2e0e409e3cbab4f4ac01a6f70931817cfd2bdcb1 - Further stabilize results of 019_replslot_limit.pl. Depending on specific values of restart_lsn or pg_current_wal_lsn() is obviously unsafe. The previous coding tried to dodge this issue by rounding off, but that's not good enough, as shown by multiple buildfarm members. Nuke all the uses of these values except for null-ness checks, pending some credible argument why we should think something else could be usefully stable. Kyotaro Horiguchi, further modified by me Discussion: https://postgr.es/m/E1jM1Sa-0003mS-99@gemulon.postgresql.org https://git.postgresql.org/pg/commitdiff/e083fa34ced0d53807a57482048bb4c135c3d006 - Doc: sync CREATE GROUP syntax synopsis with CREATE ROLE. CREATE GROUP is an exact alias for CREATE ROLE, and CREATE USER is almost an exact alias, as can easily be confirmed by checking the code. So the man page syntax descriptions ought to match up. The last few additions of role options seem to have forgotten to update create_group.sgml, though. Fix that, and add a naggy reminder to create_role.sgml in hopes of not forgetting again. Discussion: https://postgr.es/m/158647836143.655.9853963229391401576@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/7c91e9055d254524d76a72b35a919b8ff9931802 - Suppress unused-variable warning. Ashutosh Bapat Discussion: https://postgr.es/m/CAG-ACPWPB8Lc_aFj25eiPFqi31YB5vmaZnb39mbHSf5Yej=miA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/401418ca6a689f772cbfa1aedc7485cbbcde7a94 - Doc: clarify locking requirements for ALTER TABLE ADD FOREIGN KEY. The docs explained that a SHARE ROW EXCLUSIVE lock is needed on the referenced table, but failed to say the same about the table being altered. Since the page says that ACCESS EXCLUSIVE lock is taken unless otherwise stated, this left readers with the wrong conclusion. Discussion: https://postgr.es/m/834603375.3470346.1586482852542@mail.yahoo.com https://git.postgresql.org/pg/commitdiff/f333d35428c1cba8d35065b6dbb2dd46e18bd929 - Clear dangling pointer to avoid bogus EXPLAIN printout in a corner case. ExecReScanHashJoin will destroy the join's hash table if it expects that the inner relation will produce different rows on rescan. Up to now it's not bothered to clear the additional pointer to that hash table that exists in the child HashState node. However, it's possible for the query to terminate without building a fresh hash table (this happens if the outer relation is found to be empty during the final rescan). So we can end with a dangling pointer to a deleted hash table. That was harmless originally, but since 9.0 EXPLAIN ANALYZE has used that pointer to print hash table statistics. In debug builds this reproducibly results in garbage statistics. In non-debug builds there's frequently no ill effects, but in principle one could get wrong EXPLAIN ANALYZE output, or perhaps even a crash if free() has released the hashtable memory back to the OS. To fix, just make sure we clear the additional pointer when destroying the hash table. In problematic cases, EXPLAIN ANALYZE will then print no hashtable statistics (reverting to its pre-9.0 behavior). This isn't ideal, but since the problem manifests only in unusual corner cases, it's hard to justify taking any risks to do better in the back branches. A follow-on patch will improve matters in HEAD. Konstantin Knizhnik and Tom Lane, per diagnosis by Thomas Munro of a trouble report from Alvaro Herrera. Discussion: https://postgr.es/m/20200323165059.GA24950@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/5c27bce7f39ded1f027475221b732bbbc31a2bfe - Make EXPLAIN report maximum hashtable usage across multiple rescans. Before discarding the old hash table in ExecReScanHashJoin, capture its statistics, ensuring that we report the maximum hashtable size across repeated rescans of the hash input relation. We can repurpose the existing code for reporting hashtable size in parallel workers to help with this, making the patch pretty small. This also ensures that if rescans happen within parallel workers, we get the correct maximums across all instances. Konstantin Knizhnik and Tom Lane, per diagnosis by Thomas Munro of a trouble report from Alvaro Herrera. Discussion: https://postgr.es/m/20200323165059.GA24950@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/969f9d0b4ba574bb8df65683dbf7a09c030f3e67 - Suppress -Wimplicit-fallthrough warning in new LIMIT WITH TIES code. The placement of the fall-through comment in this code appears not to work to suppress the warning in recent gcc. Move it to the bottom of the case group, and add an assertion that we didn't get there through some other code path. Also improve wording of nearby comments. Julien Rouhaud, comment hacking by me Discussion: https://postgr.es/m/CAOBaU_aLdPGU5wCpaowNLF-Q8328iR7mj1yJAhMOVsdLwY+sdg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/35cb574aa84723f4661e9fc51340130e64cb5dfc Tomáš Vondra pushed: - Implement Incremental Sort. Incremental Sort is an optimized variant of multikey sort for cases when the input is already sorted by a prefix of the requested sort keys. For example when the relation is already sorted by (key1, key2) and we need to sort it by (key1, key2, key3) we can simply split the input rows into groups having equal values in (key1, key2), and only sort/compare the remaining column key3. This has a number of benefits: - Reduced memory consumption, because only a single group (determined by values in the sorted prefix) needs to be kept in memory. This may also eliminate the need to spill to disk. - Lower startup cost, because Incremental Sort produce results after each prefix group, which is beneficial for plans where startup cost matters (like for example queries with LIMIT clause). We consider both Sort and Incremental Sort, and decide based on costing. The implemented algorithm operates in two different modes: - Fetching a minimum number of tuples without check of equality on the prefix keys, and sorting on all columns when safe. - Fetching all tuples for a single prefix group and then sorting by comparing only the remaining (non-prefix) keys. We always start in the first mode, and employ a heuristic to switch into the second mode if we believe it's beneficial - the goal is to minimize the number of unnecessary comparions while keeping memory consumption below work_mem. This is a very old patch series. The idea was originally proposed by Alexander Korotkov back in 2013, and then revived in 2017. In 2018 the patch was taken over by James Coleman, who wrote and rewrote most of the current code. There were many reviewers/contributors since 2013 - I've done my best to pick the most active ones, and listed them in this commit message. Author: James Coleman, Alexander Korotkov Reviewed-by: Tomas Vondra, Andreas Karlsson, Marti Raudsepp, Peter Geoghegan, Robert Haas, Thomas Munro, Antonin Houska, Andres Freund, Alexander Kuzmenkov Discussion: https://postgr.es/m/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=goN-gfA@mail.gmail.com Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/d2d8a229bc58a2014dce1c7a4fcdb6c5ab9fb8da - Fix show_incremental_sort_info with force_parallel_mode. When executed with force_parallel_mode=regress, the function was exiting too early and thus failed to print the worker stats. Fixed by making it more like show_sort_info. Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/7d6d82a52493ad47c57662d0ac6758da551e87a5 - Fix failures in incremental_sort due to number of workers. The last test in incremental_sort suite prints a parallel plan, but some of the buildfarm animals have custom max_parallel_workers_per_gather values, causing failures. Fixed by setting the GUC to an explicit value. Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/23ba3b5ee278847e4fad913b80950edb2838fd35 - Use INT64_FORMAT when formatting int64 values in explain. Per report from lapwing. https://git.postgresql.org/pg/commitdiff/4bea576b032d6e5435ef0946194aada314e67691 - Consider Incremental Sort paths at additional places. Commit d2d8a229bc introduced Incremental Sort, but it was considered only in create_ordered_paths() as an alternative to regular Sort. There are many other places that require sorted input and might benefit from considering Incremental Sort too. This patch modifies a number of those places, but not all. The concern is that just adding Incremental Sort to any place that already adds Sort may increase the number of paths considered, negatively affecting planning time, without any benefit. So we've taken a more conservative approach, based on analysis of which places do affect a set of queries that did seem practical. This means some less common queries may not benefit from Incremental Sort yet. Author: Tomas Vondra Reviewed-by: James Coleman Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ba3e76cc571eba3dea19c9465ff15ac3ac186576 - Minor improvements in Incremental Sort explain. Some places still used "Maximum" instead of "Peak" when displaying info about sort space, so fix that. Also, add a comment clarifying why it's correct to check the number of full/prefix sort groups. Author: James Coleman Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/d22782a5392f6a1cb6cfca34031b93eb9dd2aa03 - Remove debugging elog from pgstat_recv_resetslrucounter. Reported-by: Thomas Munro https://git.postgresql.org/pg/commitdiff/9c74ceb20b991f786f71666d4b4d557d2744a567 - Track SLRU page hits in SimpleLruReadPage_ReadOnly. SLRU page hits were tracked only in SimpleLruReadPage, but that's not enough because we may hit the page in SimpleLruReadPage_ReadOnly in which case we don't call SimpleLruReadPage at all. Reported-by: Kuntal Ghosh Discussion: https://postgr.es/m/20200119143707.gyinppnigokesjok@development https://git.postgresql.org/pg/commitdiff/2b88fdde30d8e9bf833b75a014189e9148233b85 - Stabilize incremental_sort tests. The test never did ANALYZE on the test table, so the plans depended on various defaults (e.g. number of groups being 200). This worked most of the time, but with CLOBBER_CACHE_ALWAYS the autoanalyze often managed to build accurate stats, changing the plan. Fixed by increasing the size of test tables a bit, making the Sort a bit more expensive than Incremental Sort. The tests were constructed to test transitions in the Incremental Sort algorithm, and this change does not break that. Reviewed-by: James Coleman Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/cea09246e57821b8a97a6483a7df6c7345b055ef Peter Geoghegan pushed: - Fix nbtree kill_prior_tuple posting list assert. An assertion added by commit 0d861bbb checked that _bt_killitems() only processes a BTScanPosItem whose heap TID is contained in a posting list tuple when its page offset number still matches what is on the page (i.e. when it matches the posting list tuple's current offset number). This was only correct in the common case where the page can't have changed since we first read it. It was not correct in cases where we don't drop the buffer pin (and don't need to verify the page hasn't changed using its LSN). The latter category includes scans involving unlogged tables, and scans that use a non-MVCC snapshot, per the logic originally introduced by commit 2ed5b87f. The assertion still seems helpful. Fix it by taking cases where the page may have been concurrently modified into account. Reported-By: Anastasia Lubennikova, Alexander Lakhin Discussion: https://postgr.es/m/c4e38e9a-0f9c-8e53-e639-adf343f94472@postgrespro.ru https://git.postgresql.org/pg/commitdiff/ce2cee0ade8a6a360322c51201fda1fc25be7773 - Remove nbtree BTreeTupleSetAltHeapTID() function. Since heap TID is supposed to be just another key attribute to the implementation, it doesn't make much sense to have separate BTreeTupleSetNAtts() and BTreeTupleSetAltHeapTID() functions. Merge the two functions together. This slightly simplifies _bt_truncate(). https://git.postgresql.org/pg/commitdiff/60cbd7751c1ec6ffdf2ffc520ddeb35cb2f2ed70 - Add contrib/amcheck debug message. Add a DEBUG1 message indicating that verification of the index structure is underway. Also reduce the severity level of the existing "tree level" debug message to DEBUG1. It should never have been made DEBUG2. Any B-Tree index with more than a couple of levels will generally also have so many pages that the per-page DEBUG2 messages will become completely unmanageable. In passing, add a new "Tip" to the docs that advises users that run into corruption that the debug messages might provide useful additional context. https://git.postgresql.org/pg/commitdiff/20fbb711ef43ef70093378ff5efdf1b6e8cc10d8 - Doc: Fix contrib/amcheck tip. Fixes an oversight in commit 20fbb711. https://git.postgresql.org/pg/commitdiff/26640c40715c7f2045cf1b7c6753cac40b64d1e8 Thomas Munro pushed: - Add SQL type xid8 to expose FullTransactionId to users. Similar to xid, but 64 bits wide. This new type is suitable for use in various system views and administration functions. Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com> Reviewed-by: Takao Fujii <btfujiitkp@oss.nttdata.com> Reviewed-by: Yoshikazu Imai <imai.yoshikazu@fujitsu.com> Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com> Discussion: https://postgr.es/m/20190725000636.666m5mad25wfbrri%40alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/aeec457de8a8820368e343e791accffe24dc7198 - Introduce xid8-based functions to replace txid_XXX. The txid_XXX family of fmgr functions exposes 64 bit transaction IDs to users as int8. Now that we have an SQL type xid8 for FullTransactionId, define a new set of functions including pg_current_xact_id() and pg_current_snapshot() based on that. Keep the old functions around too, for now. It's a bit sneaky to use the same C functions for both, but since the binary representation is identical except for the signedness of the type, and since older functions are the ones using the wrong signedness, and since we'll presumably drop the older ones after a reasonable period of time, it seems reasonable to switch to FullTransactionId internally and share the code for both. Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com> Reviewed-by: Takao Fujii <btfujiitkp@oss.nttdata.com> Reviewed-by: Yoshikazu Imai <imai.yoshikazu@fujitsu.com> Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com> Discussion: https://postgr.es/m/20190725000636.666m5mad25wfbrri%40alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/4c04be9b05ad2ec5acd27c3417bf075c13cab134 - Support PrefetchBuffer() in recovery. Provide PrefetchSharedBuffer(), a variant that takes SMgrRelation, for use in recovery. Rename LocalPrefetchBuffer() to PrefetchLocalBuffer() for consistency. Add a return value to all of these. In recovery, tolerate and report missing files, so we can handle relations unlinked before crash recovery began. Also report cache hits and misses, so that callers can do faster buffer lookups and better I/O accounting. Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CA%2BhUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq%3DAovOddfHpA%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/3985b600f57d75b9743d86430cb5c21370057a23 - Rationalize GetWalRcv{Write,Flush}RecPtr(). GetWalRcvWriteRecPtr() previously reported the latest *flushed* location. Adopt the conventional terminology used elsewhere in the tree by renaming it to GetWalRcvFlushRecPtr(), and likewise for some related variables that used the term "received". Add a new definition of GetWalRcvWriteRecPtr(), which returns the latest *written* value. This will allow later patches to use the value for non-data-integrity purposes, without having to wait for the flush pointer to advance. Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CA%2BhUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq%3DAovOddfHpA%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/d140f2f3e225ea53e2d92ab6833b8c186c90666c Fujii Masao pushed: - Prevent archive recovery from scanning non-existent WAL files. Previously when there were multiple timelines listed in the history file of the recovery target timeline, archive recovery searched all of them, starting from the newest timeline to the oldest one, to find the segment to read. That is, archive recovery had to continuously fail scanning the segment until it reached the timeline that the segment belonged to. These scans for non-existent segment could be harmful on the recovery performance especially when archival area was located on the remote storage and each scan could take a long time. To address the issue, this commit changes archive recovery so that it skips scanning the timeline that the segment to read doesn't belong to. Author: Kyotaro Horiguchi, tweaked a bit by Fujii Masao Reviewed-by: David Steele, Pavel Suderevsky, Grigory Smolkin Discussion: https://postgr.es/m/16159-f5a34a3a04dc67e0@postgresql.org Discussion: https://postgr.es/m/20200129.120222.1476610231001551715.horikyota.ntt@gmail.com https://git.postgresql.org/pg/commitdiff/4bd0ad9e44be9fbc3ad77747d7672dab1c3df7d9 - Add note in pg_stat_statements documentation about planning statistics. The added note explains that the numbers of planning and execution in the statement are not always expected to match because their statistics are updated at their respective end phase, and only for successful operations. Author: Pascal Legrand, Julien Rouhaud, tweaked a bit by Fujii Masao Discussion: https://postgr.es/m/1585857868967-0.post@n3.nabble.com https://git.postgresql.org/pg/commitdiff/58ad961f19f7eca26e6d60eb07adcffeafd0082e - Exclude backup_manifest file that existed in database, from BASE_BACKUP. If there is already a backup_manifest file in the database cluster, it belongs to the past backup that was used to start this server. It is not correct for the backup being taken now. So this commit changes pg_basebackup so that it always skips such backup_manifest file. The backup_manifest file for the current backup will be injected separately if users want it. Author: Fujii Masao Reviewed-by: Robert Haas Discussion: https://postgr.es/m/78f76a3d-1a28-a97d-0394-5c96985dd1c0@oss.nttdata.com https://git.postgresql.org/pg/commitdiff/1ec50a81ec0acd452c7520de19e607a6de8fba5e - Fix typo in pg_validatebackup documentation. Author: Fujii Masao Reviewed-by: Robert Haas Discussion: https://postgr.es/m/78f76a3d-1a28-a97d-0394-5c96985dd1c0@oss.nttdata.com https://git.postgresql.org/pg/commitdiff/c4f82a779d2676bfca1694a6f9b5499e6cc5f60f Álvaro Herrera pushed: - Support FETCH FIRST WITH TIES. WITH TIES is an option to the FETCH FIRST N ROWS clause (the SQL standard's spelling of LIMIT), where you additionally get rows that compare equal to the last of those N rows by the columns in the mandatory ORDER BY clause. There was a proposal by Andrew Gierth to implement this functionality in a more powerful way that would yield more features, but the other patch had not been finished at this time, so we decided to use this one for now in the spirit of incremental development. Author: Surafel Temesgen <surafel3000@gmail.com> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Tomas Vondra <tomas.vondra@2ndquadrant.com> Discussion: https://postgr.es/m/CALAY4q9ky7rD_A4vf=FVQvCGngm3LOes-ky0J6euMrg=_Se+ag@mail.gmail.com Discussion: https://postgr.es/m/87o8wvz253.fsf@news-spur.riddles.org.uk https://git.postgresql.org/pg/commitdiff/357889eb17bb9c9336c4f324ceb1651da616fe57 - Allow users to limit storage reserved by replication slots. Replication slots are useful to retain data that may be needed by a replication system. But experience has shown that allowing them to retain excessive data can lead to the primary failing because of running out of space. This new feature allows the user to configure a maximum amount of space to be reserved using the new option max_slot_wal_keep_size. Slots that overrun that space are invalidated at checkpoint time, enabling the storage to be released. Author: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: Jehan-Guillaume de Rorthais <jgdr@dalibo.com> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/20170228.122736.123383594.horiguchi.kyotaro@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/c6550776394e25c1620bc8258427c8f1d448080d - Appease perlcritic. Food for the gods must always be found somehow, even when the land starves. https://git.postgresql.org/pg/commitdiff/7e2ffb3885007183af7b161e046e126be2cfba12 - Remove testing for precise LSN/reserved bytes in new TAP test. Trying to ensure that a slot's restart_lsn or amount of reserved bytes exactly match some specific values seems unnecessary, and fragile as shown by failures in multiple buildfarm members. Discussion: https://postgr.es/m/20200407232602.GA21559@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/9e9abed746280086474e2191b8c399b5fd9b0678 Alexander Korotkov pushed: - Implement waiting for given lsn at transaction start. This commit adds following optional clause to BEGIN and START TRANSACTION commands. WAIT FOR LSN lsn [ TIMEOUT timeout ] New clause pospones transaction start till given lsn is applied on standby. This clause allows user be sure, that changes previously made on primary would be visible on standby. New shared memory struct is used to track awaited lsn per backend. Recovery process wakes up backend once required lsn is applied. Author: Ivan Kartyshov, Anna Akenteva Reviewed-by: Craig Ringer, Thomas Munro, Robert Haas, Kyotaro Horiguchi Reviewed-by: Masahiko Sawada, Ants Aasma, Dmitry Ivanov, Simon Riggs Reviewed-by: Amit Kapila, Alexander Korotkov Discussion: https://postgr.es/m/0240c26c-9f84-30ea-fca9-93ab2df5f305%40postgrespro.ru https://git.postgresql.org/pg/commitdiff/0f5ca02f53ac2b211d8518f0882c49284c0c9610 - Revert 0f5ca02f53. 0f5ca02f53 introduces 3 new keywords. It appears to be too much for relatively small feature. Given now we past feature freeze, it's already late for discussion of the new syntax. So, revert. Discussion: https://postgr.es/m/28209.1586294824%40sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/1aac32df89eb19949050f6f27c268122833ad036 Etsuro Fujita pushed: - Allow partitionwise joins in more cases. Previously, the partitionwise join technique only allowed partitionwise join when input partitioned tables had exactly the same partition bounds. This commit extends the technique to some cases when the tables have different partition bounds, by using an advanced partition-matching algorithm introduced by this commit. For both the input partitioned tables, the algorithm checks whether every partition of one input partitioned table only matches one partition of the other input partitioned table at most, and vice versa. In such a case the join between the tables can be broken down into joins between the matching partitions, so the algorithm produces the pairs of the matching partitions, plus the partition bounds for the join relation, to allow partitionwise join for computing the join. Currently, the algorithm works for list-partitioned and range-partitioned tables, but not hash-partitioned tables. See comments in partition_bounds_merge(). Ashutosh Bapat and Etsuro Fujita, most of regression tests by Rajkumar Raghuwanshi, some of the tests by Mark Dilger and Amul Sul, reviewed by Dmitry Dolgov and Amul Sul, with additional review at various points by Ashutosh Bapat, Mark Dilger, Robert Haas, Antonin Houska, Amit Langote, Justin Pryzby, and Tomas Vondra Discussion: https://postgr.es/m/CAFjFpRdjQvaUEV5DJX3TW6pU5eq54NCkadtxHX2JiJG_GvbrCA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/c8434d64ce03c32e0029417a82ae937f2055268f David Rowley pushed: - Add functions to calculate the next power of 2. There are many areas in the code where we need to determine the next highest power of 2 of a given number. We tend to always do that in an ad-hoc way each time, generally with some tight for loop which performs a bitshift left once per loop and goes until it finds a number above the given number. Here we add two generic functions which make use of the existing pg_leftmost_one_pos* functions which, when available, will allow us to calculate the next power of 2 without any looping. Here we don't add any code which uses these new functions. That will be done in follow-up commits. Author: David Fetter, with some minor adjustments by me Reviewed-by: John Naylor, Jesse Zhang Discussion: https://postgr.es/m/20200114173553.GE32763%40fetter.org https://git.postgresql.org/pg/commitdiff/f0705bb6286d8a24e08ddd99641264ba947ebd03 - Modify various power 2 calculations to use new helper functions. First pass of modifying various places that obtain the next power of 2 of a number and make them use the new functions added in pg_bitutils.h instead. This also removes the _hash_log2() function. There are no longer any callers in core. Other users can swap their _hash_log2(n) call to make use of pg_ceil_log2_32(n). Author: David Fetter, with some minor adjustments by me Reviewed-by: John Naylor, Jesse Zhang Discussion: https://postgr.es/m/20200114173553.GE32763%40fetter.org https://git.postgresql.org/pg/commitdiff/d025cf88ba5a64487ee4a17ef23e8f55b1536606 - Modify additional power 2 calculations to use new helper functions. 2nd pass of modifying various places which obtain the next power of 2 of a number and make them use the new functions added in f0705bb62. In passing, also modify num_combinations(). This can be implemented using simple bitshifting rather than looping. Reviewed-by: John Naylor Discussion: https://postgr.es/m/20200114173553.GE32763%40fetter.org https://git.postgresql.org/pg/commitdiff/02a2e8b442002a698336954633b0ccc4e30061e6 Jeff Davis pushed: - Create memory context for HashAgg with a reasonable maxBlockSize. If the memory context's maxBlockSize is too big, a single block allocation can suddenly exceed work_mem. For Hash Aggregation, this can mean spilling to disk too early or reporting a confusing memory usage number for EXPLAN ANALYZE. Introduce CreateWorkExprContext(), which is like CreateExprContext(), except that it creates the AllocSet with a maxBlockSize that is reasonable in proportion to work_mem. Right now, CreateWorkExprContext() is only used by Hash Aggregation, but it may be generally useful in the future. Discussion: https://postgr.es/m/412a3fbf306f84d8d78c4009e11791867e62b87c.camel@j-davis.com https://git.postgresql.org/pg/commitdiff/50a38f65177ea7858bc97f71ba0757ba04c1c167 Andrew Dunstan pushed: - Msys2 tweaks for pg_validatebackup corruption test. 1. Tell Msys2 not to mangle the tablespace map parameter 2. If rmdir doesn't work, fall back to trying unlink on the entry in pg_tblspc. Discussion: https://postgr.es/m/7330a7c7-ce5f-9769-39a1-bdb0b32bb4a6@2ndQuadrant.com https://git.postgresql.org/pg/commitdiff/c3e4cbaab936a17b579d85c5ff28bcb2251736d0 Andrew Gierth pushed: - doc: restore intentional typo. Commit ac8623760 "fixed" a typo in an example of what would happen in the event of a typo. Restore the original typo and add a comment about its intentionality. Backpatch to 12 where the error was introduced. Per report from irc user Nicolás Alvarez. https://git.postgresql.org/pg/commitdiff/8a47b775a16fb4f1e154c0f319a030498e123164 Noah Misch pushed: - When WalSndCaughtUp, sleep only in WalSndWaitForWal(). Before sleeping, WalSndWaitForWal() sends a keepalive if MyWalSnd->write < sentPtr. That is important in logical replication. When the latest physical LSN yields no logical replication messages (a common case), that keepalive elicits a reply, and processing the reply updates pg_stat_replication.replay_lsn. WalSndLoop() lacks that; when WalSndLoop() slept, replay_lsn advancement could stall until wal_receiver_status_interval elapsed. This sometimes stalled src/test/subscription/t/001_rep_changes.pl for up to 10s. Discussion: https://postgr.es/m/20200406063649.GA3738151@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/421685812290406daea58b78dfab0346eb683bbb - Optimize RelationFindReplTupleSeq() for CLOBBER_CACHE_ALWAYS. Specifically, remember lookup_type_cache() results instead of retrieving them once per comparison. Under CLOBBER_CACHE_ALWAYS, this reduced src/test/subscription/t/001_rep_changes.pl elapsed time by an order of magnitude, which reduced check-world elapsed time by 9%. Discussion: https://postgr.es/m/20200406085420.GC162712@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/328c70997bc3518a50bd9a8ff33de349a7223413 Robert Haas pushed: - Rename pg_validatebackup to pg_verifybackup. Also, use "verify" rather than "validate" to refer to the process being undertaken here. Per discussion, that is a more appropriate term. Discussion: https://www.postgresql.org/message-id/172c9d9b-1d0a-1b94-1456-376b1e017322@2ndquadrant.com Discussion: http://postgr.es/m/CA+TgmobLgMh6p8FmLbj_rv9Uhd7tPrLnAyLgGd2SoSj=qD-bVg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/dbc60c5593f26dc777a3be032bff4fb4eab1ddd1 == Pending Patches == Michaël Paquier sent in a WIP patch to libpq which attempts an SSL connection first when both sslmode and gssencmode are both set to avert a performance hit for SSL users. Ivan N. Taranov sent in a patch to speed up VALGRIND_MEMPOOL_* calls by adding a GetMemoryChunkCapacity method which returns the size of usable space in the chunk, using same on VALGRIND_MEMPOOL_* calls, and calling VALGRIND_MEMPOOL_CHANGED only for really changed chunks. Anastasia Lubennikova sent in two more revisions of a patch to fix an infelicity between pg_upgrade and non-standard ACLs. Grigory Smolkin sent in a patch to fix an issue where archive_recovery was fetching the wrong segments. Tom Lane sent in another revision of a patch to fix checksum verification in base backups for random. Michael Banck sent in a patch to allow checking base backups in pg_checksums. Fujii Masao sent in two more revisions of a patch to avoid fetching out-of-timeline segments. Michaël Paquier sent in a patch to fix plug a file descriptor leak in xlogreader. Kyotaro HORIGUCHI and Álvaro Herrera traded patches to add a WAL relief vent for replication slots. Tomáš Vondra, Justin Pryzby, and James Coleman traded patches to implement incremental sort. Davinder Singh and Ranier Vilela traded patches to fix a compilation error with Microsoft Visual Studio 2015/2017/2019. Ivan Kartyshov and Anna Akenteva traded patches to implement BEGIN ... WAIT FOR ... [TIMEOUT ...]. Paul A Jungwirth and Álvaro Herrera traded patches to implement multi-ranges. Justin Pryzby and Aleksey Kondratov traded patches to enable CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly. Julien Rouhaud sent in another revision of a patch to expose WAL usage counters in verbose (auto)vacuum output. Masahiko Sawada sent in three more revisions of a patch to instrument buffer usage for CREATE INDEX. James Coleman sent in a patch to fix parallel explain. Jie Zhang sent in a patch to ensure that PQExpBuffers are destroyed during pg_dump. Yuzuko Hosoya sent in another revision of a patch to run autovacuum on partitioned tables. Kyotaro HORIGUCHI sent in another revision of a patch to implement the stats collector using shared memory instead of files. Zeng Wenjing sent in a patch to make the testing of equivalents of boolean values more precise in relopt. Kuntal Ghosh sent in a patch to update stats in SimpleLruReadPage_ReadOnly. David Cramer sent in another revision of a patch to implement binary support for the pgoutput logical replication plugin. Jeff Davis sent in another revision of a patch to make MemoryContextMemAllocated() more precise. David Zhang sent in another revision of a patch to ensure that the circle type parses as documented. Asif Rehman sent in two more revisions of a patch to implement parallel backup. Thomas Munro sent in three more revisions of a patch to implement WAL prefetching. Andres Freund sent in two more revisions of a patch to improve the scalability of GetSnapshotData(). Zeng Wenjing sent in five more revisions of a patch to implement global temporary tables. Andres Freund sent in another revision of a patch to fix infelicities among catalog invalidations, catalog scans, and ScanPgRelation(). Amit Kapila and Justin Pryzby traded patches to fix the documentation for parallel vacuum. Dmitry Dolgov sent in another revision of a patch to implement index skip scans. David Fetter sent in another revision of a patch to enable setting pg_hba.conf permissions from initdb. Antonin Houska sent in another revision of a patch to make foreign key reference checks more efficient. Justin Pryzby sent in another revision of a patch to make explain HashAggregate report bucket and memory stats. Justin Pryzby sent in a WIP patch to fix detaching tables with inherited triggers. Fujii Masao sent in a patch to clarify what pg_stat_statements is actually counting in the "calls" field. Andrew Dunstan sent in a patch to fix some tests backup manifests broke on Windows. Michail Nikolaev sent in another revision of a patch to help standbys use hint bits. Justin Pryzby sent in two more revisions of a patch to fix some infelicities in the documentation for 13. Justin Pryzby sent in three revisions of a patch to Allow specifying "(parallel 0)" with "vacuum(full)", even though full implies parallel 0. Michaël Paquier sent in a patch to fix some bugs in some of the TID functions and which caused a segmentation fault using currtid and partitioned tables. Dilip Kumar sent in another revision of a patch to fix an infelicity between logical_work_mem and logical streaming of large in-progress transactions. Peter Eisentraut sent in another revision of a patch to propagate ALTER TABLE ... SET STORAGE to indexes. Thomas Munro sent in a patch to implement fast DSM segments. Alexandra Wang and Ashutosh Bapat traded patches to improve the check new partition bound error position report. Álvaro Herrera sent in another revision of a patch to add ALTER .. NO DEPENDS ON, and tab completion for ALTER INDEX ... NO DEPENDS ON. Masahiko Sawada sent in a patch to fix a bug that manifested as Multiple FPI_FOR_HINT for the same block during killing btree index items by adding a check for whether an item was dead to _bt_killitems. Yugo Nagata sent in another revision of a patch to implement incremental maintenance of materialized views. Pavel Stěhule sent in another revision of a patch to implement schema variables. Nathan Bossart sent in a patch to avoid stop considering pg_init_privs entries when dumping renamed system schemas. Jesse Zhang, Denis Smirnov, and Soumyadeep Chakraborty sent in a patch to properly mark NULL returns in numeric aggregates. Fujii Masao sent in a patch to ensure a fast exit to locking when no synchronous replica names are defined. Robert Haas sent in a patch to rename pg_validatebackup to pg_verifybackup. Teja Mupparti sent in a patch to fix a bug which can cause redo recovery to fail(forever) by marking all the buffers as about-to-be-dropped, calling CacheInvalidateSmgr(), then calling Truncate on the filesystem level. Pavel Stěhule sent in another revision of a patch to make it possible to redirect tabular-only output. Andrew Dunstan sent in a patch to clean up some Perl issues pointed out by perlcritic. David Steele sent in a patch to make appropriate exceptions for perlcritic. Julien Rouhaud sent in two revisions of a patch to add "-Wimplicit-fallthrough" to the default flags.
В списке pgsql-announce по дате отправления: