Обсуждение: pgsql: Transform OR clauses to ANY expression
Transform OR clauses to ANY expression Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, ...]) on the preliminary stage of optimization when we are still working with the expression tree. Here Cn is a n-th constant expression, 'expr' is non-constant expression, 'op' is an operator which returns boolean result and has a commuter (for the case of reverse order of constant and non-constant parts of the expression, like 'Cn op expr'). Sometimes it can lead to not optimal plan. This is why there is a or_to_any_transform_limit GUC. It specifies a threshold value of length of arguments in an OR expression that triggers the OR-to-ANY transformation. Generally, more groupable OR arguments mean that transformation will be more likely to win than to lose. Discussion: https://postgr.es/m/567ED6CA.2040504%40sigaev.ru Author: Alena Rybakina <lena.ribackina@yandex.ru> Author: Andrey Lepikhov <a.lepikhov@postgrespro.ru> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Reviewed-by: Ranier Vilela <ranier.vf@gmail.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Reviewed-by: Robert Haas <robertmhaas@gmail.com> Reviewed-by: Jian He <jian.universality@gmail.com> Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/72bd38cc99a15da6f97373fae98027c908c398ea Modified Files -------------- doc/src/sgml/config.sgml | 57 ++++ src/backend/nodes/queryjumblefuncs.c | 27 ++ src/backend/optimizer/prep/prepqual.c | 399 +++++++++++++++++++++++++- src/backend/utils/misc/guc_tables.c | 12 + src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/nodes/queryjumble.h | 1 + src/include/optimizer/optimizer.h | 2 + src/test/regress/expected/create_index.out | 159 ++++++++++ src/test/regress/expected/join.out | 50 ++++ src/test/regress/expected/partition_prune.out | 37 +-- src/test/regress/sql/create_index.sql | 45 +++ src/test/regress/sql/join.sql | 11 + src/test/regress/sql/partition_prune.sql | 2 + src/tools/pgindent/typedefs.list | 2 + 14 files changed, 785 insertions(+), 20 deletions(-)
On Sun, Apr 7, 2024 at 6:28 PM Alexander Korotkov <akorotkov@postgresql.org> wrote: > > Transform OR clauses to ANY expression > > Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, ...]) > on the preliminary stage of optimization when we are still working with the > expression tree. > > Here Cn is a n-th constant expression, 'expr' is non-constant expression, 'op' > is an operator which returns boolean result and has a commuter (for the case > of reverse order of constant and non-constant parts of the expression, > like 'Cn op expr'). > > Sometimes it can lead to not optimal plan. This is why there is a > or_to_any_transform_limit GUC. It specifies a threshold value of length of > arguments in an OR expression that triggers the OR-to-ANY transformation. > Generally, more groupable OR arguments mean that transformation will be more > likely to win than to lose. I'm getting this warning now /src/backend/optimizer/prep/prepqual.c:582:33: warning: declaration of ‘lc__state’ shadows a previous local [-Wshadow=compatible-local] 582 | foreach(lc, entry->consts)
On Mon, Apr 8, 2024 at 1:35 AM Melanie Plageman <melanieplageman@gmail.com> wrote: > On Sun, Apr 7, 2024 at 6:28 PM Alexander Korotkov > <akorotkov@postgresql.org> wrote: > > > > Transform OR clauses to ANY expression > > > > Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, ...]) > > on the preliminary stage of optimization when we are still working with the > > expression tree. > > > > Here Cn is a n-th constant expression, 'expr' is non-constant expression, 'op' > > is an operator which returns boolean result and has a commuter (for the case > > of reverse order of constant and non-constant parts of the expression, > > like 'Cn op expr'). > > > > Sometimes it can lead to not optimal plan. This is why there is a > > or_to_any_transform_limit GUC. It specifies a threshold value of length of > > arguments in an OR expression that triggers the OR-to-ANY transformation. > > Generally, more groupable OR arguments mean that transformation will be more > > likely to win than to lose. > > I'm getting this warning now > > /src/backend/optimizer/prep/prepqual.c:582:33: warning: declaration of > ‘lc__state’ shadows a previous local [-Wshadow=compatible-local] > 582 | foreach(lc, entry->consts) Thank you for catching. I'm fixing this now. ------ Regards, Alexander Korotkov
At Sun, 07 Apr 2024 22:28:06 +0000, Alexander Korotkov <akorotkov@postgresql.org> wrote in
> Transform OR clauses to ANY expression
This commit introduces a message like this:
> gettext_noop("Set the minimum length of the list of OR clauses to attempt the OR-to-ANY transformation."),
Unlike the usual phrasing of similar messages in this context, which
use the form "Sets the minimum length of...", this message uses an
imperative form ("Set"). I believe it would be better to align the
tone of this message with the others by changing "Set" to "Sets".
regards.
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 83e3a59d7e..a527ffe0b0 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3670,7 +3670,7 @@ struct config_int ConfigureNamesInt[] =
{
{"or_to_any_transform_limit", PGC_USERSET, QUERY_TUNING_OTHER,
- gettext_noop("Set the minimum length of the list of OR clauses to attempt the OR-to-ANY
transformation."),
+ gettext_noop("Sets the minimum length of the list of OR clauses to attempt the OR-to-ANY
transformation."),
gettext_noop("Once the limit is reached, the planner will try to replace expression like "
"'x=c1 OR x=c2 ..' to the expression 'x = ANY(ARRAY[c1,c2,..])'"),
GUC_EXPLAIN
--
Kyotaro Horiguchi
NTT Open Source Software Center
At Mon, 08 Apr 2024 14:46:57 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> At Sun, 07 Apr 2024 22:28:06 +0000, Alexander Korotkov <akorotkov@postgresql.org> wrote in
> > Transform OR clauses to ANY expression
>
> This commit introduces a message like this:
>
> > gettext_noop("Set the minimum length of the list of OR clauses to attempt the OR-to-ANY transformation."),
>
> Unlike the usual phrasing of similar messages in this context, which
> use the form "Sets the minimum length of...", this message uses an
> imperative form ("Set"). I believe it would be better to align the
> tone of this message with the others by changing "Set" to "Sets".
>
> regards.
>
>
> diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
> index 83e3a59d7e..a527ffe0b0 100644
> --- a/src/backend/utils/misc/guc_tables.c
> +++ b/src/backend/utils/misc/guc_tables.c
> @@ -3670,7 +3670,7 @@ struct config_int ConfigureNamesInt[] =
>
> {
> {"or_to_any_transform_limit", PGC_USERSET, QUERY_TUNING_OTHER,
> - gettext_noop("Set the minimum length of the list of OR clauses to attempt the OR-to-ANY
transformation."),
> + gettext_noop("Sets the minimum length of the list of OR clauses to attempt the OR-to-ANY
transformation."),
> gettext_noop("Once the limit is reached, the planner will try to replace expression like "
> "'x=c1 OR x=c2 ..' to the expression 'x = ANY(ARRAY[c1,c2,..])'"),
> GUC_EXPLAIN
Sorry for the sequential posts, but I found the following contruct in
the same patch to be quite untranslatable.
> errmsg("%s bound of partition \"%s\" is %s %s bound of split partition",
> first ? "lower" : "upper",
> relname,
> defaultPart ? (first ? "less than" : "greater than") : "not equals to",
> first ? "lower" : "upper"),
> parser_errposition(pstate, datum->location)));
I might be misunderstanding this, but if the expressions are correct,
the message could be divided into four fixed messages based on "first"
and "defaultPart". Alternatively, it might be better to provide
simpler explation of the situation.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On Mon, Apr 8, 2024 at 9:24 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:
> At Mon, 08 Apr 2024 14:46:57 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > At Sun, 07 Apr 2024 22:28:06 +0000, Alexander Korotkov <akorotkov@postgresql.org> wrote in
> > > Transform OR clauses to ANY expression
> >
> > This commit introduces a message like this:
> >
> > > gettext_noop("Set the minimum length of the list of OR clauses to attempt the OR-to-ANY transformation."),
> >
> > Unlike the usual phrasing of similar messages in this context, which
> > use the form "Sets the minimum length of...", this message uses an
> > imperative form ("Set"). I believe it would be better to align the
> > tone of this message with the others by changing "Set" to "Sets".
> >
> > regards.
> >
> >
> > diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
> > index 83e3a59d7e..a527ffe0b0 100644
> > --- a/src/backend/utils/misc/guc_tables.c
> > +++ b/src/backend/utils/misc/guc_tables.c
> > @@ -3670,7 +3670,7 @@ struct config_int ConfigureNamesInt[] =
> >
> > {
> > {"or_to_any_transform_limit", PGC_USERSET, QUERY_TUNING_OTHER,
> > - gettext_noop("Set the minimum length of the list of OR clauses to attempt the OR-to-ANY
transformation."),
> > + gettext_noop("Sets the minimum length of the list of OR clauses to attempt the OR-to-ANY
transformation."),
> > gettext_noop("Once the limit is reached, the planner will try to replace expression like "
> > "'x=c1 OR x=c2 ..' to the expression 'x = ANY(ARRAY[c1,c2,..])'"),
> > GUC_EXPLAIN
>
> Sorry for the sequential posts, but I found the following contruct in
> the same patch to be quite untranslatable.
No worries. But these are distinct patches.
> > errmsg("%s bound of partition \"%s\" is %s %s bound of split partition",
> > first ? "lower" : "upper",
> > relname,
> > defaultPart ? (first ? "less than" : "greater than") : "not equals to",
> > first ? "lower" : "upper"),
> > parser_errposition(pstate, datum->location)));
>
> I might be misunderstanding this, but if the expressions are correct,
> the message could be divided into four fixed messages based on "first"
> and "defaultPart". Alternatively, it might be better to provide
> simpler explation of the situation.
Yes, splitting this into multiple messages helps both translating and
readability. Will be fixed in the next couple of days.
------
Regards,
Alexander Korotkov
On Mon, 8 Apr 2024 at 17:10, Alexander Korotkov <aekorotkov@gmail.com> wrote: > > On Mon, Apr 8, 2024 at 1:35 AM Melanie Plageman > <melanieplageman@gmail.com> wrote: > > /src/backend/optimizer/prep/prepqual.c:582:33: warning: declaration of > > ‘lc__state’ shadows a previous local [-Wshadow=compatible-local] > > 582 | foreach(lc, entry->consts) > > Thank you for catching. I'm fixing this now. I noticed the fix in question, and I wanted to say that this whole issue could've been avoided if the new foreach_ptr macros were used (and thus arguably would have been a better way to fix this). Then there wouldn't have been any ListCell shadowing, because no ListCell would have been declared at all.