Обсуждение: BUG #16801: Invalid memory access on WITH RECURSIVE with nested WITHs

The following bug has been logged on the website:

Bug reference:      16801
Logged by:          Alexander Lakhin
Email address:      exclusion@gmail.com
PostgreSQL version: 13.1
Operating system:   Ubuntu 20.04
Description:

When executing the following query:
WITH RECURSIVE rec(x) AS (
    WITH outermost(x) AS (
      SELECT (
        WITH innermost as (SELECT 1)
        SELECT * FROM innermost
      )
    )
    SELECT * FROM outermost
)
SELECT * FROM rec;

valgrind detects an invalid read:
==00:00:00:04.145 217144== Invalid read of size 8
==00:00:00:04.145 217144==    at 0x302CB7: makeDependencyGraphWalker
(parse_cte.c:549)
==00:00:00:04.145 217144==    by 0x302EA1: makeDependencyGraph
(parse_cte.c:439)
==00:00:00:04.145 217144==    by 0x304557: transformWithClause
(parse_cte.c:176)
==00:00:00:04.145 217144==    by 0x2DD70A: transformSelectStmt
(analyze.c:1202)
==00:00:00:04.145 217144==    by 0x2DDAB4: transformStmt (analyze.c:301)
==00:00:00:04.145 217144==    by 0x2DEDDA: transformOptionalSelectInto
(analyze.c:246)
==00:00:00:04.145 217144==    by 0x2DEE0F: transformTopLevelStmt
(analyze.c:196)
==00:00:00:04.145 217144==    by 0x2DEE71: parse_analyze (analyze.c:116)
==00:00:00:04.145 217144==    by 0x55E69F: pg_analyze_and_rewrite
(postgres.c:691)
==00:00:00:04.145 217144==    by 0x55ED66: exec_simple_query
(postgres.c:1155)
==00:00:00:04.145 217144==    by 0x560D83: PostgresMain (postgres.c:4315)
==00:00:00:04.145 217144==    by 0x4CC6B8: BackendRun (postmaster.c:4526)
==00:00:00:04.145 217144==  Address 0x50890a8 is 24 bytes inside a block of
size 32 client-defined
==00:00:00:04.145 217144==    at 0x6B4831: palloc (mcxt.c:974)
==00:00:00:04.145 217144==    by 0x42B624: new_list (list.c:134)
==00:00:00:04.145 217144==    by 0x42BF1B: lcons (list.c:458)
==00:00:00:04.145 217144==    by 0x302C73: makeDependencyGraphWalker
(parse_cte.c:542)
==00:00:00:04.145 217144==    by 0x302EA1: makeDependencyGraph
(parse_cte.c:439)
==00:00:00:04.145 217144==    by 0x304557: transformWithClause
(parse_cte.c:176)
==00:00:00:04.145 217144==    by 0x2DD70A: transformSelectStmt
(analyze.c:1202)
==00:00:00:04.145 217144==    by 0x2DDAB4: transformStmt (analyze.c:301)
==00:00:00:04.145 217144==    by 0x2DEDDA: transformOptionalSelectInto
(analyze.c:246)
==00:00:00:04.145 217144==    by 0x2DEE0F: transformTopLevelStmt
(analyze.c:196)
==00:00:00:04.145 217144==    by 0x2DEE71: parse_analyze (analyze.c:116)
==00:00:00:04.145 217144==    by 0x55E69F: pg_analyze_and_rewrite
(postgres.c:691)
==00:00:00:04.145 217144==

The first bad commit is 1cff1b95.

On Sat, Jan 02, 2021 at 03:00:00PM +0000, PG Bug reporting form wrote:
> valgrind detects an invalid read:
> ==00:00:00:04.145 217144== Invalid read of size 8
> ==00:00:00:04.145 217144==    at 0x302CB7: makeDependencyGraphWalker
> (parse_cte.c:549)
> ==00:00:00:04.145 217144==    by 0x302EA1: makeDependencyGraph
> (parse_cte.c:439)
> ==00:00:00:04.145 217144==    by 0x304557: transformWithClause
> (parse_cte.c:176)
>
> The first bad commit is 1cff1b95.

The same kind of list manipulation is done in two places in
parse_cte.c, and there are extra ones in split_pathtarget_walker().  I
cannot reproduce that here, and I have just tried with different
optimization levels on HEAD and REL_13_STABLE.  Are you using specific
options for valgrind?
--
Michael

Hello Michael,
03.01.2021 08:15, Michael Paquier wrote:
> On Sat, Jan 02, 2021 at 03:00:00PM +0000, PG Bug reporting form wrote:
>> valgrind detects an invalid read:
>> ==00:00:00:04.145 217144== Invalid read of size 8
>> ==00:00:00:04.145 217144==    at 0x302CB7: makeDependencyGraphWalker
>> (parse_cte.c:549)
>> ==00:00:00:04.145 217144==    by 0x302EA1: makeDependencyGraph
>> (parse_cte.c:439)
>> ==00:00:00:04.145 217144==    by 0x304557: transformWithClause
>> (parse_cte.c:176)
>>
>> The first bad commit is 1cff1b95.
> The same kind of list manipulation is done in two places in
> parse_cte.c, and there are extra ones in split_pathtarget_walker().  I
> cannot reproduce that here, and I have just tried with different
> optimization levels on HEAD and REL_13_STABLE.  Are you using specific
> options for valgrind?
I'm using gcc 10.2.0 and valgrind-3.15.0, and building REL_13_STABLE
(c09f6882) with:
CPPFLAGS="-DUSE_VALGRIND -Og" ./configure --enable-debug
--enable-cassert && make -j8
Also I'm using a  patch [1] to inject valgrind into the `make check`
procedure. So it's possible to reproduce the issue with the extended
with.sql:
echo "
WITH RECURSIVE rec(x) AS (

    WITH outermost(x) AS (
      SELECT (
        WITH innermost as (SELECT 1)
        SELECT * FROM innermost
      )
    )
    SELECT * FROM outermost
)
SELECT * FROM rec;
" >>src/test/regress/sql/with.sql
make check

'make check' fails and src/test/regress/log/postmaster.log contains the
aforementioned valgrind message.

If it's still not reproduced for you, please let me know your
OS/compiler/valgrind version and I'll try it in your environment too.

[1]
https://www.postgresql.org/message-id/10dae4a1-e714-601d-7518-c19414255180%40gmail.com

Best regards,
Alexander

On Sun, Jan 03, 2021 at 09:00:00AM +0300, Alexander Lakhin wrote:
> I'm using gcc 10.2.0 and valgrind-3.15.0, and building REL_13_STABLE
> (c09f6882) with:
> CPPFLAGS="-DUSE_VALGRIND -Og" ./configure --enable-debug
> --enable-cassert && make -j8
> Also I'm using a  patch [1] to inject valgrind into the `make check`
> procedure.

Thanks, -DUSE_VALGRIND is the part that mattered here.  I can now
reproduce it.  I'll try to look at that more in details.
--
Michael