Обсуждение: pgsql: Build all Flex files standalone
Build all Flex files standalone The proposed Meson build system will need a way to ignore certain generated files in order to coexist with the autoconf build system, and C files generated by Flex which are #include'd into .y files make this more difficult. In similar vein to 72b1e3a21, arrange for all Flex C files to compile to their own .o targets. Reviewed by Andres Freund Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de Discussion: https://www.postgresql.org/message-id/CAFBsxsF8Gc2StS3haXofshHCzqNMRXiSxvQEYGwnFsTmsdwNeg@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/dac048f71ebbcf2f980d280711f8ff8001331c5d Modified Files -------------- contrib/cube/.gitignore | 1 + contrib/cube/Makefile | 16 +++++---- contrib/cube/cube.c | 6 ++-- contrib/cube/cubedata.h | 6 ++-- contrib/cube/cubeparse.y | 6 +--- contrib/cube/cubescan.l | 44 ++++++++++++++--------- contrib/seg/.gitignore | 1 + contrib/seg/Makefile | 15 +++++--- contrib/seg/segparse.y | 3 -- contrib/seg/segscan.l | 28 +++++++++------ src/backend/Makefile | 7 ++-- src/backend/bootstrap/.gitignore | 1 + src/backend/bootstrap/Makefile | 11 ++++-- src/backend/bootstrap/bootparse.y | 2 -- src/backend/bootstrap/bootscanner.l | 60 +++++++++++++++++-------------- src/backend/replication/.gitignore | 2 ++ src/backend/replication/Makefile | 22 +++++++++--- src/backend/replication/repl_gram.y | 2 -- src/backend/replication/repl_scanner.l | 31 ++++++++++------ src/backend/replication/syncrep_gram.y | 2 -- src/backend/replication/syncrep_scanner.l | 21 +++++++---- src/backend/utils/adt/.gitignore | 1 + src/backend/utils/adt/Makefile | 11 ++++-- src/backend/utils/adt/jsonpath_gram.y | 27 +------------- src/backend/utils/adt/jsonpath_internal.h | 32 +++++++++++++++++ src/backend/utils/adt/jsonpath_scan.l | 29 +++++++++------ src/backend/utils/misc/Makefile | 5 +-- src/backend/utils/misc/guc-file.l | 8 +++-- src/backend/utils/misc/guc.c | 2 -- src/bin/pgbench/.gitignore | 1 + src/bin/pgbench/Makefile | 13 +++++-- src/bin/pgbench/exprparse.y | 15 -------- src/bin/pgbench/exprscan.l | 12 ++++++- src/test/isolation/.gitignore | 1 + src/test/isolation/Makefile | 15 +++++--- src/test/isolation/specparse.y | 2 -- src/test/isolation/specscanner.l | 28 ++++++++++----- src/tools/pginclude/headerscheck | 8 +++++ 38 files changed, 307 insertions(+), 190 deletions(-)
Crake fails with headerscheck: Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: Sep 04 01:56:01 /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: fatal error: jsonpath_gram.h: No such file or directory Sep 04 01:56:01 26 | #include "jsonpath_gram.h" Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ Sep 04 01:56:01 compilation terminated. Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 cpluspluscheck: Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: Sep 04 01:57:19 /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: fatal error: jsonpath_gram.h: No such file or directory Sep 04 01:57:19 26 | #include "jsonpath_gram.h" Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ I don't get the same in an already-built tree. I also don't see an exception for gramparse.h, which I imagine would have the same issue? -- John Naylor EDB: http://www.enterprisedb.com
The second failure is on drongo (MSVC):
"c:\\prog\\bf\\root\\HEAD\\pgsql.build\\pgsql.sln" (default target) (1) ->
"c:\\prog\\bf\\root\\HEAD\\pgsql.build\\isolationtester.vcxproj"
(default target) (13) ->
(Link target) ->
specparse.obj : error LNK2019: unresolved external symbol spec_yylex
referenced in function spec_yyparse
[c:\\prog\\bf\\root\\HEAD\\pgsql.build\\isolationtester.vcxproj]
specparse.obj : error LNK2019: unresolved external symbol
spec_yyerror referenced in function spec_yyparse
[c:\\prog\\bf\\root\\HEAD\\pgsql.build\\isolationtester.vcxproj]
.\\Release\\isolationtester\\isolationtester.exe : fatal error
LNK1120: 2 unresolved externals
[c:\\prog\\bf\\root\\HEAD\\pgsql.build\\isolationtester.vcxproj]
For this, Mkvcbuild.pm specifies specparse.c explicitly, so maybe
something like the following is needed:
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -362,6 +362,7 @@ sub mkvcbuild
$isolation_tester->AddFile('src/test/isolation/specparse.y');
$isolation_tester->AddFile('src/test/isolation/specscanner.l');
$isolation_tester->AddFile('src/test/isolation/specparse.c');
+ $isolation_tester->AddFile('src/test/isolation/specscanner.c');
$isolation_tester->AddIncludeDir('src/test/isolation');
--
John Naylor
EDB: http://www.enterprisedb.com
On Sun, Sep 4, 2022 at 4:36 PM John Naylor <john.naylor@enterprisedb.com> wrote:
>
> For this, Mkvcbuild.pm specifies specparse.c explicitly, so maybe
> something like the following is needed:
>
> --- a/src/tools/msvc/Mkvcbuild.pm
> +++ b/src/tools/msvc/Mkvcbuild.pm
> @@ -362,6 +362,7 @@ sub mkvcbuild
> $isolation_tester->AddFile('src/test/isolation/specparse.y');
> $isolation_tester->AddFile('src/test/isolation/specscanner.l');
> $isolation_tester->AddFile('src/test/isolation/specparse.c');
> + $isolation_tester->AddFile('src/test/isolation/specscanner.c');
> $isolation_tester->AddIncludeDir('src/test/isolation');
This gets CI green (tip o' the hat to Thomas Munro for off-list advice
on getting that working), so pushed.
--
John Naylor
EDB: http://www.enterprisedb.com
On 2022-09-04 Su 02:19, John Naylor wrote: > Crake fails with > > headerscheck: > Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: > Sep 04 01:56:01 > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > fatal error: jsonpath_gram.h: No such file or directory > Sep 04 01:56:01 26 | #include "jsonpath_gram.h" > Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ > Sep 04 01:56:01 compilation terminated. > Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 > cpluspluscheck: > Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: > Sep 04 01:57:19 > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > fatal error: jsonpath_gram.h: No such file or directory > Sep 04 01:57:19 26 | #include "jsonpath_gram.h" > Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ > > I don't get the same in an already-built tree. I also don't see an > exception for gramparse.h, which I imagine would have the same issue? Probably happens because crake does vpath builds. I think you're going to have to set up EXTRAINCLUDES for it - see near the bottom of the headerscheck script. Possibly something like src/backend/utils/adt/*) EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Hi,
On 2022-09-04 18:04:13 +0700, John Naylor wrote:
> On Sun, Sep 4, 2022 at 4:36 PM John Naylor <john.naylor@enterprisedb.com> wrote:
> >
> > For this, Mkvcbuild.pm specifies specparse.c explicitly, so maybe
> > something like the following is needed:
> >
> > --- a/src/tools/msvc/Mkvcbuild.pm
> > +++ b/src/tools/msvc/Mkvcbuild.pm
> > @@ -362,6 +362,7 @@ sub mkvcbuild
> > $isolation_tester->AddFile('src/test/isolation/specparse.y');
> > $isolation_tester->AddFile('src/test/isolation/specscanner.l');
> > $isolation_tester->AddFile('src/test/isolation/specparse.c');
> > + $isolation_tester->AddFile('src/test/isolation/specscanner.c');
> > $isolation_tester->AddIncludeDir('src/test/isolation');
>
> This gets CI green (tip o' the hat to Thomas Munro for off-list advice
> on getting that working), so pushed.
It doesn't seem to have gotten all of CI green, cpluspluscheck fails:
https://cirrus-ci.com/task/6561938334285824
I think this just needs a copy the logic you added to headerscheck also to
cpluspluscheck (and yes, we should merge these files one of these days).
Greetings,
Andres Freund
Hi,
On 2022-09-04 13:19:56 +0700, John Naylor wrote:
> Crake fails with
>
> headerscheck:
> Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2:
> Sep 04 01:56:01
> /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10:
> fatal error: jsonpath_gram.h: No such file or directory
> Sep 04 01:56:01 26 | #include "jsonpath_gram.h"
> Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~
> Sep 04 01:56:01 compilation terminated.
> Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1
> cpluspluscheck:
> Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3:
> Sep 04 01:57:19
> /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10:
> fatal error: jsonpath_gram.h: No such file or directory
> Sep 04 01:57:19 26 | #include "jsonpath_gram.h"
> Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~
>
> I don't get the same in an already-built tree. I also don't see an
> exception for gramparse.h, which I imagine would have the same issue?
It doesn't happen for gramparse.h because gram.h is copied to
src/include/parser (see src/backend/Makefile), and included as parser/gram.h.
I think this could be addressed by always adding something like
-iquote $builddir/$(dirname "$f") -iquote $srcdir/$(dirname "$f") \
to cpluspluscheck/headerscheck.
Greetings,
Andres Freund
On Sun, Sep 4, 2022 at 8:34 PM Andrew Dunstan <andrew@dunslane.net> wrote: > Probably happens because crake does vpath builds. I think you're going > to have to set up EXTRAINCLUDES for it - see near the bottom of the > headerscheck script. Possibly something like > > > src/backend/utils/adt/*) > > EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; Okay, I'll look at this and the cpp header check within 24 hours. -- John Naylor EDB: http://www.enterprisedb.com
On Sun, Sep 4, 2022 at 8:34 PM Andrew Dunstan <andrew@dunslane.net> wrote: > > > On 2022-09-04 Su 02:19, John Naylor wrote: > > Crake fails with > > > > headerscheck: > > Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: > > Sep 04 01:56:01 > > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > > fatal error: jsonpath_gram.h: No such file or directory > > Sep 04 01:56:01 26 | #include "jsonpath_gram.h" > > Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ > > Sep 04 01:56:01 compilation terminated. > > Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 > > cpluspluscheck: > > Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: > > Sep 04 01:57:19 > > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > > fatal error: jsonpath_gram.h: No such file or directory > > Sep 04 01:57:19 26 | #include "jsonpath_gram.h" > > Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ > > > > I don't get the same in an already-built tree. I also don't see an > > exception for gramparse.h, which I imagine would have the same issue? > > > > Probably happens because crake does vpath builds. I think you're going > to have to set up EXTRAINCLUDES for it - see near the bottom of the > headerscheck script. Possibly something like > > > src/backend/utils/adt/*) > > EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; I just tried a vpath build and the failing check doesn't reproduce for me there. Any ideas? -- John Naylor EDB: http://www.enterprisedb.com
Hi, On September 5, 2022 8:33:37 PM PDT, John Naylor <john.naylor@enterprisedb.com> wrote: >On Sun, Sep 4, 2022 at 8:34 PM Andrew Dunstan <andrew@dunslane.net> wrote: >> >> >> On 2022-09-04 Su 02:19, John Naylor wrote: >> > Crake fails with >> > >> > headerscheck: >> > Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: >> > Sep 04 01:56:01 >> > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: >> > fatal error: jsonpath_gram.h: No such file or directory >> > Sep 04 01:56:01 26 | #include "jsonpath_gram.h" >> > Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ >> > Sep 04 01:56:01 compilation terminated. >> > Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 >> > cpluspluscheck: >> > Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: >> > Sep 04 01:57:19 >> > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: >> > fatal error: jsonpath_gram.h: No such file or directory >> > Sep 04 01:57:19 26 | #include "jsonpath_gram.h" >> > Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ >> > >> > I don't get the same in an already-built tree. I also don't see an >> > exception for gramparse.h, which I imagine would have the same issue? >> >> >> >> Probably happens because crake does vpath builds. I think you're going >> to have to set up EXTRAINCLUDES for it - see near the bottom of the >> headerscheck script. Possibly something like >> >> >> src/backend/utils/adt/*) >> >> EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; > >I just tried a vpath build and the failing check doesn't reproduce for >me there. Any ideas? Presumably it only happens if you previously triggered the parsers to be generated. Andres -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On Tue, Sep 6, 2022 at 10:54 AM Andres Freund <andres@anarazel.de> wrote: > >I just tried a vpath build and the failing check doesn't reproduce for > >me there. Any ideas? > > Presumably it only happens if you previously triggered the parsers to be generated. If you attempt to run the check before building, the first thing you get is fatal error: utils/errcodes.h: No such file or directory so not sure what you mean. -- John Naylor EDB: http://www.enterprisedb.com
John Naylor <john.naylor@enterprisedb.com> writes:
> On Tue, Sep 6, 2022 at 10:54 AM Andres Freund <andres@anarazel.de> wrote:
>> Presumably it only happens if you previously triggered the parsers to be generated.
> If you attempt to run the check before building, the first thing you get is
> fatal error: utils/errcodes.h: No such file or directory
> so not sure what you mean.
crake is still unhappy:
Sep 06 00:09:55 In file included from /tmp/headerscheck.AwLxya/test.c:2:
Sep 06 00:09:55 /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: fatal
error:jsonpath_gram.h: No such file or directory
Sep 06 00:09:55 26 | #include "jsonpath_gram.h"
Sep 06 00:09:55 | ^~~~~~~~~~~~~~~~~
I wonder if that file is not getting installed in a directory that
this test expects to search.
regards, tom lane
On Tue, Sep 6, 2022 at 11:49 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > > crake is still unhappy: > > Sep 06 00:09:55 In file included from /tmp/headerscheck.AwLxya/test.c:2: > Sep 06 00:09:55 /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: fatal error:jsonpath_gram.h: No such file or directory > Sep 06 00:09:55 26 | #include "jsonpath_gram.h" > Sep 06 00:09:55 | ^~~~~~~~~~~~~~~~~ > > I wonder if that file is not getting installed in a directory that > this test expects to search. There are two proposed fixes upthread, which I will copy below, but if I can't reproduce locally, I don't have much confidence in which one is the way to go to get crake green again: Andrew: > Probably happens because crake does vpath builds. I think you're going > to have to set up EXTRAINCLUDES for it - see near the bottom of the > headerscheck script. Possibly something like > > > src/backend/utils/adt/*) > > EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; Andres: > It doesn't happen for gramparse.h because gram.h is copied to > src/include/parser (see src/backend/Makefile), and included as parser/gram.h. > > I think this could be addressed by always adding something like > -iquote $builddir/$(dirname "$f") -iquote $srcdir/$(dirname "$f") \ > to cpluspluscheck/headerscheck. -- John Naylor EDB: http://www.enterprisedb.com
I see why I couldn't reproduce: I neglected to add jsonpath_gram.h to the backend's maintainer-clean so it failed to be deleted. Working on the fix now. -- John Naylor EDB: http://www.enterprisedb.com