Обсуждение: Thoughts on using Text::Template for our autogenerated code?
Is there a barrier to us using non-core perl modules, in this case Text::Template?
I think it would be a tremendous improvement in readability and maintainability over our current series of print statements, some multiline, some not.
The module itself works like this https://www.perlmonks.org/?node_id=33296
Some other digging around shows that the module has been around since 1996 (Perl5 was 1994) and hasn't had a feature update (or any update for that matter) since 2003. So it should meet our baseline 5.14 requirement, which came out in 2011.
I'm happy to proceed with a proof-of-concept so that people can see the costs/benefits, but wanted to first make sure it wasn't a total non-starter.
I think it would be a tremendous improvement in readability and maintainability over our current series of print statements, some multiline, some not.
The module itself works like this https://www.perlmonks.org/?node_id=33296
Some other digging around shows that the module has been around since 1996 (Perl5 was 1994) and hasn't had a feature update (or any update for that matter) since 2003. So it should meet our baseline 5.14 requirement, which came out in 2011.
I'm happy to proceed with a proof-of-concept so that people can see the costs/benefits, but wanted to first make sure it wasn't a total non-starter.
Corey Huinker <corey.huinker@gmail.com> writes:
> Is there a barrier to us using non-core perl modules, in this case
> Text::Template?
Use for what exactly?
I'd be hesitant to require such things to build from a tarball,
or to run regression tests. If it's used to build a generated file
that we include in tarballs, that might be workable ... but I'd bet
a good fraction of the buildfarm falls over (looks like all four of
my animals would), and you might get push-back from developers too.
> I think it would be a tremendous improvement in readability and
> maintainability over our current series of print statements, some
> multiline, some not.
I suspect it'd have to be quite a remarkable improvement to justify
adding a new required build tool ... but show us an example.
regards, tom lane
Hi, On 2023-03-30 13:06:46 -0400, Corey Huinker wrote: > Is there a barrier to us using non-core perl modules, in this case > Text::Template? I don't think we should have a hard build-time dependency on non-core perl modules. On some operating systems having to install such dependencies is quite painful (e.g. windows). > I think it would be a tremendous improvement in readability and > maintainability over our current series of print statements, some > multiline, some not. I think many of those could just be replaced by multi-line strings and/or here documents to get most of the win. Greetings, Andres Freund
> On 30 Mar 2023, at 19:06, Corey Huinker <corey.huinker@gmail.com> wrote:
> Some other digging around shows that the module has been around since 1996 (Perl5 was 1994) and hasn't had a feature
update(or any update for that matter) since 2003. So it should meet our baseline 5.14 requirement, which came out in
2011.
I assume you then mean tying this to 1.44 (or another version?), since AFAICT
there has been both features and bugfixes well after 2003?
https://metacpan.org/dist/Text-Template/changes
--
Daniel Gustafsson
On 2023-03-30 Th 15:06, Daniel Gustafsson wrote:
On 30 Mar 2023, at 19:06, Corey Huinker <corey.huinker@gmail.com> wrote:Some other digging around shows that the module has been around since 1996 (Perl5 was 1994) and hasn't had a feature update (or any update for that matter) since 2003. So it should meet our baseline 5.14 requirement, which came out in 2011.I assume you then mean tying this to 1.44 (or another version?), since AFAICT there has been both features and bugfixes well after 2003? https://metacpan.org/dist/Text-Template/changes
I don't think that's remotely a starter. Asking people to install an old and possibly buggy version of a perl module is not something we should do.
I think the barrier for this is pretty high. I try to keep module requirements for the buildfarm client pretty minimal, and this could affect a much larger group of people.
cheers
andrew
-- Andrew Dunstan EDB: https://www.enterprisedb.com
I think many of those could just be replaced by multi-line strings and/or here
documents to get most of the win.
I agree that a pretty good chunk of it can be done with here-docs, but template files do have attractive features (separation of concerns, syntax highlighting, etc) that made it worth asking.
I don't think that's remotely a starter. Asking people to install an old and possibly buggy version of a perl module is not something we should do.
I think the barrier for this is pretty high. I try to keep module requirements for the buildfarm client pretty minimal, and this could affect a much larger group of people.
Those are good reasons.
For those who already responded, are your concerns limited to the dependency issue? Would you have concerns about a templating library that was developed by us and included in-tree? I'm not suggesting suggesting we write one at this time, at least not until after we make a here-doc-ing pass first, but I want to understand the concerns before embarking on any refactoring.
For those who already responded, are your concerns limited to the dependency issue? Would you have concerns about a templating library that was developed by us and included in-tree? I'm not suggesting suggesting we write one at this time, at least not until after we make a here-doc-ing pass first, but I want to understand the concerns before embarking on any refactoring.
Hi, On 2023-03-30 17:15:20 -0400, Corey Huinker wrote: > For those who already responded, are your concerns limited to the > dependency issue? Would you have concerns about a templating library that > was developed by us and included in-tree? I'm not suggesting suggesting we > write one at this time, at least not until after we make a here-doc-ing > pass first, but I want to understand the concerns before embarking on any > refactoring. The dependency is/was my main issue. But I'm also somewhat doubtful that what we do warrants the use of a template library in the first place, but I could be convinced by a patch showing it to be a significant improvement. Greetings, Andres Freund
Andres Freund <andres@anarazel.de> writes:
> On 2023-03-30 17:15:20 -0400, Corey Huinker wrote:
>> For those who already responded, are your concerns limited to the
>> dependency issue? Would you have concerns about a templating library that
>> was developed by us and included in-tree? I'm not suggesting suggesting we
>> write one at this time, at least not until after we make a here-doc-ing
>> pass first, but I want to understand the concerns before embarking on any
>> refactoring.
> The dependency is/was my main issue. But I'm also somewhat doubtful that what
> we do warrants the use of a template library in the first place, but I could
> be convinced by a patch showing it to be a significant improvement.
Yeah, it's somewhat hard to believe that the cost/benefit ratio would be
attractive. But maybe you could mock up some examples of what the input
could look like, and get people on board (or not) before writing any
code.
regards, tom lane
On Fri, Mar 31, 2023 at 4:15 AM Corey Huinker <corey.huinker@gmail.com> wrote:
> For those who already responded, are your concerns limited to the dependency issue? Would you have concerns about a templating library that was developed by us and included in-tree?
Libraries (and abstractions in general) require some mental effort to interface with them (that also means debugging when the output fails to match expectations), not to mention maintenance cost. There has to be a compensating benefit in return. The cost-to-benefit ratio here seems unfavorable -- seems like inventing a machine that ties shoelaces, but we usually wear sandals.
Yeah, it's somewhat hard to believe that the cost/benefit ratio would be
attractive. But maybe you could mock up some examples of what the input
could look like, and get people on board (or not) before writing any
code.
tl;dr - I tried a few things, nothing that persuades myself let alone the community, but perhaps some ideas for the future.
I borrowed Bertrand's ongoing work for waiteventnames.* because that is what got me thinking about this in the first place. I considered a few different templating libraries:
There is no perl implementation of the golang template library (example of that here: https://blog.gopheracademy.com/advent-2017/using-go-templates/ ) that I could find.
Text::Template does not support loops, and as such it is no better than here-docs.
I borrowed Bertrand's ongoing work for waiteventnames.* because that is what got me thinking about this in the first place. I considered a few different templating libraries:
There is no perl implementation of the golang template library (example of that here: https://blog.gopheracademy.com/advent-2017/using-go-templates/ ) that I could find.
Text::Template does not support loops, and as such it is no better than here-docs.
Template Toolkit seems to do what we need, but it has a kitchen sink of dependencies that make it an unattractive option, so I didn't even attempt it.
HTML::Template has looping and if/then/else constructs, and it is a single standalone library. It also does a "separation of concerns" wherein you pass in parameter names and values, and some parameters can be for loops, which means you pass an arrayref of hashrefs that the template engine loops over. That's where the advantages stop, however. It is fairly verbose, and because it is HTML-centric it isn't very good about controlling whitespace, which leads to piling template directives onto the same line in order to avoid spurious newlines. As such I cannot recommend it.
My ideal template library would have text something like this:
My ideal template library would have text something like this:
[% loop events %][% $enum_value %]
[% if __first__ +%] = [%+ $inital_value %][% endif %]
[% if ! __last__ %],[% endif +%][% end loop %]
[% loop xml_blocks indent: relative,spaces,4 %]
<row>
<SomeElement attrib=[%attrib_val%]>[%element_body%]/>
</row>
[% end loop %]
[%+ means "leading whitespace matters", +%] means "trailing whitespace matters"
That pseudocode is a mix of ASP, HTML::Template. The special variables __first__ and __last__ refer to which iteration of the loop we are on. You would pass it a data structure like this:
That pseudocode is a mix of ASP, HTML::Template. The special variables __first__ and __last__ refer to which iteration of the loop we are on. You would pass it a data structure like this:
{events: [ { enum_value: "abc", initial_value: "def"}, ... { enum_value: "wuv", initial_value: "xyz" } ],xml_block: [ {attrib_val: "one", element_body: "two"} ]}
I did one initial pass with just converting printf statements to here-docs, and the results were pretty unsatisfying. It wasn't really possible to "see" the output files take shape.
My next attempt was to take the "separation of concerns" code from the HTML::Template version, constructing the nested data structure of resolved output values, and then iterating over that once per output file. This resulted in something cleaner, partly because we're only writing one file type at a time, partly because the interpolated variables have names much closer to their output meaning.
My next attempt was to take the "separation of concerns" code from the HTML::Template version, constructing the nested data structure of resolved output values, and then iterating over that once per output file. This resulted in something cleaner, partly because we're only writing one file type at a time, partly because the interpolated variables have names much closer to their output meaning.
In doing this, it occurred to me that a lot of this effort is in getting the code to conform to our own style guide, at the cost of the generator code being less readable. What if we wrote the generator and formatted the code in a way that made sense for the generator, and then pgindented it. That's not the workflow right now, but perhaps it could be.
Conclusions:
Conclusions:
- There is no "good enough" template engine that doesn't require big changes in dependencies.
- pgindent will not save you from a run-on sentence, like putting all of a typdef enum values on one line
- There is some clarity value in either separating input processing from the output processing, or making the input align more closely with the outputs
- There is some clarity value in either separating input processing from the output processing, or making the input align more closely with the outputs
- Fiddling with indentation and spacing detracts from legibility no matter what method is used.
- here docs are basically ok but they necessarily confuse output indentation with code indentation. it is possible to de-indent them them with <<~ but that's a 5.25+ feature.
- Any of these principles can be applied at any time, with no overhaul required.
"sorted-" is the slightly modified version of Bertrand's code.
"eof-as-is-" is a direct conversion of the original but using here-docs.
"heredoc-fone-file-at-a-time-" first generates an output-friendly data structure
"needs-pgindent-" is what is possible if we format for our own readability and make pgindent fix the output, though it was not a perfect output match