Re: [PATCH] Log CSV by default

Le 03/12/2018 à 19:25, David Fetter a écrit :
> On Mon, Dec 03, 2018 at 07:18:31PM +0100, Gilles Darold wrote:
>> Le 30/11/2018 à 19:53, David Fetter a écrit :
>>> This makes it much simpler for computers to use the logs while not
>>> making it excessively difficult for humans to use them.
>> I'm also not very comfortable with this change. For a pgBadger point
>> of view I don't know an existing library to parse csv log in
>> multi-process mode. The only way I know is to split the file or load
>> chunks in memory which is not really recommended with huge log
>> files. I am not saying that this is not possible to have a Perl CSV
>> library allowing multi-process but this require some additional days
>> of work.
> How about other structured log formats? Would one for JSON work
> better?


pgBadger is able to parse jsonlog
(https://github.com/michaelpq/pg_plugins/tree/master/jsonlog) output in
multi-process mode because it do not use an external library to parse
the json. In this minimalist implementation I assume that each log line
is a full json object. Honestly I do not have enough feedback on this
format to be sure that my basic jsonlog parser is enough, maybe it is
not. What is really annoying for a log parser is the preservation of
newline character in queries, if the format outputs each atomic log
messages on a single line each, this is ideal.


>
>> Also at this time I have not received any feature request for that,
>> maybe the use of csvlog format is not so common.
> Part of the reason it's uncommon is that it's not the default we ship
> with, and defaults influence this very much.

I agree. I also think that syslog or stderr are more human readable than
csvlog or jsonlog, which have probably contributed to their large
adoption too.



--
Gilles Darold
http://www.darold.net/