Re: Unfiltered server logs routing via a new elog hook or existing emit_log_hook bypassing log_min_message check

On Mon, May 2, 2022 at 6:32 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>
> Hi,
>
> On Mon, May 02, 2022 at 05:11:34PM +0530, Bharath Rupireddy wrote:
> >
> > Currently the emit_log_hook gets called only for the log messages of
> > type <= log_min_message i.e when edata->output_to_server is true [1],
> > which means that I can't use an implementation of emit_log_hook to
> > just intercept, say, all DEBUGX messages without interrupting the
> > actual server logs flow and changing the log_min_message.
> > [...]
> > If I had
> > postgres elog hook, say emit_unfiltered_log_hook [2], I can basically
> > write an external module (with a bunch of GUCs say log_level to route,
> > place to store the logs, even an option to filter logs based on text
> > say logs containing word 'replication', max disk space that these
> > routed logs would occupy etc.) implementing emit_unfiltered_log_hook
> > to just route the interested logs to a cheaper storage (for debugging
> > purposes), after analysis I can disable the external module and blow
> > away the routed logs.
>
> Unless I'm missing something you can already do all of that with the current
> hook, since as mentioned in the comment above the hook can disable the server's
> logging:
>
>          * Call hook before sending message to log.  The hook function is allowed
>          * to turn off edata->output_to_server, so we must recheck that afterward.
>
> So you can configure your server with a very verbose log_min_message, and have
> the same setting in your own extension to disable output_to_server after its
> own processing is done.

No. The emit_log_hook isn't called for all the log messages, but only
when output_to_server = true which means, say my log_min_messages is
'WARNING', the hook isn't called for the messages say elevel above it
(NOTICE, INFO, DEBUGX).

Regards,
Bharath Rupireddy.