Re: Middleware Messages for FE/BE

Hi,

On 8/19/21 1:13 PM, Simon Riggs wrote:
> We need to be able to send the middleware a message. Replies can be
> sent as NoticeResponse, if needed, which already exists.
> 

Yeah, but you need the client driver - of all languages - to understand 
that notice. And, if different middleware uses different codes then the 
problem is just pushed to the client;

Java: M|12|0 (Do you support failover ?) MW1: N|6|-|0 (meaning: yes)
Rust: M|12|0 (Do you support load balancing ?) MW2: N|6|-|0 (meaning: no)

Should the middleware guess the client driver based on the startup 
message ? Or are you thinking about having a class identifier group for 
each client driver and then a massive "switch/case" inside each middleware ?

>> Once you have established what is supported and what isn't you need the
>> client driver (libpq is the very easy part) - so lets include Java, C#,
>> Rust, golang, ... - to understand their environment to trigger the
>> correct message in the configured scenario. F.ex. how is multiple
>> application clusters connecting to the same middleware instance
>> (connection pool) going to coordinate their 'M' message ?
> 
> Each component can speak to the middleware to discover that, if it
> wishes. The middleware can coordinate.
> 

So, application cluster 1 (written in Java) detects that a failover is 
needed, and application cluster 2..N (written in Rust, C# and golang) 
should detect that mid-transaction - or rollback and send their 'M' for 
the same thing ?

> I want to empower all future middleware, not just support one. Perhaps
> we can take code or ideas from pgagroal and put it in core? Please
> explain the binary protocol some more.
> 

There is nothing fancy there (see management.c) - so the official 
protocol should be the focus.

>> Could you expand on typical scenarios that you want to see implemented ?
> 
> I see this as a generalized interface that can be used for a great many things.
> * Failover managers
> * Load Balancers
> * Routing agents
> * Things I haven't thought of yet, but others may have
> etc..
>

Moving control to the client will make some of this harder. Maybe 
PeterE, Andrey and Tatsuo-san have additional feedback based on their 
experience.

> We are currently limited by the messages we can send efficiently.
> 

There are a lot of good ideas for the PostgreSQL protocol v4 already.

Best regards,
  Jesper