Обсуждение: Re: cached plan must not change result type

 forwarded to pgjdbc,  we want to understand why JDBC failed to reexecute the SQL instead of throw error out. Like this document https://jdbc.postgresql.org/documentation/server-prepare/#re-execution-of-failed-statements .

 

protected final void execute(CachedQuery cachedQuery,
      @Nullable ParameterList queryParameters, int flags)
      throws SQLException {
    try {
      executeInternal(cachedQuery, queryParameters, flags);
    } catch (SQLException e) {
      // Don't retry composite queries as it might get partially executed
      if (cachedQuery.query.getSubqueries() != null                                 <<< no idea how this  cachedQuery.query.getSubqueries() != null
          || !connection.getQueryExecutor().willHealOnRetry(e)) {
        throw e;
      }
      cachedQuery.query.close();
      // Execute the query one more time
      executeInternal(cachedQuery, queryParameters, flags);
    }
  }

cachedQuery.query.getSubqueries() != null, how this code decide composite queries  here ?   that mean some query having subquery or having many JOIN or LEFT JOINs like  select .... A left join B ...

Thanks,

James 

Dave Cramer <davecramer@postgres.rocks> writes:
> This is really an issue that needs to be solved in the backend. The error
> is coming from PostgreSQL and what should happen is that when you alter a
> table that a server prepared statement relies on the backend should send a
> message to tell us that all of the prepared statements that rely on are now
> invalid and we can reprepare them.

This is something that can't change without a wire protocol change.
There is nothing in the protocol that allows the backend to send out
a message like "oh, that Describe I sent you awhile back?  It might
be a lie now" at random times. 

Also, what do you want to do about
race conditions --- that is, what if you fire off an Execute only
to find that one of those messages was already in flight to you?

A non-racy way to handle it might be for Bind/Execute to refuse to
run the query if its output has changed since the last Describe,
which we could check after acquiring table locks during Bind.
But we'd want to define "refuse" in a way that doesn't abort the
transaction, and that's a concept that doesn't exist in the
protocol at all.

On Fri, 29 Mar 2024 at 19:42, James Pang <jamespang886@gmail.com> wrote:

   we did DDL "alter table ... alter column increase varchar(512) to varchar(1024)", after that done, hours later, new query select on this table still failed this error.  From this https://jdbc.postgresql.org/documentation/server-prepare/#re-execution-of-failed-statements , looks like pgjdbc try to handle this exception and retry, but in our case, it did not happen.  Could you direct me how to make this retry work? we only want new transactions,queries work that after the DDL changes. 

protected boolean willHealViaReparse(SQLException e) {
    if (e == null || e.getSQLState() == null) {
      return false;
    }

    // "prepared statement \"S_2\" does not exist"
    if (PSQLState.INVALID_SQL_STATEMENT_NAME.getState().equals(e.getSQLState())) {
      return true;
    }
    if (!PSQLState.NOT_IMPLEMENTED.getState().equals(e.getSQLState())) {
      return false;
    }

    if (!(e instanceof PSQLException)) {
      return false;
    }

    PSQLException pe = (PSQLException) e;

    ServerErrorMessage serverErrorMessage = pe.getServerErrorMessage();
    if (serverErrorMessage == null) {
      return false;
    }
    // "cached plan must not change result type"
    String routine = serverErrorMessage.getRoutine();
    return "RevalidateCachedQuery".equals(routine) // 9.2+
        || "RevalidateCachedPlan".equals(routine); // <= 9.1
  }

This only works if there was no transaction.

https://github.com/pgjdbc/pgjdbc/blob/0fbd31d43b1013829da3a8f21120d8bb94df803d/pgjdbc/src/main/java/org/postgresql/core/QueryExecutorBase.java#L442 

Dave

Thanks,

 

1) turn off server side prepared statements with prepareThreshold=0

       with this threshold=0, if it's still possible to avoid parsing, planning from Postgresql server side, like set plan_cache_mode=force_generic_plan or with default "auto" mode.  Postgresql server can make the plan cached and reused.

2) change varchar(n) to varchar or text, then restart application, so avoid future similar increase varchar(n) changes.

      it's safe to performance or optimizer ,right? 

Thanks,

James

Laurenz Albe <laurenz.albe@cybertec.at> 於 2024年3月30日週六 下午10:06寫道：

On Sat, 2024-03-30 at 08:27 -0400, Dave Cramer wrote:
> > On Fri, 29 Mar 2024 at 19:42, James Pang <jamespang886@gmail.com> wrote:
> > > we did DDL "alter table ... alter column increase varchar(512) to varchar(1024)",
> > > after that done, hours later, new query select on this table still failed this error.
> > > From this https://jdbc.postgresql.org/documentation/server-prepare/#re-execution-of-failed-statements ,
> > > looks like pgjdbc try to handle this exception and retry, but in our case, it did not happen.
> > > Could you direct me how to make this retry work?
> > > we only want new transactions,queries work that after the DDL changes. 
>
> I think the best option for you is to turn off server side prepared statements with prepareThreshold=0

An alternative option is to take downtime for DDL and restart the application.
Or to chppse the appropriate data type right away: in your case, that would
have been "text".

Yours,
Laurenz Albe