Re: The dangers of streaming across versions of glibc: A cautionary tale

On Wed, Aug  6, 2014 at 09:24:17PM +0000, Matthew Kelly wrote:
> The following is a real critical problem that we ran into here at TripAdvisor,
> but have yet figured out a clear way to mitigate.
>
> TL;DR:
> Streaming replicas—and by extension, base backups—can become dangerously broken
> when the source and target machines run slightly different versions of glibc.
>  Particularly, differences in strcoll and strcoll_l leave "corrupt" indexes on
> the slave.  These indexes are sorted out of order with respect to the strcoll
> running on the slave.  Because postgres is unaware of the discrepancy is uses
> these "corrupt" indexes to perform merge joins; merges rely heavily on the
> assumption that the indexes are sorted and this causes all the results of the
> join past the first poison pill entry to not be returned.  Additionally, if the
> slave becomes master, the "corrupt" indexes will in cases be unable to enforce
> uniqueness, but quietly allow duplicate values.

No surprise;  I have been expecting to hear about such breakage, and am
surprised we hear about it so rarely.  We really have no way of testing
for breakage either.  :-(

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +