Обсуждение: Debian PostgreSQL 15 pg_createcluster - could not open collator for locale
I am having trouble with a fresh installation of PostgreSQL 15 from the apt repository 'http://apt.postgresql.org/pub/repos/apt bullseye-pgdg main'.
After installing the packages the following locations were empty:
- /etc/postgresql/15/main/
- /var/lib/postgresql/
Searching for information suggested running 'sudo pg_createcluster 15 main --start' to initialise the database.
The process failed with the message 'could not open collator for locale "yue-Hans"'.
Following is the stdout for the process. Any suggestions welcome
Matt.
Creating new PostgreSQL cluster 15/main ...
/usr/lib/postgresql/15/bin/initdb -D /var/lib/postgresql/15/main --auth-local peer --auth-host scram-sha-256 --no-instructions
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_AU.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/15/main ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Australia/Canberra
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... 2023-03-25 11:50:31.124 AEDT [12104] FATAL: could not open collator for locale "yue-Hans": U_INVALID_FORMAT_ERROR
2023-03-25 11:50:31.124 AEDT [12104] STATEMENT: SELECT pg_import_system_collations('pg_catalog');
child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/postgresql/15/main"
Error: initdb failed
Re: Debian PostgreSQL 15 pg_createcluster - could not open collator for locale
> On 25/03/2023 01:55 CET Matt Gibbins <matt_gibbins@fastmail.com.au> wrote: > > I am having trouble with a fresh installation of PostgreSQL 15 from the apt > repository 'http://apt.postgresql.org/pub/repos/apt bullseye-pgdg main'. > After installing the packages the following locations were empty: > * /etc/postgresql/15/main/ > * /var/lib/postgresql/ > Searching for information suggested running 'sudo pg_createcluster 15 main --start' > to initialise the database. > The process failed with the message 'could not open collator for locale "yue-Hans"'. > > Following is the stdout for the process. Any suggestions welcome > Matt. > > ------------------------------ > Creating new PostgreSQL cluster 15/main ... > /usr/lib/postgresql/15/bin/initdb -D /var/lib/postgresql/15/main --auth-local peer --auth-host scram-sha-256 --no-instructions > The files belonging to this database system will be owned by user "postgres". > This user must also own the server process. > > The database cluster will be initialized with locale "en_AU.UTF-8". > The default database encoding has accordingly been set to "UTF8". > The default text search configuration will be set to "english". > > Data page checksums are disabled. > > fixing permissions on existing directory /var/lib/postgresql/15/main ... ok > creating subdirectories ... ok > selecting dynamic shared memory implementation ... posix > selecting default max_connections ... 100 > selecting default shared_buffers ... 128MB > selecting default time zone ... Australia/Canberra > creating configuration files ... ok > running bootstrap script ... ok > performing post-bootstrap initialization ... 2023-03-25 11:50:31.124 AEDT [12104] FATAL: could not open collator for locale"yue-Hans": U_INVALID_FORMAT_ERROR > 2023-03-25 11:50:31.124 AEDT [12104] STATEMENT: SELECT pg_import_system_collations('pg_catalog'); > > child process exited with exit code 1 > initdb: removing contents of data directory "/var/lib/postgresql/15/main" > Error: initdb failed Looks like an issue with libicu. What version is installed? dpkg -l 'libicu*' ls -l /usr/lib/x86_64-linux-gnu/libicu* -- Erik
Re: Debian PostgreSQL 15 pg_createcluster - could not open collator for locale
Erik Thanks. The version currently installed as identified from dpkg -l is. Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-==================================== ii libicu67:amd64 67.1-7 amd64 International Components for Unicode The ls command returns. lrwxrwxrwx 1 root root 18 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicudata.so.67 -> libicudata.so.67.1 -rw-r--r-- 1 root root 28407344 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicudata.so.67.1 lrwxrwxrwx 1 root root 18 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicui18n.so.67 -> libicui18n.so.67.1 -rw-r--r-- 1 root root 3160560 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicui18n.so.67.1 lrwxrwxrwx 1 root root 16 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicuio.so.67 -> libicuio.so.67.1 -rw-r--r-- 1 root root 55456 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicuio.so.67.1 lrwxrwxrwx 1 root root 18 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicutest.so.67 -> libicutest.so.67.1 -rw-r--r-- 1 root root 81400 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicutest.so.67.1 lrwxrwxrwx 1 root root 16 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicutu.so.67 -> libicutu.so.67.1 -rw-r--r-- 1 root root 211880 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicutu.so.67.1 lrwxrwxrwx 1 root root 16 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicuuc.so.67 -> libicuuc.so.67.1 -rw-r--r-- 1 root root 1988784 Jul 1 2021 /usr/lib/x86_64-linux-gnu/libicuuc.so.67.1 Information obtained from 'apt-cache policy libicu67'. libicu67: Installed: 67.1-7 Candidate: 67.1-7 Version table: *** 67.1-7 500 500 http://mirror.linux.org.au/debian bullseye/main amd64 Packages 100 /var/lib/dpkg/status Matt On 26/3/23 03:39, Erik Wienhold wrote: >> On 25/03/2023 01:55 CET Matt Gibbins <matt_gibbins@fastmail.com.au> wrote: >> >> I am having trouble with a fresh installation of PostgreSQL 15 from the apt >> repository 'http://apt.postgresql.org/pub/repos/apt bullseye-pgdg main'. >> After installing the packages the following locations were empty: >> * /etc/postgresql/15/main/ >> * /var/lib/postgresql/ >> Searching for information suggested running 'sudo pg_createcluster 15 main --start' >> to initialise the database. >> The process failed with the message 'could not open collator for locale "yue-Hans"'. >> >> Following is the stdout for the process. Any suggestions welcome >> Matt. >> >> ------------------------------ >> Creating new PostgreSQL cluster 15/main ... >> /usr/lib/postgresql/15/bin/initdb -D /var/lib/postgresql/15/main --auth-local peer --auth-host scram-sha-256 --no-instructions >> The files belonging to this database system will be owned by user "postgres". >> This user must also own the server process. >> >> The database cluster will be initialized with locale "en_AU.UTF-8". >> The default database encoding has accordingly been set to "UTF8". >> The default text search configuration will be set to "english". >> >> Data page checksums are disabled. >> >> fixing permissions on existing directory /var/lib/postgresql/15/main ... ok >> creating subdirectories ... ok >> selecting dynamic shared memory implementation ... posix >> selecting default max_connections ... 100 >> selecting default shared_buffers ... 128MB >> selecting default time zone ... Australia/Canberra >> creating configuration files ... ok >> running bootstrap script ... ok >> performing post-bootstrap initialization ... 2023-03-25 11:50:31.124 AEDT [12104] FATAL: could not open collator forlocale "yue-Hans": U_INVALID_FORMAT_ERROR >> 2023-03-25 11:50:31.124 AEDT [12104] STATEMENT: SELECT pg_import_system_collations('pg_catalog'); >> >> child process exited with exit code 1 >> initdb: removing contents of data directory "/var/lib/postgresql/15/main" >> Error: initdb failed > Looks like an issue with libicu. What version is installed? > > dpkg -l 'libicu*' > ls -l /usr/lib/x86_64-linux-gnu/libicu* > > -- > Erik
Re: Debian PostgreSQL 15 pg_createcluster - could not open collator for locale
> On 26/03/2023 00:06 CET Matt Gibbins <matt_gibbins@fastmail.com.au> wrote: > > The version currently installed as identified from dpkg -l is. > > Desired=Unknown/Install/Remove/Purge/Hold > | > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend > |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) > ||/ Name Version Architecture Description > > +++-==============-============-============-==================================== > ii libicu67:amd64 67.1-7 amd64 International Components > for Unicode > > The ls command returns. > > lrwxrwxrwx 1 root root 18 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicudata.so.67 -> libicudata.so.67.1 > -rw-r--r-- 1 root root 28407344 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicudata.so.67.1 > lrwxrwxrwx 1 root root 18 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicui18n.so.67 -> libicui18n.so.67.1 > -rw-r--r-- 1 root root 3160560 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicui18n.so.67.1 > lrwxrwxrwx 1 root root 16 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicuio.so.67 -> libicuio.so.67.1 > -rw-r--r-- 1 root root 55456 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicuio.so.67.1 > lrwxrwxrwx 1 root root 18 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicutest.so.67 -> libicutest.so.67.1 > -rw-r--r-- 1 root root 81400 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicutest.so.67.1 > lrwxrwxrwx 1 root root 16 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicutu.so.67 -> libicutu.so.67.1 > -rw-r--r-- 1 root root 211880 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicutu.so.67.1 > lrwxrwxrwx 1 root root 16 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicuuc.so.67 -> libicuuc.so.67.1 > -rw-r--r-- 1 root root 1988784 Jul 1 2021 > /usr/lib/x86_64-linux-gnu/libicuuc.so.67.1 > > Information obtained from 'apt-cache policy libicu67'. > > libicu67: > Installed: 67.1-7 > Candidate: 67.1-7 > Version table: > *** 67.1-7 500 > 500 http://mirror.linux.org.au/debian bullseye/main amd64 Packages > 100 /var/lib/dpkg/status Same output as for me. Would you please build icu-test from the following source and run it with locale "yue-Hans". Let's see if ucol_open fails with U_INVALID_FORMAT_ERROR in this case. There are only two functions in Postgres that call ucol_open and emit error message "could not open collator for locale" for ICU >=54: initdb.c:check_icu_locale pg_locale.c:pg_ucol_open (called by pg_import_system_collations) File icu-test.c: #include <stdio.h> #include <unicode/ucol.h> #include <unicode/uloc.h> int main(int argc, char** argv) { if (argc < 2) { fprintf(stderr, "usage: %s LOCALE\n", argv[0]); return 1; } const char *test_locale = argv[1]; printf("version = %s\n", U_ICU_VERSION); UErrorCode status = U_ZERO_ERROR; UCollator *coll = ucol_open(test_locale, &status); printf("ucol_open\n status = %s\n", u_errorName(status)); if (U_SUCCESS(status)) { const char *valid_locale = ucol_getLocaleByType(coll, ULOC_VALID_LOCALE, &status); printf("ucol_getLocaleByType\n status = %s\n locale = %s\n", u_errorName(status), valid_locale); } ucol_close(coll); } Build and test: apt-get install build-essential pkg-config libicu-dev gcc -o icu-test icu-test.c $(pkg-config --cflags --libs icu-i18n) ./icu-test yue-Hans I get the following on Debian Bullseye: version = 67.1 ucol_open status = U_USING_FALLBACK_WARNING ucol_getLocaleByType status = U_USING_FALLBACK_WARNING locale = zh_Hans -- Erik
Re: Debian PostgreSQL 15 pg_createcluster - could not open collator for locale
On 26/3/23 15:30, Erik Wienhold wrote: > ./icu-test yue-Hans Erik Thanks for the test code. I compiled and executed as instructed with the following result Matt __ version = 67.1 ucol_open status = U_INVALID_FORMAT_ERROR
Re: Debian PostgreSQL 15 pg_createcluster - could not open collator for locale
> On 26/03/2023 10:26 CEST Matt Gibbins <matt_gibbins@fastmail.com.au> wrote: > > On 26/3/23 15:30, Erik Wienhold wrote: > > ./icu-test yue-Hans > > Erik > > Thanks for the test code. > > I compiled and executed as instructed with the following result > > Matt > > __ > > version = 67.1 > ucol_open > status = U_INVALID_FORMAT_ERROR Okay, at least not a problem with Postgres. I wonder if libicudata is corrupt or simply not compatible even though it's the same version according to the filenames. Whats the hash of your libicu files? find /usr/lib/x86_64-linux-gnu -name 'libicu*.so*' | xargs sha256sum | sort -k2 I get: daecc24ce07ac15b77833042bd07fa5c62e8b82f5d2c38361b45fe76658b0e45 /usr/lib/x86_64-linux-gnu/libicudata.so daecc24ce07ac15b77833042bd07fa5c62e8b82f5d2c38361b45fe76658b0e45 /usr/lib/x86_64-linux-gnu/libicudata.so.67 daecc24ce07ac15b77833042bd07fa5c62e8b82f5d2c38361b45fe76658b0e45 /usr/lib/x86_64-linux-gnu/libicudata.so.67.1 0dd9fd815ec02f5052cb666433d380cb372f6016213c83b13bbf4a402b09f198 /usr/lib/x86_64-linux-gnu/libicui18n.so 0dd9fd815ec02f5052cb666433d380cb372f6016213c83b13bbf4a402b09f198 /usr/lib/x86_64-linux-gnu/libicui18n.so.67 0dd9fd815ec02f5052cb666433d380cb372f6016213c83b13bbf4a402b09f198 /usr/lib/x86_64-linux-gnu/libicui18n.so.67.1 1702a12144088442b222fcbef1de64f319656c7b58001fdfd639b8689d6721c5 /usr/lib/x86_64-linux-gnu/libicuio.so 1702a12144088442b222fcbef1de64f319656c7b58001fdfd639b8689d6721c5 /usr/lib/x86_64-linux-gnu/libicuio.so.67 1702a12144088442b222fcbef1de64f319656c7b58001fdfd639b8689d6721c5 /usr/lib/x86_64-linux-gnu/libicuio.so.67.1 fc098d6df6825c1e46ac31d984a0c1839a514eef018b1f835b7610b00e008b92 /usr/lib/x86_64-linux-gnu/libicutest.so fc098d6df6825c1e46ac31d984a0c1839a514eef018b1f835b7610b00e008b92 /usr/lib/x86_64-linux-gnu/libicutest.so.67 fc098d6df6825c1e46ac31d984a0c1839a514eef018b1f835b7610b00e008b92 /usr/lib/x86_64-linux-gnu/libicutest.so.67.1 742eedc54a2b6b12003fbb5a71105fae1ed3062b5c2765083d5310738fc2b2b6 /usr/lib/x86_64-linux-gnu/libicutu.so 742eedc54a2b6b12003fbb5a71105fae1ed3062b5c2765083d5310738fc2b2b6 /usr/lib/x86_64-linux-gnu/libicutu.so.67 742eedc54a2b6b12003fbb5a71105fae1ed3062b5c2765083d5310738fc2b2b6 /usr/lib/x86_64-linux-gnu/libicutu.so.67.1 650d19e2164b13dbcac1f1cd493dbc01f0e3d873086e3a718d9ca09ddd59f2fd /usr/lib/x86_64-linux-gnu/libicuuc.so 650d19e2164b13dbcac1f1cd493dbc01f0e3d873086e3a718d9ca09ddd59f2fd /usr/lib/x86_64-linux-gnu/libicuuc.so.67 650d19e2164b13dbcac1f1cd493dbc01f0e3d873086e3a718d9ca09ddd59f2fd /usr/lib/x86_64-linux-gnu/libicuuc.so.67.1 Maybe reinstalling libicu67 solves your issue. I'm out of ideas at this point. I recommend addressing your issue to Debian or the ICU mailing list: https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=libicu67 https://sourceforge.net/p/icu/mailman/ -- Erik
Re: Debian PostgreSQL 15 pg_createcluster - could not open collator for locale
On 26/03/2023 10:26 CEST Matt Gibbins <matt_gibbins@fastmail.com.au> wrote: On 26/3/23 15:30, Erik Wienhold wrote:./icu-test yue-HansErik Thanks for the test code. I compiled and executed as instructed with the following result Matt __ version = 67.1 ucol_open status = U_INVALID_FORMAT_ERROROkay, at least not a problem with Postgres. I wonder if libicudata is corrupt or simply not compatible even though it's the same version according to the filenames. Whats the hash of your libicu files? find /usr/lib/x86_64-linux-gnu -name 'libicu*.so*' | xargs sha256sum | sort -k2 I get: daecc24ce07ac15b77833042bd07fa5c62e8b82f5d2c38361b45fe76658b0e45 /usr/lib/x86_64-linux-gnu/libicudata.so daecc24ce07ac15b77833042bd07fa5c62e8b82f5d2c38361b45fe76658b0e45 /usr/lib/x86_64-linux-gnu/libicudata.so.67 daecc24ce07ac15b77833042bd07fa5c62e8b82f5d2c38361b45fe76658b0e45 /usr/lib/x86_64-linux-gnu/libicudata.so.67.1 0dd9fd815ec02f5052cb666433d380cb372f6016213c83b13bbf4a402b09f198 /usr/lib/x86_64-linux-gnu/libicui18n.so 0dd9fd815ec02f5052cb666433d380cb372f6016213c83b13bbf4a402b09f198 /usr/lib/x86_64-linux-gnu/libicui18n.so.67 0dd9fd815ec02f5052cb666433d380cb372f6016213c83b13bbf4a402b09f198 /usr/lib/x86_64-linux-gnu/libicui18n.so.67.1 1702a12144088442b222fcbef1de64f319656c7b58001fdfd639b8689d6721c5 /usr/lib/x86_64-linux-gnu/libicuio.so 1702a12144088442b222fcbef1de64f319656c7b58001fdfd639b8689d6721c5 /usr/lib/x86_64-linux-gnu/libicuio.so.67 1702a12144088442b222fcbef1de64f319656c7b58001fdfd639b8689d6721c5 /usr/lib/x86_64-linux-gnu/libicuio.so.67.1 fc098d6df6825c1e46ac31d984a0c1839a514eef018b1f835b7610b00e008b92 /usr/lib/x86_64-linux-gnu/libicutest.so fc098d6df6825c1e46ac31d984a0c1839a514eef018b1f835b7610b00e008b92 /usr/lib/x86_64-linux-gnu/libicutest.so.67 fc098d6df6825c1e46ac31d984a0c1839a514eef018b1f835b7610b00e008b92 /usr/lib/x86_64-linux-gnu/libicutest.so.67.1 742eedc54a2b6b12003fbb5a71105fae1ed3062b5c2765083d5310738fc2b2b6 /usr/lib/x86_64-linux-gnu/libicutu.so 742eedc54a2b6b12003fbb5a71105fae1ed3062b5c2765083d5310738fc2b2b6 /usr/lib/x86_64-linux-gnu/libicutu.so.67 742eedc54a2b6b12003fbb5a71105fae1ed3062b5c2765083d5310738fc2b2b6 /usr/lib/x86_64-linux-gnu/libicutu.so.67.1 650d19e2164b13dbcac1f1cd493dbc01f0e3d873086e3a718d9ca09ddd59f2fd /usr/lib/x86_64-linux-gnu/libicuuc.so 650d19e2164b13dbcac1f1cd493dbc01f0e3d873086e3a718d9ca09ddd59f2fd /usr/lib/x86_64-linux-gnu/libicuuc.so.67 650d19e2164b13dbcac1f1cd493dbc01f0e3d873086e3a718d9ca09ddd59f2fd /usr/lib/x86_64-linux-gnu/libicuuc.so.67.1 Maybe reinstalling libicu67 solves your issue. I'm out of ideas at this point. I recommend addressing your issue to Debian or the ICU mailing list: https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=libicu67 https://sourceforge.net/p/icu/mailman/ -- Erik
Erik
Thank you for your assistance.
The offending library was libicudata.so.67.1.
Reinstalling libicu67 has resolved the issue.
Not sure why it wasn't upgraded as I have been doing several upgrades on the system.
Matt.
Re: Debian PostgreSQL 15 pg_createcluster - could not open collator for locale
> On 26/03/2023 13:41 CEST Matt Gibbins <matt_gibbins@fastmail.com.au> wrote: > > Thank you for your assistance. > The offending library was libicudata.so.67.1. > Reinstalling libicu67 has resolved the issue. > Not sure why it wasn't upgraded as I have been doing several upgrades on the > system. You're welcome. It's weird and I have never seen this before. Maybe disk failure or accidentally modified the files after installing? I just found out about debsums to resolve this very issue. https://manpages.debian.org/bullseye/debsums/debsums.1.en.html#apt_get -- Erik