Обсуждение: PostgreSQL - HA Cluster / Etcd Issue

Поиск
Список
Период
Сортировка

PostgreSQL - HA Cluster / Etcd Issue

От
SOzcn
Дата:
Hello,

In my database architecture, I use Etcd, Patroni, HAProxy, and Keepalived.

Although many things in this architecture work as expected, Etcd encounters the error "memberID:5960773920904691185 alarm:NOSPACE" once every 2 or 3 months. The root cause is that the database size reaches the specified limit and Etcd does not clean it up through rotation. I am aware of this, but despite reviewing the maintenance documentation, I could not find the most appropriate parameter values for this rotation. I am sharing an example of my own Etcd configuration below. I kindly ask you to share your experience on this matter. I am kinda stuck on this issue.

In particular, is it possible to schedule the parameter that performs this cleanup with a time setting? For example, can I tell it to run the compact operation at around 01:00 AM?

Best regards.


[Unit]
Description=etcd service
Documentation=https://github.com/etcd-io/etcd
After=network.target

[Service]
User=etcd
Type=notify
ExecStart=/usr/local/sbin/etcd \
  --name etcd1 \
  --data-dir=/etc/etcd \
  --initial-advertise-peer-urls=http://10.115.208.168:2380 \
  --listen-peer-urls=http://192.168.1.1:2380 \
  --listen-client-urls=http://192.168.1.1:2379,http://127.0.0.1:2379 \
  --advertise-client-urls=192.168.1.1:2379 \
  --initial-cluster-token="etcd-cluster" \
  --initial-cluster="etcd1=http://192.168.1.1:2380,etcd2=http://192.168.1.2:2380,etcd3=http://3:2380" \
  --initial-cluster-state="new" \
  --log-level="error"

[Install]
WantedBy=multi-user.target

Re: PostgreSQL - HA Cluster / Etcd Issue

От
Scott Ribe
Дата:
This is a PostgreSQL mailing list, and your question has nothing to do with PostgreSQL.

--
Scott Ribe
scott_ribe@elevated-dev.com
https://www.linkedin.com/in/scottribe/



> On Aug 20, 2025, at 1:40 AM, SOzcn <selahattinozcnma@gmail.com> wrote:
>
> Hello,
>
> In my database architecture, I use Etcd, Patroni, HAProxy, and Keepalived.
>
> Although many things in this architecture work as expected, Etcd encounters the error "memberID:5960773920904691185
alarm:NOSPACE"once every 2 or 3 months. The root cause is that the database size reaches the specified limit and Etcd
doesnot clean it up through rotation. I am aware of this, but despite reviewing the maintenance documentation, I could
notfind the most appropriate parameter values for this rotation. I am sharing an example of my own Etcd configuration
below.I kindly ask you to share your experience on this matter. I am kinda stuck on this issue. 
>
> In particular, is it possible to schedule the parameter that performs this cleanup with a time setting? For example,
canI tell it to run the compact operation at around 01:00 AM? 
>
> Best regards.
>
>
> [Unit]
> Description=etcd service
> Documentation=https://github.com/etcd-io/etcd
> After=network.target
>
> [Service]
> User=etcd
> Type=notify
> ExecStart=/usr/local/sbin/etcd \
>   --name etcd1 \
>   --data-dir=/etc/etcd \
>   --initial-advertise-peer-urls=http://10.115.208.168:2380 \
>   --listen-peer-urls=http://192.168.1.1:2380 \
>   --listen-client-urls=http://192.168.1.1:2379,http://127.0.0.1:2379 \
>   --advertise-client-urls=192.168.1.1:2379 \
>   --initial-cluster-token="etcd-cluster" \
>   --initial-cluster="etcd1=http://192.168.1.1:2380,etcd2=http://192.168.1.2:2380,etcd3=http://3:2380" \
>   --initial-cluster-state="new" \
>   --log-level="error"
>
> [Install]
> WantedBy=multi-user.target




Re: PostgreSQL - HA Cluster / Etcd Issue

От
SOzcn
Дата:
This is high availability environment and etcd is one management tools of them . Many PostgreSQL managed by this architec. I advice to you research. Etcs - Patrono - Postgresql- Haproxy- Keepalived. Anyway if you doesn't have experience of it, that's ok. 

20 Ağu 2025 Çar 15:46 tarihinde Scott Ribe <scott_ribe@elevated-dev.com> şunu yazdı:
This is a PostgreSQL mailing list, and your question has nothing to do with PostgreSQL.

--
Scott Ribe
scott_ribe@elevated-dev.com
https://www.linkedin.com/in/scottribe/



> On Aug 20, 2025, at 1:40 AM, SOzcn <selahattinozcnma@gmail.com> wrote:
>
> Hello,
>
> In my database architecture, I use Etcd, Patroni, HAProxy, and Keepalived.
>
> Although many things in this architecture work as expected, Etcd encounters the error "memberID:5960773920904691185 alarm:NOSPACE" once every 2 or 3 months. The root cause is that the database size reaches the specified limit and Etcd does not clean it up through rotation. I am aware of this, but despite reviewing the maintenance documentation, I could not find the most appropriate parameter values for this rotation. I am sharing an example of my own Etcd configuration below. I kindly ask you to share your experience on this matter. I am kinda stuck on this issue.
>
> In particular, is it possible to schedule the parameter that performs this cleanup with a time setting? For example, can I tell it to run the compact operation at around 01:00 AM?
>
> Best regards.
>
>
> [Unit]
> Description=etcd service
> Documentation=https://github.com/etcd-io/etcd
> After=network.target
>
> [Service]
> User=etcd
> Type=notify
> ExecStart=/usr/local/sbin/etcd \
>   --name etcd1 \
>   --data-dir=/etc/etcd \
>   --initial-advertise-peer-urls=http://10.115.208.168:2380 \
>   --listen-peer-urls=http://192.168.1.1:2380 \
>   --listen-client-urls=http://192.168.1.1:2379,http://127.0.0.1:2379 \
>   --advertise-client-urls=192.168.1.1:2379 \
>   --initial-cluster-token="etcd-cluster" \
>   --initial-cluster="etcd1=http://192.168.1.1:2380,etcd2=http://192.168.1.2:2380,etcd3=http://3:2380" \
>   --initial-cluster-state="new" \
>   --log-level="error"
>
> [Install]
> WantedBy=multi-user.target

Re: PostgreSQL - HA Cluster / Etcd Issue

От
Mike Artz
Дата:

Are you saying that you already have a compaction job running (e.g.

etcdctl compact $(etcdctl endpoint status --write-out=json | jq -r '.[0].Status.header.revision - 1000')

or similar), but you’re hoping there’s a parameter in etcd itself to handle that scheduling?




On Wed, Aug 20, 2025 at 8:53 AM SOzcn <selahattinozcnma@gmail.com> wrote:
This is high availability environment and etcd is one management tools of them . Many PostgreSQL managed by this architec. I advice to you research. Etcs - Patrono - Postgresql- Haproxy- Keepalived. Anyway if you doesn't have experience of it, that's ok. 

20 Ağu 2025 Çar 15:46 tarihinde Scott Ribe <scott_ribe@elevated-dev.com> şunu yazdı:
This is a PostgreSQL mailing list, and your question has nothing to do with PostgreSQL.

--
Scott Ribe
scott_ribe@elevated-dev.com
https://www.linkedin.com/in/scottribe/



> On Aug 20, 2025, at 1:40 AM, SOzcn <selahattinozcnma@gmail.com> wrote:
>
> Hello,
>
> In my database architecture, I use Etcd, Patroni, HAProxy, and Keepalived.
>
> Although many things in this architecture work as expected, Etcd encounters the error "memberID:5960773920904691185 alarm:NOSPACE" once every 2 or 3 months. The root cause is that the database size reaches the specified limit and Etcd does not clean it up through rotation. I am aware of this, but despite reviewing the maintenance documentation, I could not find the most appropriate parameter values for this rotation. I am sharing an example of my own Etcd configuration below. I kindly ask you to share your experience on this matter. I am kinda stuck on this issue.
>
> In particular, is it possible to schedule the parameter that performs this cleanup with a time setting? For example, can I tell it to run the compact operation at around 01:00 AM?
>
> Best regards.
>
>
> [Unit]
> Description=etcd service
> Documentation=https://github.com/etcd-io/etcd
> After=network.target
>
> [Service]
> User=etcd
> Type=notify
> ExecStart=/usr/local/sbin/etcd \
>   --name etcd1 \
>   --data-dir=/etc/etcd \
>   --initial-advertise-peer-urls=http://10.115.208.168:2380 \
>   --listen-peer-urls=http://192.168.1.1:2380 \
>   --listen-client-urls=http://192.168.1.1:2379,http://127.0.0.1:2379 \
>   --advertise-client-urls=192.168.1.1:2379 \
>   --initial-cluster-token="etcd-cluster" \
>   --initial-cluster="etcd1=http://192.168.1.1:2380,etcd2=http://192.168.1.2:2380,etcd3=http://3:2380" \
>   --initial-cluster-state="new" \
>   --log-level="error"
>
> [Install]
> WantedBy=multi-user.target

Re: PostgreSQL - HA Cluster / Etcd Issue

От
Scott Ribe
Дата:
> On Aug 20, 2025, at 7:52 AM, SOzcn <selahattinozcnma@gmail.com> wrote:
>
> This is high availability environment and etcd is one management tools of them . Many PostgreSQL managed by this
architec.I advice to you research. Etcs - Patrono - Postgresql- Haproxy- Keepalived. Anyway if you doesn't have
experienceof it, that's ok.  

I am perfectly well aware of all that. However, you have an etcd problem, not a postgres problem. You'd have a much
betterchance of getting answers to etcd issues on an etcd mailing list. 


Re: PostgreSQL - HA Cluster / Etcd Issue

От
Ozgur Kulu
Дата:
Hi there, 

Etcd database should be extended from 2 gb to 8gb. On the other hand, you can use defrag or again created the etcd db.

Have a nice day,

20 Ağu 2025 Çar 10:40 tarihinde SOzcn <selahattinozcnma@gmail.com> şunu yazdı:
Hello,

In my database architecture, I use Etcd, Patroni, HAProxy, and Keepalived.

Although many things in this architecture work as expected, Etcd encounters the error "memberID:5960773920904691185 alarm:NOSPACE" once every 2 or 3 months. The root cause is that the database size reaches the specified limit and Etcd does not clean it up through rotation. I am aware of this, but despite reviewing the maintenance documentation, I could not find the most appropriate parameter values for this rotation. I am sharing an example of my own Etcd configuration below. I kindly ask you to share your experience on this matter. I am kinda stuck on this issue.

In particular, is it possible to schedule the parameter that performs this cleanup with a time setting? For example, can I tell it to run the compact operation at around 01:00 AM?

Best regards.


[Unit]
Description=etcd service
Documentation=https://github.com/etcd-io/etcd
After=network.target

[Service]
User=etcd
Type=notify
ExecStart=/usr/local/sbin/etcd \
  --name etcd1 \
  --data-dir=/etc/etcd \
  --initial-advertise-peer-urls=http://10.115.208.168:2380 \
  --listen-peer-urls=http://192.168.1.1:2380 \
  --listen-client-urls=http://192.168.1.1:2379,http://127.0.0.1:2379 \
  --advertise-client-urls=192.168.1.1:2379 \
  --initial-cluster-token="etcd-cluster" \
  --initial-cluster="etcd1=http://192.168.1.1:2380,etcd2=http://192.168.1.2:2380,etcd3=http://3:2380" \
  --initial-cluster-state="new" \
  --log-level="error"

[Install]
WantedBy=multi-user.target

Re: PostgreSQL - HA Cluster / Etcd Issue

От
SOzcn
Дата:
Hello Ozgur,

Thanks for the reply. 

If i up to etcd size from 2 to 8 GB it's make much problem but if defrag is fix the "no space" issue can you share your experience? By example is it spent too much rss etc. and are you doing that with cronjob or manuel. As i know if etcd went to error with "no space" defrag methot also doesn't work.

Have a nice day.

Ozgur Kulu <kuluozgur@gmail.com>, 20 Ağu 2025 Çar, 19:51 tarihinde şunu yazdı:
Hi there, 

Etcd database should be extended from 2 gb to 8gb. On the other hand, you can use defrag or again created the etcd db.

Have a nice day,

20 Ağu 2025 Çar 10:40 tarihinde SOzcn <selahattinozcnma@gmail.com> şunu yazdı:
Hello,

In my database architecture, I use Etcd, Patroni, HAProxy, and Keepalived.

Although many things in this architecture work as expected, Etcd encounters the error "memberID:5960773920904691185 alarm:NOSPACE" once every 2 or 3 months. The root cause is that the database size reaches the specified limit and Etcd does not clean it up through rotation. I am aware of this, but despite reviewing the maintenance documentation, I could not find the most appropriate parameter values for this rotation. I am sharing an example of my own Etcd configuration below. I kindly ask you to share your experience on this matter. I am kinda stuck on this issue.

In particular, is it possible to schedule the parameter that performs this cleanup with a time setting? For example, can I tell it to run the compact operation at around 01:00 AM?

Best regards.


[Unit]
Description=etcd service
Documentation=https://github.com/etcd-io/etcd
After=network.target

[Service]
User=etcd
Type=notify
ExecStart=/usr/local/sbin/etcd \
  --name etcd1 \
  --data-dir=/etc/etcd \
  --initial-advertise-peer-urls=http://10.115.208.168:2380 \
  --listen-peer-urls=http://192.168.1.1:2380 \
  --listen-client-urls=http://192.168.1.1:2379,http://127.0.0.1:2379 \
  --advertise-client-urls=192.168.1.1:2379 \
  --initial-cluster-token="etcd-cluster" \
  --initial-cluster="etcd1=http://192.168.1.1:2380,etcd2=http://192.168.1.2:2380,etcd3=http://3:2380" \
  --initial-cluster-state="new" \
  --log-level="error"

[Install]
WantedBy=multi-user.target