thank you for the quick reply (again!). I am not using pgpool admin so is it possible to share the shell script that does the rebuilding?
For failover detection and failover the script should be as simple as:
Check the failing node
if failing_node=primary_node
create trigger file on secondary;
end if;
if failing_node=secondary_node
set synchornous replication off;
end if;
The logic for rebuilding was simple (if I remember it correctly):
node_to_rebuild=node_2;
active_node=node_1;
node_to_rebuild=$1; //pass this as a parameter
if node_1=node_to_rebuild
then active_node=node_2;
end if;
copy data directory from active_node to node_to_rebuild;
start postgresql on node_to_rebuild;
use ppc_attach_node to attach newly created node;
You need to use ssh (remote command execution) for some of these commands and make sure that ssh can login without password prompt to both the nodes. If you are going to use pgpool in HA mode, then keep a copy of this rebuilding script on each node so that you can rebuild any node from any node.
With pgpoolAdmin the process must be much more simpler, but I could never get it installed. :-(