Обсуждение: postgresql-R

Поиск
Список
Период
Сортировка

postgresql-R

От
Louis Gonzales
Дата:
Hello,
I was wondering if anybody has had much experience with postgresql-R, or
being able to do database replication in a WAN environment.
Basically, does anybody have any good links/docs on doing this with
Solaris 9, postgresql v8.0?

Thanks in advance.

Вложения

Re: postgresql-R

От
"Jim C. Nasby"
Дата:
On Tue, Mar 07, 2006 at 03:29:37PM -0500, Louis Gonzales wrote:
> Hello,
> I was wondering if anybody has had much experience with postgresql-R, or
> being able to do database replication in a WAN environment.
> Basically, does anybody have any good links/docs on doing this with
> Solaris 9, postgresql v8.0?

If I'm not mistaken, pgreplication (which I'm guessing is what you mean
by postgresql-R) is part of the basis for the upcomming Slony-II.

Right now, depending on what you mean by 'WAN', your best bet is
probably Slony-I. If you're not doing a large update volume and the
connections to your remote sites are reliable it should work OK.
--
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

Re: postgresql-R

От
Louis Gonzales
Дата:
Based on:<br /><p><b>Postgres based Replication Projects</b><ul><li><a
href="http://gborg.postgresql.org/project/pgreplication/projdisplay.php"><b>PGReplication</b></a><li>Postgres-R: <a
href="http://www.cs.mcgill.ca/%7Ekemme"><b>Dr.Kemme's Site</b></a>, <a
href="http://www.cs.mcgill.ca/%7Ekemme/papers/vldb00.html"><b>Paper</b></a>,<a
href="http://www.cs.mcgill.ca/%7Ekemme/publications.html"><b>Publications</b></a>,<a
href="http://www.cs.mcgill.ca/%7Ekemme/disl/replication.html"><b>ReplicationWork</b></a>, <a
href="http://www.cs.cornell.edu/Info/Projects/HORUS/"><b>TheHorus Project</b></a> and <a
href="http://www.cs.cornell.edu/Info/Projects/Ensemble/"><b>Emsemble</b></a><li><a
href="http://www.inf.ethz.ch/department/IS/iks/research/dragon.html"><b>DRAGON</b></a>:Database Replication based on
GroupCommunication <li><a href="http://gborg.postgresql.org/project/slony1/projdisplay.php"><b>The Slony-1
Project</b></a><li><ahref="http://hiroshima.sraw.co.jp/people/mitani/jpug/pgcluster/en/"><b>PGCluster</b></a><li><a
href="http://sourceforge.net/projects/dbbalancer/"><b>DBBalancer</b></a><li><a
href="http://pgreplicator.sourceforge.net/"><b>PostgreSQLReplicator</b></a>: <a
href="http://pgreplicator.sourceforge.net/tutorial.html"><b>Tutorial</b></a>,<a
href="http://pgreplicator.sourceforge.net/5074-69225.pdf"><b>Paper</b></a>,<a
href="http://techdocs.postgresql.org/installguides.php#replicator"><b>Techdocs</b></a><li><a
href="http://gborg.postgresql.org/project/erserver/projdisplay.php"><b>eRServer:</b></a><a
href="http://techdocs.postgresql.org/installguides.php#rserv"><b>Techdocs</b></a><li><a
href="http://archives.postgresql.org/pgsql-patches/2002-02/msg00004.php"><b>DBMirror</b></a>:Will be in the 7.3 contrib
directory.<li><a href="http://usogres.good-day.net/"><b>Usogres</b></a>: <a
href="http://techdocs.postgresql.org/installguides.php#usogres"><b>Techdocs</b></a></ul>From:  <a
class="moz-txt-link-freetext"
href="http://gborg.postgresql.org/project/pgreplication/genpage.php?replication_research">http://gborg.postgresql.org/project/pgreplication/genpage.php?replication_research</a><br
/>I was under the impression that "Postgres-R" - I misspelled it as "postgresql-R" - is a database clustering
software. I downloaded the sources for this and it's supposed to work with "spread."  I apologize if I'm mistaken?<br
/><br/> I will certainly take a look into Slony-1, as you suggest.  Did you have any good links to this?  Have you gone
throughan install on Solaris 9 w/Postgresl v8.0?<br /><br /> WAN != LAN, where my database server is not in the same
cityas the target system to cluster with; for my worse case scenario, could be from east coast to west coast.<br /><br
/>         Kind regards,<br /><br /><br /> Jim C. Nasby wrote:<br /><blockquote
cite="mid20060307203530.GG58405@pervasive.com"type="cite"><pre wrap="">On Tue, Mar 07, 2006 at 03:29:37PM -0500, Louis
Gonzaleswrote: </pre><blockquote type="cite"><pre wrap="">Hello, 
I was wondering if anybody has had much experience with postgresql-R, or
being able to do database replication in a WAN environment.
Basically, does anybody have any good links/docs on doing this with
Solaris 9, postgresql v8.0?   </pre></blockquote><pre wrap="">
If I'm not mistaken, pgreplication (which I'm guessing is what you mean
by postgresql-R) is part of the basis for the upcomming Slony-II.

Right now, depending on what you mean by 'WAN', your best bet is
probably Slony-I. If you're not doing a large update volume and the
connections to your remote sites are reliable it should work OK. </pre></blockquote><br />

Re: postgresql-R

От
Tom Lane
Дата:
Louis Gonzales <louis.gonzales@linuxlouis.net> writes:
> I was wondering if anybody has had much experience with postgresql-R,

Postgres-R doesn't exist in any production-ready form.  That software
is an academic project that's based on a very old, buggy release of
Postgres ... you *don't* want to use it.

            regards, tom lane

Re: postgresql-R

От
Louis Gonzales
Дата:
Tom Lane wrote:<br /><blockquote cite="mid3834.1141765903@sss.pgh.pa.us" type="cite"><pre wrap="">Louis Gonzales <a
class="moz-txt-link-rfc2396E"href="mailto:louis.gonzales@linuxlouis.net"><louis.gonzales@linuxlouis.net></a>
writes:</pre><blockquote type="cite"><pre wrap="">I was wondering if anybody has had much experience with postgresql-R,
 </pre></blockquote><pre wrap=""> 
Postgres-R doesn't exist in any production-ready form.  That software
is an academic project that's based on a very old, buggy release of
Postgres ... you *don't* want to use it.
        regards, tom lane </pre></blockquote> Tom,<br /> Thank you!  I'll get rid of the source now.  Jim Nasby,
actuallyreferred me to Slony... thanks Jim, that looks like what I want.<br /> 

Re: postgresql-R

От
Jim Nasby
Дата:
On Mar 7, 2006, at 2:58 PM, Louis Gonzales wrote:

> Based on:
> Postgres based Replication Projects
>
> PG Replication
> Postgres-R: Dr. Kemme's Site, Paper, Publications, Replication
> Work, The Horus Project and Emsemble
> DRAGON: Database Replication based on Group Communication
> The Slony-1 Project
> PGCluster
> DBBalancer
> PostgreSQL Replicator: Tutorial, Paper, Techdocs
> eRServer: Techdocs
> DBMirror: Will be in the 7.3 contrib directory.
> Usogres: Techdocs
> From:  http://gborg.postgresql.org/project/pgreplication/
> genpage.php?replication_research
> I was under the impression that "Postgres-R" - I misspelled it as
> "postgresql-R" - is a database clustering software.  I downloaded
> the sources for this and it's supposed to work with "spread."  I
> apologize if I'm mistaken?

Well, AfAIK it's quasi-clustering. I'd describe it as a mix between
sync and async multi-master replication. Its concepts are the basis
behind Slony-II AFAIK.

> I will certainly take a look into Slony-1, as you suggest.  Did you
> have any good links to this?  Have you gone through an install on
> Solaris 9 w/Postgresl v8.0?
>
> WAN != LAN, where my database server is not in the same city as the
> target system to cluster with; for my worse case scenario, could be
> from east coast to west coast.

The issues you'll run into are that in a high-load scenario, the
latency on that link could become a real issue, as could bandwidth
(actually, bandwidth is probably the bigger issue). The other
downside is that slony is decidedly async; if you have to fail over
you're probably going to lose some recently committed data. As long
as that's not an issue then it's your best bet. If it is an issue,
you'll probably be stuck rolling your own solution using something
like 2PC.

You should hop on the slony mailing list if you have any questions
about it.
--
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461



Re: postgresql-R

От
Louis Gonzales
Дата:
Hey Jim,
Thanks again for the pointer to this.  I've already compiled and
installed on one of the two Solaris nodes, that I needed to.  Yeah upon
further reading, I can't wait for Slony-II to come out - is there truth
behind the "anyday" rumor? - is it also true that it's going to
implement the true multi-master scenario, where updates can be made at
any of the clustered nodes.

I'm going to deploy the "slon worker process" locally on every
participating node, rather than letting the master host the "slon"
processes for every cluster participant, for performance reasons.

I will also join the Slony mailing list, very cool stuff.


         Kind Regards,


Jim Nasby wrote:

> On Mar 7, 2006, at 2:58 PM, Louis Gonzales wrote:
>
>> Based on:
>> Postgres based Replication Projects
>>
>> PG Replication
>> Postgres-R: Dr. Kemme's Site, Paper, Publications, Replication  Work,
>> The Horus Project and Emsemble
>> DRAGON: Database Replication based on Group Communication
>> The Slony-1 Project
>> PGCluster
>> DBBalancer
>> PostgreSQL Replicator: Tutorial, Paper, Techdocs
>> eRServer: Techdocs
>> DBMirror: Will be in the 7.3 contrib directory.
>> Usogres: Techdocs
>> From:  http://gborg.postgresql.org/project/pgreplication/
>> genpage.php?replication_research
>> I was under the impression that "Postgres-R" - I misspelled it as
>> "postgresql-R" - is a database clustering software.  I downloaded
>> the sources for this and it's supposed to work with "spread."  I
>> apologize if I'm mistaken?
>
>
> Well, AfAIK it's quasi-clustering. I'd describe it as a mix between
> sync and async multi-master replication. Its concepts are the basis
> behind Slony-II AFAIK.
>
>> I will certainly take a look into Slony-1, as you suggest.  Did you
>> have any good links to this?  Have you gone through an install on
>> Solaris 9 w/Postgresl v8.0?
>>
>> WAN != LAN, where my database server is not in the same city as the
>> target system to cluster with; for my worse case scenario, could be
>> from east coast to west coast.
>
>
> The issues you'll run into are that in a high-load scenario, the
> latency on that link could become a real issue, as could bandwidth
> (actually, bandwidth is probably the bigger issue). The other
> downside is that slony is decidedly async; if you have to fail over
> you're probably going to lose some recently committed data. As long
> as that's not an issue then it's your best bet. If it is an issue,
> you'll probably be stuck rolling your own solution using something
> like 2PC.
>
> You should hop on the slony mailing list if you have any questions
> about it.
> --
> Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
> Pervasive Software      http://pervasive.com    work: 512-231-6117
> vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461
>


Re: postgresql-R

От
Louis Gonzales
Дата:
Jim Nasby wrote:

> On Mar 7, 2006, at 2:58 PM, Louis Gonzales wrote:
>
>> Based on:
>> Postgres based Replication Projects
>>
>> PG Replication
>> Postgres-R: Dr. Kemme's Site, Paper, Publications, Replication  Work,
>> The Horus Project and Emsemble
>> DRAGON: Database Replication based on Group Communication
>> The Slony-1 Project
>> PGCluster
>> DBBalancer
>> PostgreSQL Replicator: Tutorial, Paper, Techdocs
>> eRServer: Techdocs
>> DBMirror: Will be in the 7.3 contrib directory.
>> Usogres: Techdocs
>> From:  http://gborg.postgresql.org/project/pgreplication/
>> genpage.php?replication_research
>> I was under the impression that "Postgres-R" - I misspelled it as
>> "postgresql-R" - is a database clustering software.  I downloaded
>> the sources for this and it's supposed to work with "spread."  I
>> apologize if I'm mistaken?
>
>
> Well, AfAIK it's quasi-clustering. I'd describe it as a mix between
> sync and async multi-master replication. Its concepts are the basis
> behind Slony-II AFAIK.
>
>> I will certainly take a look into Slony-1, as you suggest.  Did you
>> have any good links to this?  Have you gone through an install on
>> Solaris 9 w/Postgresl v8.0?
>>
>> WAN != LAN, where my database server is not in the same city as the
>> target system to cluster with; for my worse case scenario, could be
>> from east coast to west coast.
>
>
> The issues you'll run into are that in a high-load scenario, the
> latency on that link could become a real issue, as could bandwidth
> (actually, bandwidth is probably the bigger issue). The other
> downside is that slony is decidedly async; if you have to fail over
> you're probably going to lose some recently committed data. As long
> as that's not an issue then it's your best bet. If it is an issue,
> you'll probably be stuck rolling your own solution using something
> like 2PC.

Actually, one more question.  Upon further reading, the documentation
stated that any modification done on the master, will have to be
committed to all cluster participants, otherwise it will not commit?  Is
it still possible to lose any committed data, with that being true?

As a supplement, I was thinking about using some sort of PITR, in
conjuction with Slony-I to ensure all bases are covered.

>
> You should hop on the slony mailing list if you have any questions
> about it.
> --
> Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
> Pervasive Software      http://pervasive.com    work: 512-231-6117
> vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461
>


Re: postgresql-R

От
Christopher Browne
Дата:
Clinging to sanity, louis.gonzales@linuxlouis.net (Louis Gonzales) mumbled into her beard:
> Hey Jim, Thanks again for the pointer to this.  I've already
> compiled and installed on one of the two Solaris nodes, that I
> needed to.  Yeah upon further reading, I can't wait for Slony-II to
> come out - is there truth behind the "anyday" rumor? - is it also
> true that it's going to implement the true multi-master scenario,
> where updates can be made at any of the clustered nodes.

I'm not holding my breath, at this point.

Performance results have indicated the Slony-II approach wouldn't work
out well for systems where there is heavy competition for locks on
some objects.  The trouble is that you wind up having to distribute
locks, and if the application has common heavily-updated objects, the
cost winds up prohibitive...

> I'm going to deploy the "slon worker process" locally on every
> participating node, rather than letting the master host the "slon"
> processes for every cluster participant, for performance reasons.

I run quite a lot of Slony-I nodes, and let me be pointed...
Performance is NOT a good reason (or a reason at all) to spread slon
processes across a bunch of hosts.

Performance of replication is based on the I/O taking place in the
databases; the costs of distributing some of the slon network traffic
will be fairly immaterial.

The "savings" from hosting slons on specific DB nodes is an illusion.
The data will all have to cross the network to get from sources to
providers, so whether the slon is on one host or another, you'll have
the same traffic transmitted around.

Having slons centralized makes it way simpler to manage them; I can't
see there being anywhere near enough benefit from any savings of
network traffic to actually represent a material performance
improvement.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','cbbrowne.com').
http://linuxdatabases.info/info/slony.html
Never criticize anybody until  you have walked  a mile in their shoes,
because by that time you will be a mile away and have their shoes.
-- email sig, Brian Servis