Обсуждение: 'COPY' Statement

Поиск
Список
Период
Сортировка

'COPY' Statement

От
Renato Oliveira
Дата:

Dear all,

 

I just had a thought, another one. Please the guys who have the powerful knowledge I need your help ;-)

 

Let’s suppose

I have Three  DB servers ,physical servers – let’s call them:

·         Original_DB01

·         New_DB02

·         Third_DB03

 

1 – Let’s suppose I have a PostgreSQL on ‘Original_DB01’ server with a Data Base of 162GB in size.

2 – Let’s suppose I buy a new server ‘New_DB02’ - all singing all dancing cool server, RAID10, 25GB RAM, 64Bit etc.

3 – Let’s suppose I backup the old server with pg_dump to a file “pg_dump_original_DB01”

4 – Let’s also suppose I backup the schema only from ‘Original_DB01’and restore it to ‘Third_DB03’.

5 – Straight after I backup the schema, I stop PostgreSQL on ‘Original_DB01’.

6 – Then I restore the schema to ‘Third_DB03’ and point the application to it.

7 – Let’s suppose the application can create tables and can start using the database on ‘Third_DB03’ and adding new data.

8 – Let’s suppose I restore the FULL  pg_dump file “pg_dump_original_DB01” to ‘New_DB02’

9 – Once the pg_dump has been restored to ‘New_DB02’ I can then point the application to it and data will be added to it straight away.

 

Question:

10 – Can I use ‘COPY’ statement to transfer the data from ‘Third_DB03’ to ‘New_DB02’ without ‘WIPING’ or Deleting the existing data?

I want to merge the DATA from ‘Third_DB03’ to ‘New_DB02’ without deleting the Database or the existing data on the existing database.

 

Let me explain the Idea behind this craziness.

I am using ‘Third_DB03’ because the amount of data to be transferred from it, will be minimum, and easy to and quick to ‘COPY’

 

This application has two part components (Data and Alarms)

We can get the data back in a very hard way, but without the data being in the database we can’t generate alarms.

 

I would very much appreciate all your help on this idea, please.

I would be very glad and grateful for any comments and improvements on this idea, picking the holes etc.

 

Thank you very much in advance

 

Renato

 

 

 

 
Renato Oliveira
Systems Administrator
e-mail: renato.oliveira@grant.co.uk
 
Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk
 
Grant Instruments (Cambridge) Ltd
 
Company registered in England, registration number 658133
 
Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK
 
 
 
 

 

P Please consider the environment before printing this email

CONFIDENTIALITY: The information in this e-mail and any attachments is confidential. It is intended only for the named recipients(s). If you are not the named recipient please notify the sender immediately and do not disclose the contents to another person or take copies.
 
VIRUSES: The contents of this e-mail or attachment(s) may contain viruses which could damage your own computer system. Whilst Grant Instruments (Cambridge) Ltd has taken every reasonable precaution to minimise this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should therefore carry out your own virus checks before opening the attachment(s).
 
OpenXML: For information about the OpenXML file format in use within Grant Instruments please visit our website

Re: 'COPY' Statement

От
Iñigo Martinez Lasala
Дата:
I think it depends of your datamodel and specifically of your table constraints.

For example, if you have a table with a pk generated by a sequence, you will have to deal with it before importing into your New_DB02. But for a plain log table without constrains you can dump it an import into your "just loaded" New_DB02 without any problem.


-----Original Message-----
From: Renato Oliveira <renato.oliveira@grant.co.uk>
To: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org>
Cc: Steve Fisher <steve.fisher@grant.co.uk>
Subject: [ADMIN] 'COPY' Statement
Date: Tue, 13 Apr 2010 08:42:19 +0100

Dear all,

 

I just had a thought, another one. Please the guys who have the powerful knowledge I need your help ;-)

 

Let’s suppose

I have Three  DB servers ,physical servers – let’s call them:

·        Original_DB01

·        New_DB02

·        Third_DB03

 

1 – Let’s suppose I have a PostgreSQL on ‘Original_DB01’ server with a Data Base of 162GB in size.

2 – Let’s suppose I buy a new server ‘New_DB02’ - all singing all dancing cool server, RAID10, 25GB RAM, 64Bit etc.

3 – Let’s suppose I backup the old server with pg_dump to a file “pg_dump_original_DB01”

4 – Let’s also suppose I backup the schema only from ‘Original_DB01’and restore it to ‘Third_DB03’.

5 – Straight after I backup the schema, I stop PostgreSQL on ‘Original_DB01’.

6 – Then I restore the schema to ‘Third_DB03’ and point the application to it.

7 – Let’s suppose the application can create tables and can start using the database on ‘Third_DB03’ and adding new data.

8 – Let’s suppose I restore the FULL  pg_dump file “pg_dump_original_DB01” to ‘New_DB02’

9 – Once the pg_dump has been restored to ‘New_DB02’ I can then point the application to it and data will be added to it straight away.

 

Question:

10 – Can I use ‘COPY’ statement to transfer the data from ‘Third_DB03’ to ‘New_DB02’ without ‘WIPING’ or Deleting the existing data?

I want to merge the DATA from ‘Third_DB03’ to ‘New_DB02’ without deleting the Database or the existing data on the existing database.

 

Let me explain the Idea behind this craziness.

I am using ‘Third_DB03’ because the amount of data to be transferred from it, will be minimum, and easy to and quick to ‘COPY’

 

This application has two part components (Data and Alarms)

We can get the data back in a very hard way, but without the data being in the database we can’t generate alarms.

 

I would very much appreciate all your help on this idea, please.

I would be very glad and grateful for any comments and improvements on this idea, picking the holes etc.

 

Thank you very much in advance

 

Renato

 

 

 


  Renato Oliveira
Systems Administrator
e-mail: renato.oliveira@grant.co.uk   Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk   Grant Instruments (Cambridge) Ltd
 
Company registered in England, registration number 658133
 
Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK  
 
 
   

P Please consider the environment before printing this email


CONFIDENTIALITY: The information in this e-mail and any attachments is confidential. It is intended only for the named recipients(s). If you are not the named recipient please notify the sender immediately and do not disclose the contents to another person or take copies.  
VIRUSES: The contents of this e-mail or attachment(s) may contain viruses which could damage your own computer system. Whilst Grant Instruments (Cambridge) Ltd has taken every reasonable precaution to minimise this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should therefore carry out your own virus checks before opening the attachment(s).  
OpenXML: For information about the OpenXML file format in use within Grant Instruments please visit our website

Re: 'COPY' Statement

От
Iñigo Martinez Lasala
Дата:
Primary Key. :)

A Primary Key is always a constraint, since it cannot be duplicated. For example, if you have a PK generated by a sequence starting at zero, when you restore your third database into new server you will have duplicated values in this column and COPY command will fail.

-----Original Message-----
From: Renato Oliveira <renato.oliveira@grant.co.uk>
To: Iñigo Martinez Lasala <imartinez@vectorsf.com>
Subject: RE: [ADMIN] 'COPY' Statement
Date: Tue, 13 Apr 2010 08:59:34 +0100

 Dear Iñigo,

 

Thank you very much. What do you mean by ‘pk’?

I am sorry.

 

Thank you very much

 

Renato

 


  Renato Oliveira
Systems Administrator
e-mail: renato.oliveira@grant.co.uk   Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk   Grant Instruments (Cambridge) Ltd
 
Company registered in England, registration number 658133
 
Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK  
  From: Iñigo Martinez Lasala [mailto:imartinez@vectorsf.com]
Sent: 13 April 2010 08:56
To: Renato Oliveira
Cc: pgsql-admin@postgresql.org; Steve Fisher
Subject: Re: [ADMIN] 'COPY' Statement


 

I think it depends of your datamodel and specifically of your table constraints.

For example, if you have a table with a pk generated by a sequence, you will have to deal with it before importing into your New_DB02. But for a plain log table without constrains you can dump it an import into your "just loaded" New_DB02 without any problem.


-----Original Message-----
From: Renato Oliveira <renato.oliveira@grant.co.uk>
To: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org>
Cc: Steve Fisher <steve.fisher@grant.co.uk>
Subject: [ADMIN] 'COPY' Statement
Date: Tue, 13 Apr 2010 08:42:19 +0100

Dear all,

 

I just had a thought, another one. Please the guys who have the powerful knowledge I need your help ;-)

 

Let’s suppose

I have Three  DB servers ,physical servers – let’s call them:

·        Original_DB01

·        New_DB02

·        Third_DB03

 

1 – Let’s suppose I have a PostgreSQL on ‘Original_DB01’ server with a Data Base of 162GB in size.

2 – Let’s suppose I buy a new server ‘New_DB02’ - all singing all dancing cool server, RAID10, 25GB RAM, 64Bit etc.

3 – Let’s suppose I backup the old server with pg_dump to a file “pg_dump_original_DB01”

4 – Let’s also suppose I backup the schema only from ‘Original_DB01’and restore it to ‘Third_DB03’.

5 – Straight after I backup the schema, I stop PostgreSQL on ‘Original_DB01’.

6 – Then I restore the schema to ‘Third_DB03’ and point the application to it.

7 – Let’s suppose the application can create tables and can start using the database on ‘Third_DB03’ and adding new data.

8 – Let’s suppose I restore the FULL  pg_dump file “pg_dump_original_DB01” to ‘New_DB02’

9 – Once the pg_dump has been restored to ‘New_DB02’ I can then point the application to it and data will be added to it straight away.

 

Question:

10 – Can I use ‘COPY’ statement to transfer the data from ‘Third_DB03’ to ‘New_DB02’ without ‘WIPING’ or Deleting the existing data?

I want to merge the DATA from ‘Third_DB03’ to ‘New_DB02’ without deleting the Database or the existing data on the existing database.

 

Let me explain the Idea behind this craziness.

I am using ‘Third_DB03’ because the amount of data to be transferred from it, will be minimum, and easy to and quick to ‘COPY’

 

This application has two part components (Data and Alarms)

We can get the data back in a very hard way, but without the data being in the database we can’t generate alarms.

 

I would very much appreciate all your help on this idea, please.

I would be very glad and grateful for any comments and improvements on this idea, picking the holes etc.

 

Thank you very much in advance

 

Renato

 

 

 


  Renato Oliveira
Systems Administrator
e-mail: renato.oliveira@grant.co.uk   Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk   Grant Instruments (Cambridge) Ltd
 
Company registered in England, registration number 658133
 
Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK  
 
 
   

P Please consider the environment before printing this email


CONFIDENTIALITY: The information in this e-mail and any attachments is confidential. It is intended only for the named recipients(s). If you are not the named recipient please notify the sender immediately and do not disclose the contents to another person or take copies.  
VIRUSES: The contents of this e-mail or attachment(s) may contain viruses which could damage your own computer system. Whilst Grant Instruments (Cambridge) Ltd has taken every reasonable precaution to minimise this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should therefore carry out your own virus checks before opening the attachment(s).  
OpenXML: For information about the OpenXML file format in use within Grant Instruments please visit our website


 
   

P Please consider the environment before printing this email


CONFIDENTIALITY: The information in this e-mail and any attachments is confidential. It is intended only for the named recipients(s). If you are not the named recipient please notify the sender immediately and do not disclose the contents to another person or take copies.  
VIRUSES: The contents of this e-mail or attachment(s) may contain viruses which could damage your own computer system. Whilst Grant Instruments (Cambridge) Ltd has taken every reasonable precaution to minimise this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should therefore carry out your own virus checks before opening the attachment(s).  
OpenXML: For information about the OpenXML file format in use within Grant Instruments please visit our website

Re: 'COPY' Statement

От
Iñigo Martinez Lasala
Дата:
Then you will have no problem.

When using pg_dump do NOT include the "-c" flag (clean). For example:

pg_dump -U postgres -Fc -d DB_3 -f DB_3.dmp DB3
pg_restore -U postgres -Fc -d DB_2 DB3_dmp 2>error_import.log 1>import.log

Or if you prefer, use plan text (in order to check dump file with an editor prior restoring):
pg_dump -U postgres -Fp -d DB_3 -f DB_3.dmp DB3
psql -U postgres -d DB_2 -f DB_3.dmp 2>error_import.log 1>import.log

You could alse use the "insert" way, but restore will take much longer.


-----Original Message-----
From: Renato Oliveira <renato.oliveira@grant.co.uk>
To: Iñigo Martinez Lasala <imartinez@vectorsf.com>
Subject: RE: [ADMIN] 'COPY' Statement
Date: Tue, 13 Apr 2010 09:13:06 +0100

 I see! Primary Key. Cool.

I think you can work out and change the primary key, that’s is the reason I am moving the application to third_server so we have very little data to worry about.

 

Thank you very much for all your help and comments.

 

In resume I can ‘COPY all the data from third_server and use the same COPY statement to migrate the data back to new_server?

 

Thank you

 

Renato

 


  Renato Oliveira
Systems Administrator
e-mail: renato.oliveira@grant.co.uk   Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk   Grant Instruments (Cambridge) Ltd
 
Company registered in England, registration number 658133
 
Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK  
  From: Iñigo Martinez Lasala [mailto:imartinez@vectorsf.com]
Sent: 13 April 2010 09:02
To: Renato Oliveira
Cc: pgsql-admin
Subject: RE: [ADMIN] 'COPY' Statement


 

Primary Key. :)

A Primary Key is always a constraint, since it cannot be duplicated. For example, if you have a PK generated by a sequence starting at zero, when you restore your third database into new server you will have duplicated values in this column and COPY command will fail.

-----Original Message-----
From: Renato Oliveira <renato.oliveira@grant.co.uk>
To: Iñigo Martinez Lasala <imartinez@vectorsf.com>
Subject: RE: [ADMIN] 'COPY' Statement
Date: Tue, 13 Apr 2010 08:59:34 +0100

 Dear Iñigo,

 

Thank you very much. What do you mean by ‘pk’?

I am sorry.

 

Thank you very much

 

Renato

 


  Renato Oliveira
Systems Administrator
e-mail: renato.oliveira@grant.co.uk   Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk   Grant Instruments (Cambridge) Ltd
 
Company registered in England, registration number 658133
 
Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK  
  From: Iñigo Martinez Lasala [mailto:imartinez@vectorsf.com]
Sent:13 April 2010 08:56
To:Renato Oliveira
Cc:pgsql-admin@postgresql.org; Steve Fisher
Subject: Re: [ADMIN] 'COPY' Statement


 

I think it depends of your datamodel and specifically of your table constraints.

For example, if you have a table with a pk generated by a sequence, you will have to deal with it before importing into your New_DB02. But for a plain log table without constrains you can dump it an import into your "just loaded" New_DB02 without any problem.


-----Original Message-----
From: Renato Oliveira <renato.oliveira@grant.co.uk>
To: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org>
Cc: Steve Fisher <steve.fisher@grant.co.uk>
Subject: [ADMIN] 'COPY' Statement
Date: Tue, 13 Apr 2010 08:42:19 +0100

Dear all,

 

I just had a thought, another one. Please the guys who have the powerful knowledge I need your help ;-)

 

Let’s suppose

I have Three  DB servers ,physical servers – let’s call them:

·        Original_DB01

·        New_DB02

·        Third_DB03

 

1 – Let’s suppose I have a PostgreSQL on ‘Original_DB01’ server with a Data Base of 162GB in size.

2 – Let’s suppose I buy a new server ‘New_DB02’ - all singing all dancing cool server, RAID10, 25GB RAM, 64Bit etc.

3 – Let’s suppose I backup the old server with pg_dump to a file “pg_dump_original_DB01”

4 – Let’s also suppose I backup the schema only from ‘Original_DB01’and restore it to ‘Third_DB03’.

5 – Straight after I backup the schema, I stop PostgreSQL on ‘Original_DB01’.

6 – Then I restore the schema to ‘Third_DB03’ and point the application to it.

7 – Let’s suppose the application can create tables and can start using the database on ‘Third_DB03’ and adding new data.

8 – Let’s suppose I restore the FULL  pg_dump file “pg_dump_original_DB01” to ‘New_DB02’

9 – Once the pg_dump has been restored to ‘New_DB02’ I can then point the application to it and data will be added to it straight away.

 

Question:

10 – Can I use ‘COPY’ statement to transfer the data from ‘Third_DB03’ to ‘New_DB02’ without ‘WIPING’ or Deleting the existing data?

I want to merge the DATA from ‘Third_DB03’ to ‘New_DB02’ without deleting the Database or the existing data on the existing database.

 

Let me explain the Idea behind this craziness.

I am using ‘Third_DB03’ because the amount of data to be transferred from it, will be minimum, and easy to and quick to ‘COPY’

 

This application has two part components (Data and Alarms)

We can get the data back in a very hard way, but without the data being in the database we can’t generate alarms.

 

I would very much appreciate all your help on this idea, please.

I would be very glad and grateful for any comments and improvements on this idea, picking the holes etc.

 

Thank you very much in advance

 

Renato

 

 

 


  Renato Oliveira
Systems Administrator
e-mail: renato.oliveira@grant.co.uk   Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk   Grant Instruments (Cambridge) Ltd
 
Company registered in England, registration number 658133
 
Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK  
 
 
   

P Please consider the environment before printing this email


CONFIDENTIALITY: The information in this e-mail and any attachments is confidential. It is intended only for the named recipients(s). If you are not the named recipient please notify the sender immediately and do not disclose the contents to another person or take copies.  
VIRUSES: The contents of this e-mail or attachment(s) may contain viruses which could damage your own computer system. Whilst Grant Instruments (Cambridge) Ltd has taken every reasonable precaution to minimise this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should therefore carry out your own virus checks before opening the attachment(s).  
OpenXML: For information about the OpenXML file format in use within Grant Instruments please visit our website


 
   

P Please consider the environment before printing this email


CONFIDENTIALITY: The information in this e-mail and any attachments is confidential. It is intended only for the named recipients(s). If you are not the named recipient please notify the sender immediately and do not disclose the contents to another person or take copies.  
VIRUSES: The contents of this e-mail or attachment(s) may contain viruses which could damage your own computer system. Whilst Grant Instruments (Cambridge) Ltd has taken every reasonable precaution to minimise this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should therefore carry out your own virus checks before opening the attachment(s).  
OpenXML: For information about the OpenXML file format in use within Grant Instruments please visit our website


 
   

P Please consider the environment before printing this email


CONFIDENTIALITY: The information in this e-mail and any attachments is confidential. It is intended only for the named recipients(s). If you are not the named recipient please notify the sender immediately and do not disclose the contents to another person or take copies.  
VIRUSES: The contents of this e-mail or attachment(s) may contain viruses which could damage your own computer system. Whilst Grant Instruments (Cambridge) Ltd has taken every reasonable precaution to minimise this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should therefore carry out your own virus checks before opening the attachment(s).  
OpenXML: For information about the OpenXML file format in use within Grant Instruments please visit our website