problem with PQsendQuery/PQgetResult and COPY FROM statement
| От | max.poletto@gmail.com | 
|---|---|
| Тема | problem with PQsendQuery/PQgetResult and COPY FROM statement | 
| Дата | |
| Msg-id | 1148064660.016529.192030@g10g2000cwb.googlegroups.com обсуждение исходный текст | 
| Ответы | Re: problem with PQsendQuery/PQgetResult and COPY FROM statement Re: problem with PQsendQuery/PQgetResult and COPY FROM statement Re: problem with PQsendQuery/PQgetResult and COPY FROM statement | 
| Список | pgsql-hackers | 
I attempt to execute a "COPY test FROM STDIN" using PQsendQuery.
Subsequently, after the appropriate poll()/PQconsumeInput/PQisBusy
calls, PQgetResult returns tens of millions of (invalid?) non-null
PGresults.  This behavior seems incorrect, and sometimes causes my
application to exhaust memory and crash.
The postgres version is 8.1.3.
I can reproduce the problem in about 50 lines of C.  I include below
(1) the code, (2) a psql dump of the table in question, (3) the code's
output.
I'd appreciate any insight or suggestions you may have.
Thanks,
max poletto
======================================================================
(1) code
// compile with: g++ -Wall -O -o pgtest pgtest.cc -lpq
#include <cerrno>
#include <cstdio>
#include <ctime>
#include <vector>
using namespace std;
#include <libpq-fe.h>
#include <sys/poll.h>
void retrieve(PGconn *conn, time_t timeout)
{ vector<PGresult *> res; while (1) {   int r;   do {     struct pollfd pfds[1];     pfds[0].fd = PQsocket(conn);
pfds[0].events= POLLIN;     pfds[0].revents = 0;     r = poll(pfds, sizeof(pfds) / sizeof(struct pollfd), 1000 *
 
timeout);   } while (r < 0 && errno == EINTR);   if (r <= 0 || !PQconsumeInput(conn))     return;   int i = 0;
PGresult*oldr = 0;   while (!PQisBusy(conn)) {     PGresult *r = PQgetResult(conn);     if (r) {
res.push_back(r);      if (++i % 5000000 == 0) { printf("%d results\n", i); }       if (r == oldr) { printf("r==oldr
(%p)\n",r); }       oldr = r;     } else {       printf("PQgetResult return 0 after %d results\n", i);       return;
}   } }
 
}
int main()
{ PGconn *conn = PQconnectdb("dbname=testdb user=postgres"); if (!conn)   return -1; if (PQsendQuery(conn, "COPY test
FROMSTDIN") > 0) {   retrieve(conn, 20);   if (PQputCopyEnd(conn, 0) < 0)     return -1; } PQfinish(conn); return 0;
 
}
======================================================================
(2) psql session
root@tm01-5% psql testdb postgres
Welcome to psql 8.1.3, the PostgreSQL interactive terminal.
Type:  \copyright for distribution terms      \h for help with SQL commands      \? for help with psql commands      \g
orterminate with semicolon to execute query      \q to quit
 
testdb=# \d test    Table "public.test"Column |  Type   | Modifiers
--------+---------+-----------a      | integer |b      | integer |c      | integer |
testdb=# select * from test;a | b | c
---+---+---1 | 2 | 3
(1 row)
testdb=#
======================================================================
(3) output
root@tm01-5% ./pgtest
5000000 results
10000000 results
15000000 results
20000000 results
25000000 results
PQgetResult return 0 after 25649299 results
4.640u 4.696s 0:09.34 99.8%     0+0k 0+0io 0pf+0w
In this toy example, the process VM size exceeds 2GB before PQgetResult
finally returns 0.  The real application, which has already allocated
~1GB prior to the query, runs out of memory (3GB limit on 32-bit linux)
before PQgetResult ever returns 0.
		
	В списке pgsql-hackers по дате отправления: