Обсуждение: libpg: large object problems

Hi,

I'm trying to figure out how to get large objects working using libpg.
The problem is that whenever I do an lo_open on an object, I get back
"-1" as the file descriptor (and of course, an lo_read on that fails).

  Oid id = lo_creat (dbConn, INV_READ | INV_WRITE);
  fd = lo_open (dbConn, id, INV_WRITE);

That's what I'm basically doing. "id" ends up being a somewhat sensible
looking large-ish number.

I've also tried doing lo_open's of files that I lo_import'ed (and can
successfully lo_export using the console, so I know they're "in there").
Same problem.

Any ideas what I'm doing wrong? I've included the full text of my simple
test case below. I've also tried this with a remote PostgreSQL server,
both times verifying that I could run queries and such (so the
connection to the server is working on at least that level :-)

thanks!

-Seth


---------Full Text of Test Program--------

static PGconn *dbConn = NULL;

int main(int argc, char **argv) {
  char * error;
  Oid id;
  int fd;

  dbConn = PQconnectdb("");
  error = PQerrorMessage(dbConn);

  if(strlen(error) > 0){
    fprintf(stderr, "DB connection error to gargamel: %s\n", error);
    return -1;
  }

  Oid id = lo_creat (dbConn, INV_READ | INV_WRITE);

  fd = lo_open (dbConn, id, INV_WRITE);

  printf ("fd is: %d, id is: %d\n", fd, id);

  return 0;
}

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> I'm trying to figure out how to get large objects working using libpg.
> The problem is that whenever I do an lo_open on an object, I get back
> "-1" as the file descriptor (and of course, an lo_read on that fails).

Large object operations must be done within a transaction. Try issuing
a "begin" before you call lo_creat:

res = PQexec(dbConn, "begin");
PQclear(res);

// Large object stuff goes here....

res = PQexec(dbConn, "end");
PQclear(res);
PQfinish(dbConn);


- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200305030957
-----BEGIN PGP SIGNATURE-----
Comment: http://www.turnstep.com/pgp.html

iD8DBQE+s8tnvJuQZxSWSsgRAog6AKCAmVAsn0SF8mnrKQiVp+7SdxTZSQCfWvUf
op+BcWSQCJIwyVJ72J8+KUA=
=HMkb
-----END PGP SIGNATURE-----

> > I'm trying to figure out how to get large objects working using libpg.
> > The problem is that whenever I do an lo_open on an object, I get back
> > "-1" as the file descriptor (and of course, an lo_read on that fails).
>
> Large object operations must be done within a transaction. Try issuing
> a "begin" before you call lo_creat:

Oh! Thanks Greg. I shouldn't have missed that in the docs.

I'm unfortunately still having problems :-( Can I only do one lo_
operation per transaction? e.g. is this valid?

  PQexec(dbConn, "begin")
  id = lo_creat(...)
  fd = lo_open(id, ...)
  lo_write(fd, ...)
  PQexec(dbConn, "end")

If I only put one lo_ operation per transaction, I get back "0" for the
fd (is this a valid fd?). I guess this is an improvement over -1 ;-) It
seems I can do an lo_write at this point, but lo_read still doesn't let
me read anything.

I've pasted the test code I'm using below. When I run it on my computer,
I get:

id is: 567036
WRITE fd is 0
Wrote 68 bytes
READ fd is 0
object size is 0
Read 0 bytes
Buffer Contents {

}

Thanks,

-Seth

-------------Test Source Code--------------

#include <stdio.h>
#include <libpq-fe.h> //in /usr/include
#include <libpq/libpq-fs.h>
#include <string.h>

static PGconn *dbConn = NULL;

static int writeData (PGconn *dbConn) {
  Oid id;
  char *buffer;
  PGresult *res;
  int bytes_written, fd;

  res = PQexec(dbConn, "begin");
  PQclear(res);

  id = lo_creat (dbConn, INV_READ | INV_WRITE);
  printf ("id is: %d\n", id);

  res = PQexec(dbConn, "end");
  PQclear(res);

  res = PQexec(dbConn, "begin");
  PQclear(res);

  fd = lo_open (dbConn, id, INV_WRITE);
  printf ("WRITE fd is %d\n", fd);

  buffer = "Test, test test. Test of the emergency broadcast
system.\nTest test!\n";
  bytes_written = lo_write(dbConn, fd, buffer, strlen(buffer));
  printf ("Wrote %d bytes\n", bytes_written);

  lo_close(dbConn, id);

  res = PQexec(dbConn, "end");
  PQclear(res);

  return id;
}

static void readData (PGconn *dbConn, Oid id) {
  PGresult *res;
  char *buffer;
  int bytes_read, fd, object_size;

  res = PQexec(dbConn, "begin");
  PQclear(res);

  fd = lo_open (dbConn, id, INV_WRITE);
  printf ("READ fd is %d\n", fd);

  /* Get the Object's Size by seeking to the end and back */
  lo_lseek(dbConn, fd, 0, SEEK_END);
  object_size = lo_tell(dbConn, fd);
  printf ("object size is %d\n", object_size);
  lo_lseek(dbConn, fd, 0, SEEK_SET);

  /* Do the read */
  buffer = (char *)malloc (sizeof(char) * (object_size + 1));
  buffer[0] = '\0';
  bytes_read = lo_read(dbConn, fd, buffer, object_size);

  res = PQexec(dbConn, "end");
  PQclear(res);

  printf ("Read %d bytes\n", bytes_read);
  printf ("Buffer Contents {\n%s\n}\n", buffer);
}

int main(int argc, char **argv) {
  char * error;
  Oid id;

  dbConn = PQconnectdb("");
  error = PQerrorMessage(dbConn);

  if(strlen(error) > 0){
    fprintf(stderr, "DB connection error to gargamel: %s\n", error);
    return -1;
  }

  id = writeData(dbConn);
  readData(dbConn, id);

  PQfinish(dbConn);

  return 0;
}

Seth Nickell <snickell@stanford.edu> writes:
> If I only put one lo_ operation per transaction, I get back "0" for the
> fd (is this a valid fd?). I guess this is an improvement over -1 ;-) It
> seems I can do an lo_write at this point, but lo_read still doesn't let
> me read anything.

I think the lo_open() for reading needs to specify INV_READ.

An error that may or may not be hurting you is that the lo_close in
the write part should specify fd not id.  It is possible that that
mistake is making the writer transaction abort.  The lack of checks
for errors makes it difficult to tell, but you could possibly look
in the postmaster log for clues...

            regards, tom lane

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> I'm unfortunately still having problems :-( Can I only do one lo_
> operation per transaction? e.g. is this valid?

Perfectly valid.

> If I only put one lo_ operation per transaction, I get back "0" for the
> fd (is this a valid fd?).

Yes, "0" is not only valid, but fairly common, as it is the fist "slot"
used for large objects.

Other than that, I think you are almost there if you institute the changes
Tom mentioned. Particularly, make sure you check the values of the lo_*
operations to see if they are returning a negative number.

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200305032058

-----BEGIN PGP SIGNATURE-----
Comment: http://www.turnstep.com/pgp.html

iD8DBQE+tGctvJuQZxSWSsgRAr9aAJ4zX5Kl3TxO6CyzPifuXXGgjH+UUACggNs3
zIwyRHIV2UymA3fCy8miJSY=
=NYiw
-----END PGP SIGNATURE-----