Re: lots of large objects and toast

Tom Lane wrote:
> Lincoln Yeoh <lylyeoh@mecomb.com> writes:
> >> There's never been much enthusiasm among the core developers for large
> >> objects at all --- we see them as a poor substitute for allowing large
> >> values directly.  (The "TOAST" work scheduled for 7.1 will finally
> >> resolve that issue, I hope.)  So no one's felt like working on improving
> >> the large-object implementation.
>
> > On the practical side, say I want to insert/read a large amount of
> > information into/from a TOAST field. How should I do it?
> > Is there a pipe method where I can continuously print to/read from?
>
> Not at the moment, but that's obviously going to be a necessary feature
> if we want to make the existing flavor of large objects obsolete.  There
> have been some preliminary discussions about it --- AFAIR no one's laid
> out a complete proposal yet.

    Yes,  we  already saw that problem. And looking at some other
    databases they seem to deal with it the same way I'm inclined
    to do.

    The  BLOB/CLOB  data  types  have  to  be references. A dummy
    reference is created by a special function that's used in the
    values  for  INSERT.   If  such  a dummy ref occurs in a heap
    tuple to be  stored,  a  real  empty  reference  is  created.
    Wherever  we store the content, we need some new interface in
    libpq to access them like files with seek/read/write.

    It's somehow like our current LO, but all LOs for one  column
    reside  in  one shadow table, and the system definitely knows
    when an item is obsolete.

    The type output functions just output the reference, and  new
    libpq  functions  then  gain  access to it. It will need some
    enhancements on the SQL level too, to make  pg_dump  able  to
    save/restore  them.  The shadow tables might have a different
    relkind, so they aren't accessible via  normal  SQL.  But  if
    COPY can, why not?

    All  that  is  still  a little vague and there are problems I
    don't want to talk about right now.   I  need  to  get  TOAST
    ready  for 7.1, then go back to FOREIGN KEY and implement the
    file buffered trigger queue.  So I don't expect BLOB/CLOB  to
    be ready before 7.2.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #