Re: Bootstrap DATA is a pita

On 3/7/15 6:02 PM, Stephen Frost wrote:
> * Andrew Dunstan (andrew@dunslane.net) wrote:
>> On 03/07/2015 05:46 PM, Andres Freund wrote:
>>> On 2015-03-07 16:43:15 -0600, Jim Nasby wrote:
>>>> Semi-related... if we put some special handling in some places for bootstrap
>>>> mode, couldn't most catalog objects be created using SQL, once we got
>>>> pg_class, pg_attributes and pg_type created? That would theoretically allow
>>>> us to drive much more of initdb with plain SQL (possibly created via
>>>> pg_dump).
>>> Several people have now made that suggestion, but I *seriously* doubt
>>> that we actually want to go there. The overhead of executing SQL
>>> commands in comparison to the bki stuff is really rather
>>> noticeable. Doing the majority of the large number of insertions via SQL
>>> will make initdb noticeably slower. And it's already annoyingly
>>> slow. Besides make install it's probably the thing I wait most for
>>> during development.
>>
>> My reaction exactly. We should not make users pay a price for
>> developers' convenience.

How often does a normal user actually initdb? I don't think it's that 
incredibly common. Added time to our development cycle certainly is a 
concern though.

> Just to clarify, since Jim was responding to my comment, my thought was
> *not* to use SQL commands inside initdb, but rather to use PG to create
> the source files that we have today in our tree, which wouldn't slow
> down initdb at all.

Yeah, I was thinking SQL would make it even easier, but perhaps not. 
Since the other options here seem to have hit a dead end though, it 
seems your load it into tables idea is what we've got left...

>>> That's besides the fact that SQL commands aren't actually that
>>> comfortably editable in bulk.
>>
>> Indeed.
>
> No, they aren't, but having the data in a table in PG, with a way to
> easily export to the format needed by BKI, would make bulk updates much
> easier..

My thought was that pg_dump would be useful here, so instead of hand 
editing you'd just make changes in a live database and then dump it.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com