Re: Regression testing for psql

On tis, 2010-05-25 at 06:23 -0400, Stephen Frost wrote:
> * Robert Haas (robertmhaas@gmail.com) wrote:
> > > Of course, if people want to suggest tests that just shouldn't be
> > > included, I can go through and strip things out.
> > 
> > Well...  I'm a little reluctant to believe that we should have 3.3M of
> > tests for the entire backend and 5M of tests just for psql.  Then,
> > too, there's the fact that many of these tests fail on my machine
> > because my username is not sfrost, and/or because of row-ordering
> > differences on backslash commands without enough ORDER BY to fully
> > determine the output order.
> 
> Yeah, you know, I had fully intended to go grepping through the output
> last night to check for things like that, but my wife decided I needed
> sleep instead. :)  Sorry about that.  Still, it's more of a general
> proposal than something I think should be committed as-is.  Should we
> try to deal with those kinds of differences, or just eliminate the tests
> which are dependent on username, etc?  It definitely strikes me that
> there's a fair bit of code in psql we're not exercising in some fashion
> in the regression suite... :/

Maybe pg_regress is not the right framework to test that sort of thing.