Обсуждение: Testing

Поиск
Список
Период
Сортировка

Testing

От
Jussi Mikkola
Дата:
I noticed, that there is not any story of how PostgreSQL is tested. I
could find a FAQ, and some stuff about regression tests, but what I am
looking for, is that is there a small group, that is responsible for
testing, what tools they are using, if bugs are found, what happens then
etc.

Is there a location, where I could find something, or is there someone I
could ask, how it is done? (So I can write something about it...)

Rgs,

Jussi



Re: Testing

От
Shridhar Daithankar
Дата:
Jussi Mikkola wrote:

> I noticed, that there is not any story of how PostgreSQL is tested. I
> could find a FAQ, and some stuff about regression tests, but what I am
> looking for, is that is there a small group, that is responsible for
> testing, what tools they are using, if bugs are found, what happens then
> etc.

When you compile postgresql from sources, after make is done, you can run make
check which runs lots of tests to make sure that it works according to design.

After that, you can use it for development and production. If you find a bug,
submit a bug report to pgsql-bugs@postgresql.org.

There is no special testing team. The user base is the testing team..:-)

HTH

  Shridhar


Re: Testing

От
Jussi Mikkola
Дата:
Okay. The tests. Is it so, that if I make a new feature, I also make a
test for it? What if that new feature runs without errors, but is very
slow, uses plenty of memory etc. ? Can it leak memory? Is there a test
for all new features? What happens, if there is no test for a feature?
Is there a link from a bug to change tests?

Well, when I look at the release history, and for example 7.3, there was
7.3.4 that came out before 7.4. So there are not very many releases.
That means, that there can't be very many serious bugs left around. Many
bugs would mean that there would need to be minor releases quite often.

Usually it is not just luck, that a product works. But I would like to
tell, that there is a reason, and that also the future releases are very
reliable ;-) Certainly, I would not like to tell, that as a user, you
are the tester... But of course, it is different, if we say, that as a
user, you have the ability to test the software, and verify that it
works properly ;-)

Jussi


Shridhar Daithankar wrote:

> Jussi Mikkola wrote:
>
>> I noticed, that there is not any story of how PostgreSQL is tested. I
>> could find a FAQ, and some stuff about regression tests, but what I
>> am looking for, is that is there a small group, that is responsible
>> for testing, what tools they are using, if bugs are found, what
>> happens then etc.
>
>
> When you compile postgresql from sources, after make is done, you can
> run make check which runs lots of tests to make sure that it works
> according to design.
>
> After that, you can use it for development and production. If you find
> a bug, submit a bug report to pgsql-bugs@postgresql.org.
>
> There is no special testing team. The user base is the testing team..:-)
>
> HTH
>
>  Shridhar
>
>


--
Jussi Mikkola                Partner, Project Manager
Bonware Oy                   gsm +358 40 830 7561
Tekniikantie 21              tel +358 9 2517 5570
02150 Espoo                  fax +358 9 2517 5571
Finland                      www.bonware.com





Re: Testing

От
Richard Huxton
Дата:
On Friday 28 November 2003 13:41, Jussi Mikkola wrote:
> Okay. The tests. Is it so, that if I make a new feature, I also make a
> test for it?

Depends on the feature really.

> What if that new feature runs without errors, but is very
> slow, uses plenty of memory etc. ? Can it leak memory? Is there a test
> for all new features? What happens, if there is no test for a feature?
> Is there a link from a bug to change tests?

If the feature is slow/incomplete etc then it will only be used by those that
really need that feature. Badly written code wont make it into the system
because the core developers will refuse to commit it.
It is not easy to define a test for all features - do you test to see if it
works or it won't break? What about testing how all features interact? How do
you define what is/is not a feature? Having said that, you've seen the
regression tests and they check the standard operations work as they should.

> Well, when I look at the release history, and for example 7.3, there was
> 7.3.4 that came out before 7.4. So there are not very many releases.
> That means, that there can't be very many serious bugs left around. Many
> bugs would mean that there would need to be minor releases quite often.

If there were many bugs, or bugs that were common then PostgreSQL wouldn't be
in use. If you want to see the process of releasing a new version check the
mailing list archives for pgsql-hackers and -bugs. A new release goes through
a number of beta-tests and release candidates before being finalised.

> Usually it is not just luck, that a product works. But I would like to
> tell, that there is a reason, and that also the future releases are very
> reliable ;-) Certainly, I would not like to tell, that as a user, you
> are the tester... But of course, it is different, if we say, that as a
> user, you have the ability to test the software, and verify that it
> works properly ;-)

With PG, some of the users are also the testers. Some of the users are the
developers, some are advocates, some build add-on tools and some help out on
the mailing lists.
Of course, you don't have to do any of this, you could just run 7.3.x and wait
until 7.4.1 or 7.4.2 is out, by which time any bugs should be pretty obscure.
What I try to do is test the development version of my applications against
new releases while sticking with the last stable release on my customers'
machines.

--
  Richard Huxton
  Archonet Ltd

Re: Testing

От
Shridhar Daithankar
Дата:
On Friday 28 November 2003 19:11, Jussi Mikkola wrote:
> Okay. The tests. Is it so, that if I make a new feature, I also make a
> test for it? What if that new feature runs without errors, but is very

Yes. New tests are to be developed.

> slow, uses plenty of memory etc. ? Can it leak memory? Is there a test
> for all new features? What happens, if there is no test for a feature?
> Is there a link from a bug to change tests?

No. Postgresql does not have any bug tracking system and that kind of
tracability. It is a TODO ATM.

> Usually it is not just luck, that a product works. But I would like to
> tell, that there is a reason, and that also the future releases are very
> reliable ;-) Certainly, I would not like to tell, that as a user, you
> are the tester... But of course, it is different, if we say, that as a
> user, you have the ability to test the software, and verify that it
> works properly ;-)

Well, that's the way open source works. User is tester, user is developer if
he offers work and user is support person as well because he knows by
experience what works and why etc.

 Shridhar