Re: [GSoC 2021 Proposal] Developing Testing Suite for PGWEB

On Tue, Apr 6, 2021 at 5:37 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
>
> Hi Chirag,
>
> On 4/6/21 11:21 AM, Chirag Kasat wrote:
> > Hello Everyone,
> > I am Chirag Kasat and I am interested in the following project for GSoC
> > 2021. I would be grateful if you can have a look and give me suggestions
> > to improve.
> > This is the link to my draft proposal:
> > https://docs.google.com/document/d/1V2FsPKmWdG8Ng0-p4ID3la_0sYAtYDD8MM1O55b-cCQ/edit?usp=sharing
> > <https://docs.google.com/document/d/1V2FsPKmWdG8Ng0-p4ID3la_0sYAtYDD8MM1O55b-cCQ/edit?usp=sharing>
> > I have also attached the PDF version.
>
> Thank you for putting this proposal together. The overall approach
> appears to be reasonable and will certainly help add some more assurance
> around the pgweb codebase.
>
> My recommendation would be around the scope of the project. I think it
> is conceivable that you could easily spend the entire summer just
> focused on the unit test portion of the project based upon both the size
> of the codebase and current lack of test automation.
>
> I have familiarity with Selenium as far back as when I was doing my own
> summer college projects -- I know a lot has changed since then, but I
> have often found that the cost of upkeeping a Selenium stack can often
> outweigh the benefits. However, this was typically in a rapidly changing
> environment -- the frontend of pgweb does not change all that often.
>
> I don't know if we really need performance testing at this point in time
> based on our traffic pattern and usage of varnish. It could certainly be
> interesting to see, and may help identify a few areas we could optimize,
> but I would certainly rank it below unit testing.

To follow up on that one, I don't think we need "generic performance
testing", but as I understand the proposal it would also cover things
like "a page runs too many SQL queries" and *that* would be a very
useful thing to notice if it happens, as it can severely  effect the
cache miss performance, so that's always worth looking into.

I agree that it's probably not the most important part of things
though -- functionality testing is a lot *more* important.


> Accessibility testing would be very much welcome. Even having some
> design guidelines on what we could do better on that front would be a
> huge next step.
>
> So in other words, my recommendations would be to focus on building up a
> core set of unit tests along with a focus on how we can improve our
> overall accessibility.
>
> This is just one person's opinion, but I hope it provides a good
> starting point on what to look at.

Agreed on other accounts, and also to state that I definitely think
having a proper test suite for pgweb is going to be a great
enhancement.

-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/