Обсуждение: RFC: a new try for an official community approved certification

Hi everyone,

At PgCon I mentioned a was going to create this thread so here we go:

== why do we need one?

The fact that there is no official certification is a real problem
because anyone can offer theirs and have bad answers or confuse people
about how things work. You can think that people looking for a
certification should ensure first but what if the company has a good
reputation because other certifications they have are good?

And this is an example of this happening, recently I got a
certification from a less-known company, and I found at least one
question in which there weren't any correct answers so I had to choose
the answer that was at least partially correct. And of course this
could happen to anyone.

== we have tried… and failed

Several years ago there was a project for creating a pensum that all
companies that wanted to offer the "official" certification should
comply with. But being postgres developed and improved so fast we
ended up with half the pensum and we already had to adapt part of it
because of the changes in the new version.

But of course there was the problem of not really prepared trainers, a
know a guy that was certified by a well known postgres company but he
clearly understood wrong somethings because he was teaching that you
can restore a pg_dump'ed database and then apply wals from the
original server and have PITR (doh).

== so the problems

- mantaining/updating a pensum would be difficult
- is not only a matter of covering the pensum, it should be done right
- other problems are:
-- current certifications are not neutral and cover also tools from
the company doing the certification
-- even if they have the same name, certifications cover normally
different subjects in different companies

== so, my idea

- Let's focus on creating just one certification covering only
core/contrib functionalities of postgres, and let companies have their
own certifications for third party products.
- don't try to force the pensum, just let's create a database of
validated questions and validated answers. If we focus on only one
certification (something like associate postgresql or something like
that) we can let companies decide if they want to have 1, 2 or 3
modules to prepare people to give the exam.
- Who creates the questions? And who validates them? The answer to
both questions is "trainers" from companies that want to offer the
exam. So, a trainer creates a question and another trainer (hopefully
from another company) validates the question and the answers for the
question.
Questions and answers should also be validated for postgres version
and operating system.

If a company cannot successfully prepare people for the exam they
could check if they are teaching right (I mean that they are not
saying old/not exact things).

So, having a certification made from a pool of validated
questions/answers don't only will help companies choosing trainings
but will in effect improve the quality of trainers.

There are a lot of open questions yet, like where the exams will be
taken. In the community infrastructure? in the company's?

but I guess we have still enough for start commenting.

--
Jaime Casanova
Director de Servicios Profesionales
SYSTEMGUARDS - Consultores de PostgreSQL

On 6/1/23 15:04, Jaime Casanova wrote:
> Hi everyone,
> 
> At PgCon I mentioned a was going to create this thread so here we go:
> 

> 
> So, having a certification made from a pool of validated
> questions/answers don't only will help companies choosing trainings
> but will in effect improve the quality of trainers.
> 
> There are a lot of open questions yet, like where the exams will be
> taken. In the community infrastructure? in the company's?
> 
> but I guess we have still enough for start commenting.
1) Who oversees all of the above?

2) How are the questions and answers developed without leaking the 
information out to test takers?

3) Does core mean just the community Postgres?

> 
> --
> Jaime Casanova
> Director de Servicios Profesionales
> SYSTEMGUARDS - Consultores de PostgreSQL
> 
> 

-- 
Adrian Klaver
adrian.klaver@aklaver.com

Am 02.06.23 um 00:31 schrieb Adrian Klaver:

> 2) How are the questions and answers developed without leaking the 
> information out to test takers?
I guess it'd make sense to ask the LPI folks how they handle this.

Maybe we could even slip under their umbrella, once the community agrees 
on a (closed) group working on the actual exam questions...?

Just my 2p,
-- 
Gunnar "Nick" Bluth

Eimermacherweg 106
D-48159 Münster

Mobil +49 172 8853339
Email: gunnar.bluth@pro-open.de
__________________________________________________________________________
"Ceterum censeo SystemD esse delendam" - Cato

> but I guess we have still enough for start commenting.


Note that LPI-Japan (a non profit organization) already has a Postgres-certification
exam that can be taken world-wide through Pearson-VUE:

https://oss-db.jp/eng_exam

AFAIK right now this is Japanese-only. But there have been attempts to offer this in
English too - see this old thread:


https://www.postgresql.org/message-id/flat/8ae8dc03d25c7d3364a5eea2eec81c1f%40smtp.hushmail.com#ce98b3345adb10949ceb454d567f9039

I guess LPI-Japan could have interesting input to this discussion.


Bye,
Chris.

On Thu, Jun 1, 2023 at 5:04 PM Jaime Casanova
<jcasanov@systemguards.com.ec> wrote:
>
> Hi everyone,
>
> At PgCon I mentioned a was going to create this thread so here we go:
>
[...]
>
> There are a lot of open questions yet, like where the exams will be
> taken. In the community infrastructure? in the company's?
>
> but I guess we have still enough for start commenting.
>

Hi everyone,

Thanks for your comments. Mine was a long trip to return to Ecuador
and then I had a lot of pending things to do.

First, let me mention the good ideas I read on this thread:

- Gunnar and Chris suggested that we could ask a NPO that is already
successfully doing a cert, namely LPI. Do any of you have a contact
there to ask this?
- Katherine mentioned something that is what I had in mind and maybe
didn't express it correctly. The idea is that companies doing training
work together for this to happen.

When I said "community developed" I certainly wasn't thinking of any
person to be involved. We should have a closed group, the development
of the cert should be done through a system and we need a process to
give credentials to anyone that will be involved. For putting
companies on the professional services we follow a process outlined
here https://www.postgresql.org/about/policies/services-and-hosting/
we could have something similar for this.

Now, some of the questions:

> 1) Who oversees all of the above?

while I spoke of teachers developing the questions and validating them
(and that could be a medium-size group),  we could have a small group
of well-known community members (not necessarily teachers) that could
act as a committee for administrative actions.

> 2) How are the questions and answers developed without leaking the information out to test takers?

My idea is to have a system, very similar to the one for adding
professional services or news or whatever. Of course, that system
publishes after validation, something we don't want to happen with
this. Not everyone has access to that system.

> 3) Does core mean just the community Postgres?

Yes. I really think that it is not correct to say that you cannot
certify because you know how to use repmgr instead of patroni or
pg_auto_failover, and we cannot force everyone to know all the tools.
Every company developing a tool could decide to create their own certs
about their tools.




--
Jaime Casanova
Director de Servicios Profesionales
SYSTEMGUARDS - Consultores de PostgreSQL

> Hi everyone,
> 
> Thanks for your comments. Mine was a long trip to return to Ecuador
> and then I had a lot of pending things to do.
> 
> First, let me mention the good ideas I read on this thread:
> 
> - Gunnar and Chris suggested that we could ask a NPO that is already
> successfully doing a cert, namely LPI. Do any of you have a contact
> there to ask this?

I have a contact to LPI-Japan. Can you clarify what questions/requests
you want to ask them?

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Mon, Jun 5, 2023 at 8:35 PM Tatsuo Ishii <ishii@sraoss.co.jp> wrote:
>
> > Hi everyone,
> >
> > Thanks for your comments. Mine was a long trip to return to Ecuador
> > and then I had a lot of pending things to do.
> >
> > First, let me mention the good ideas I read on this thread:
> >
> > - Gunnar and Chris suggested that we could ask a NPO that is already
> > successfully doing a cert, namely LPI. Do any of you have a contact
> > there to ask this?
>
> I have a contact to LPI-Japan. Can you clarify what questions/requests
> you want to ask them?
>

I think what scares people the most is about the security, meaning
that if the cert is somewhat community-made how they keep the
questions not public,
I guess part of the solution was already explained in which companies
using a software are the ones making the standard (with users having
to log in).

Probably other questions have to do with the method to take the exams,
actually once the exam is online (no controlled environment, which is
the way most companies do it I guess) no matter what we do people can
reveal the questions just by taking pictures or video of the screen.

ANother question was about who oversees the process, probably about
the organization of all this.

Basically what we need is some guidance about all of it ;)

--
Jaime Casanova
Director de Servicios Profesionales
SYSTEMGUARDS - Consultores de PostgreSQL

On 6/7/23 13:48, Jaime Casanova wrote:
> On Mon, Jun 5, 2023 at 8:35 PM Tatsuo Ishii <ishii@sraoss.co.jp> wrote:
>>

> ANother question was about who oversees the process, probably about
> the organization of all this.

That is the main question. Without an organization in place the rest of 
it is wasted effort.

> 
> Basically what we need is some guidance about all of it ;)
> 

Who is we?

-- 
Adrian Klaver
adrian.klaver@aklaver.com

On Wed, Jun 7, 2023 at 6:09 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:
>
> On 6/7/23 13:48, Jaime Casanova wrote:
[...]
> >
> > Basically what we need is some guidance about all of it ;)
> >
>
> Who is we?
>

we: the community
we: the ones that suggested to ask LPI how they organized everything
we: me and my other personality

you choose ;)

--
Jaime Casanova
Director de Servicios Profesionales
SYSTEMGUARDS - Consultores de PostgreSQL

Guys,

I still have bad memories when we tried this the last time some 15 years ago.

Let's do this in small steps:
- Create a mail list just for this thread
- Instead of creating an approved certification how's about developing an RFC of sorts outlining expected competencies
forvarious classifications of knowledge (that should be real fun) is expected to carry out certain tasks. Then, after
there'ssome sort of agreement in the community.
 

Hope this helps.

Robert Bernier
robert.bernier@percona.com

On 6/8/23 06:26, Robert Bernier wrote:
> Guys,
> 
> I still have bad memories when we tried this the last time some 15 years ago.
> 
> Let's do this in small steps:
> - Create a mail list just for this thread
> - Instead of creating an approved certification how's about developing an RFC of sorts outlining expected
competenciesfor various classifications of knowledge (that should be real fun) is expected to carry out certain tasks.
Then,after there's some sort of agreement in the community.
 

+1

> 
> Hope this helps.
> 
> Robert Bernier
> robert.bernier@percona.com
> 
> 

-- 
Adrian Klaver
adrian.klaver@aklaver.com

>> but I guess we have still enough for start commenting.
> 
> 
> Note that LPI-Japan (a non profit organization) already has a Postgres-certification
> exam that can be taken world-wide through Pearson-VUE:
> 
> https://oss-db.jp/eng_exam
> 
> AFAIK right now this is Japanese-only. But there have been attempts to offer this in
> English too - see this old thread:

I heard from LPI-Japan that they have already offered English
PostgreSQL certifications:

https://oss-db.jp/eng_exam
https://lpi.or.jp/en/
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

>>
>> Note that LPI-Japan (a non profit organization) already has a Postgres-certification
>> exam that can be taken world-wide through Pearson-VUE:
>>
>> https://oss-db.jp/eng_exam
>>
>> AFAIK right now this is Japanese-only. But there have been attempts to offer this in
>> English too - see this old thread:
>
> I heard from LPI-Japan that they have already offered English
> PostgreSQL certifications:
>
> https://oss-db.jp/eng_exam
> https://lpi.or.jp/en/
> --
> Tatsuo Ishii
> SRA OSS LLC
> English: http://www.sraoss.co.jp/index_en/
> Japanese:http://www.sraoss.co.jp
>

Hi all,

we have confirmation that LPI-Japan is offering their OSS-DB in english
once again! I'm forwarding, with permission, the below message that was sent
off-list.

Bye,
Chris.


> From: 浅利康二 <koji.asari@lpi.or.jp>
> Subject: Re: RFC: a new try for an official community approved certification
> Date: 14 June 2023 at 08:17:05 CEST
>
> Hello all,
>
> Thank you for the introduction, Ishii-san!
> We started to conduct the OSS-DB certification in 2011, which is a PostgreSQL certification that was taken over from
SRAOSS. 
> And, many candidates take the exam and the number is still increasing every year.
> Major exam takers are Japanese currently, but we also have the English version of the OSS-DB exam and you can take
theexam at a number of Pearson Vue locations around the world. 
> Feel free to ask us, if you have some questions.
>
> Regards,
>
> ---
> Koji Asari
> LPI-Japan
> https://lpi.or.jp/en/

On Thu, Jun 1, 2023 at 6:05 PM Jaime Casanova
<jcasanov@systemguards.com.ec> wrote:
> And this is an example of this happening, recently I got a
> certification from a less-known company, and I found at least one
> question in which there weren't any correct answers so I had to choose
> the answer that was at least partially correct. And of course this
> could happen to anyone.

I don't think a community approved certification is a good idea. The
problem that you're complaining about here is that the certification
was bad, and the way to solve that problem is to have something
better. I'm in favor of certifications being better. If they are going
to exist, they should be good, just like anything else. But having
something be community-approved doesn't make it automatically better,
because community members can make mistakes and do shoddy work just
like anybody else. Making something a community effort does the
following things:

- It makes it the official version of a thing, which means more people
will use that thing even if it's worse than some other version of the
thing.

- It means there's a group of people who are in charge of that thing,
and it's usually very hard to replace that group of people if they
stop doing the thing well.

- It means that decisions need to be made by consensus, which is good
for things where the cost of mistakes is high (like core development)
and things where by their nature only one can exist (like the
postgresql.org web site).

A great example of competition-is-good-for-the-project is psycopg2. If
there were just one Python connector for PostgreSQL, it wouldn't be
called psycopg2. Because we never picked an official one, it helped a
bunch of projects thrive, and the one most people use now is the one
that won the competition. If there had only ever been one, it probably
wouldn't be as good.

A good example of the difficulty of managing a large overhaul through
the community process is the documentation. There's a lot of great
content in our documentation, but there's also a lot of old stuff that
doesn't really get updated much and maybe isn't even really that
relevant. Much of the valuable content is buried multiple layers down
in the documentation in places where it's not necessarily that easy to
find, while things of more peripheral importance are quite prominent.
But getting agreement on how to address these problems, or even what
the problems are, is pretty hard. However, since the official
documentation is a thing of which there can by nature only be one,
we're kind of locked in to accepting only those improvements that can
make their way through that process.

But we're not locked into such a ponderous process for certifications.
The way we're going to get a great certification if someone goes and
writes a bunch of great questions and then updates them regularly and
vigorously based on feedback and changes in each new release -- and
that requires either a fanatically dedicated volunteer, or for that
person to be getting paid to do that work.

--
Robert Haas
EDB: http://www.enterprisedb.com

On 2023-06-14 22:42, Robert Haas wrote:
> The way we're going to get a great certification if someone goes and
> writes a bunch of great questions and then updates them regularly and
> vigorously based on feedback and changes in each new release -- and
> that requires either a fanatically dedicated volunteer, or for that
> person to be getting paid to do that work.

The PG project isn't exactly short of cash though, so that *might* be
worth exploring.

+ Justin