Обсуждение: Soliciting Feedback on Improving Server-Side Programming Documentation

Поиск
Список
Период
Сортировка

Soliciting Feedback on Improving Server-Side Programming Documentation

От
Corey Huinker
Дата:
Over the past few months, I've been familiarizing myself with postgres server side programming in C.

My attempts to educate myself were slow and halting. The existing server side programming documentation has some examples, but those examples didn't show me how do what I wanted to do, and my research-via-google was highly circular, almost always pointing back to the documentation I had already found lacking, or a copy of it.

Most of what I have learned I have culled from asking people on IRC, or bugging people I've met through user groups and PgConf. In all cases, people have been extremely helpful. However, this method is inefficient, because we're using two people's time, one of whom has to tolerate my incessant questions and slow learning pace. 

Furthermore, the helpful suggestions I received boiled down to:
1. The function/macro/var you're looking for is PG_FOO, git grep PG_FOO
2. Look in blah.c which does something like what you're trying to do
3. The comments in blah.h do a good job of listing and explaining this macro or that

#1 git grep is a helpful reflex for discovering examples on my own, but it requires that I have a term to search on in the first place, and too often I don't know what I don't know.

#2 is the gold standard in terms of correctness (the code had to have worked at least up to the last checkin date), and in terms of discoverability it often gave me names of new macros to search for, coding patterns, etc. However, I was always left with the questions: How would I have figured this out on my own? How is the next person going to figure it out? Why doesn't anybody document this?

#3 Often answers the last question in #2: It *is* documented, but that documentation is not easily discoverable by conventional means.

So what I'd like to do is migrate some of the helpful information in the header files into pages of web searchable documentation, and also to revamp the existing documentation to be more relevant.

Along the way, I collected a list of things I wished I'd had from the start:
  • A list of all the GETARG_* macros. It would have been especially great if this were in table form:   Your Parameter Is A / Use This Macro / Which Gives This Result Type / Working example.
  • A list/table of the DatumGet* macros. I'm aware that many of them overlap/duplicate others. That'd be good to know too.
  • The table at http://www.postgresql.org/docs/9.1/static/errcodes-appendix.html has the numeric codes and PL/PGSQL constants enumerated. It'd be nice if it had the C #define as well
  • The SPI documentation mentions most/all of the SPI functions, but I couldn't find documentation on the SPI variables like SPI_processed and SPI_tuptable.
  • Examples and explanation of how PG_TRY()/PG_CATCH work. How to add context callbacks.
  • Direct Function Calls
  • A comparison of the two modes of writing SRF functions (Materialize vs multi-call)
  • Less explanation of how to do write V0-style functions. That was called the "old style" back in version 7.1. Why is that information up front in the documentation when so much else is sequestered in header files?
Some of these things may seem obvious/trivial to you. I would argue that they're only obvious in retrospect, and the more obvious-to-you things we robustly document, the quicker we accumulate programmers who are capable of agreeing that it's obvious, and that's good for the community.

I'm aware that some of these APIs change frequently. In those cases, I suggest that we make note of that on the same page.

Because I'm still going through the learning curve, I'm probably the least qualified to write the actual documentation. However, I have a clear memory of what was hard to learn and I have the motivation to make it easier on the next person. That makes me a good focal point for gathering, formatting, and submitting the documentation in patch form. I'm volunteering to do so. What I need from you is:
  • Citations of existing documentation in header files that could/should be exposed in our more formal documentation.
  • Explanations of any of the things above, which I can then reformat into proposed documentation.
  • A willingness to review the proposed new documentation 
  • Reasoned explanations for why this is a fool's errand
You supply the expertise, I'll write the patch.

Thanks in advance.

Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
"Joshua D. Drake"
Дата:
On 03/15/2016 10:02 AM, Corey Huinker wrote:

> Some of these things may seem obvious/trivial to you. I would argue that
> they're only obvious in retrospect, and the more obvious-to-you things
> we robustly document, the quicker we accumulate programmers who are
> capable of agreeing that it's obvious, and that's good for the community.
>
> I'm aware that some of these APIs change frequently. In those cases, I
> suggest that we make note of that on the same page.

I think this is all great. You may find some automated assistance from 
doxygen.postgresql.org .

Sincerely,

JD


-- 
Command Prompt, Inc.                  http://the.postgres.company/                        +1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.



Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
"Shulgin, Oleksandr"
Дата:
On Tue, Mar 15, 2016 at 6:02 PM, Corey Huinker <corey.huinker@gmail.com> wrote:
Over the past few months, I've been familiarizing myself with postgres server side programming in C.

My attempts to educate myself were slow and halting. The existing server side programming documentation has some examples, but those examples didn't show me how do what I wanted to do, and my research-via-google was highly circular, almost always pointing back to the documentation I had already found lacking, or a copy of it.

Most of what I have learned I have culled from asking people on IRC, or bugging people I've met through user groups and PgConf. In all cases, people have been extremely helpful. However, this method is inefficient, because we're using two people's time, one of whom has to tolerate my incessant questions and slow learning pace. 

Furthermore, the helpful suggestions I received boiled down to:
1. The function/macro/var you're looking for is PG_FOO, git grep PG_FOO
2. Look in blah.c which does something like what you're trying to do
3. The comments in blah.h do a good job of listing and explaining this macro or that

There's also a good deal of README files in the source tree, so I would add:

4. find src -name 'README*'

Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
Corey Huinker
Дата:

I think this is all great. You may find some automated assistance from doxygen.postgresql.org .

Sincerely,

JD

doxygen is great as far as it goes, but it does a great job of separating function definition from the comment explaining the function, so I have to drill into the raw source anyway.

Also, doxygen isn't very helpful with macros, and a lot of functions in the internals are actually macros.
 

Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
Corey Huinker
Дата:

On Tue, Mar 15, 2016 at 1:19 PM, Shulgin, Oleksandr <oleksandr.shulgin@zalando.de> wrote:
There's also a good deal of README files in the source tree, so I would add:

4. find src -name 'README*'

That too. But README's don't show up (easily) in a google search, so they elude discovery. We should want to make discovery easy to the uninitiated.

Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
"Joshua D. Drake"
Дата:
On 03/15/2016 10:30 AM, Corey Huinker wrote:
>
> On Tue, Mar 15, 2016 at 1:19 PM, Shulgin, Oleksandr
> <oleksandr.shulgin@zalando.de <mailto:oleksandr.shulgin@zalando.de>> wrote:
>
>     There's also a good deal of README files in the source tree, so I
>     would add:
>
>     4. find src -name 'README*'
>
>
> That too. But README's don't show up (easily) in a google search, so
> they elude discovery. We should want to make discovery easy to the
> uninitiated.

I don't think anyone is arguing with you. I think we are trying to point 
you to sources for your project.

JD

-- 
Command Prompt, Inc.                  http://the.postgres.company/                        +1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.



Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
Corey Huinker
Дата:
On Tue, Mar 15, 2016 at 1:35 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
On 03/15/2016 10:30 AM, Corey Huinker wrote:

On Tue, Mar 15, 2016 at 1:19 PM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de <mailto:oleksandr.shulgin@zalando.de>> wrote:

    There's also a good deal of README files in the source tree, so I
    would add:

    4. find src -name 'README*'


That too. But README's don't show up (easily) in a google search, so
they elude discovery. We should want to make discovery easy to the
uninitiated.

I don't think anyone is arguing with you. I think we are trying to point you to sources for your project.


I didn't mean to imply that anyone was arguing. All responses so far have been positive.

Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
Álvaro Hernández Tortosa
Дата:

    I started a similar thread with probably similar concerns: http://www.postgresql.org/message-id/56D1A6AA.6080303@8kdata.com

    I believe this effort should be done. I added to my TODO list to compile a list of used functions in a selection of picked extensions to use that as a starting point of an "API".

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



On 15/03/16 13:02, Corey Huinker wrote:
Over the past few months, I've been familiarizing myself with postgres server side programming in C.

My attempts to educate myself were slow and halting. The existing server side programming documentation has some examples, but those examples didn't show me how do what I wanted to do, and my research-via-google was highly circular, almost always pointing back to the documentation I had already found lacking, or a copy of it.

Most of what I have learned I have culled from asking people on IRC, or bugging people I've met through user groups and PgConf. In all cases, people have been extremely helpful. However, this method is inefficient, because we're using two people's time, one of whom has to tolerate my incessant questions and slow learning pace. 

Furthermore, the helpful suggestions I received boiled down to:
1. The function/macro/var you're looking for is PG_FOO, git grep PG_FOO
2. Look in blah.c which does something like what you're trying to do
3. The comments in blah.h do a good job of listing and explaining this macro or that

#1 git grep is a helpful reflex for discovering examples on my own, but it requires that I have a term to search on in the first place, and too often I don't know what I don't know.

#2 is the gold standard in terms of correctness (the code had to have worked at least up to the last checkin date), and in terms of discoverability it often gave me names of new macros to search for, coding patterns, etc. However, I was always left with the questions: How would I have figured this out on my own? How is the next person going to figure it out? Why doesn't anybody document this?

#3 Often answers the last question in #2: It *is* documented, but that documentation is not easily discoverable by conventional means.

So what I'd like to do is migrate some of the helpful information in the header files into pages of web searchable documentation, and also to revamp the existing documentation to be more relevant.

Along the way, I collected a list of things I wished I'd had from the start:
  • A list of all the GETARG_* macros. It would have been especially great if this were in table form:   Your Parameter Is A / Use This Macro / Which Gives This Result Type / Working example.
  • A list/table of the DatumGet* macros. I'm aware that many of them overlap/duplicate others. That'd be good to know too.
  • The table at http://www.postgresql.org/docs/9.1/static/errcodes-appendix.html has the numeric codes and PL/PGSQL constants enumerated. It'd be nice if it had the C #define as well
  • The SPI documentation mentions most/all of the SPI functions, but I couldn't find documentation on the SPI variables like SPI_processed and SPI_tuptable.
  • Examples and explanation of how PG_TRY()/PG_CATCH work. How to add context callbacks.
  • Direct Function Calls
  • A comparison of the two modes of writing SRF functions (Materialize vs multi-call)
  • Less explanation of how to do write V0-style functions. That was called the "old style" back in version 7.1. Why is that information up front in the documentation when so much else is sequestered in header files?
Some of these things may seem obvious/trivial to you. I would argue that they're only obvious in retrospect, and the more obvious-to-you things we robustly document, the quicker we accumulate programmers who are capable of agreeing that it's obvious, and that's good for the community.

I'm aware that some of these APIs change frequently. In those cases, I suggest that we make note of that on the same page.

Because I'm still going through the learning curve, I'm probably the least qualified to write the actual documentation. However, I have a clear memory of what was hard to learn and I have the motivation to make it easier on the next person. That makes me a good focal point for gathering, formatting, and submitting the documentation in patch form. I'm volunteering to do so. What I need from you is:
  • Citations of existing documentation in header files that could/should be exposed in our more formal documentation.
  • Explanations of any of the things above, which I can then reformat into proposed documentation.
  • A willingness to review the proposed new documentation 
  • Reasoned explanations for why this is a fool's errand
You supply the expertise, I'll write the patch.

Thanks in advance.

Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
Corey Huinker
Дата:
On Tue, Mar 15, 2016 at 4:38 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    I started a similar thread with probably similar concerns: http://www.postgresql.org/message-id/56D1A6AA.6080303@8kdata.com

    I believe this effort should be done. I added to my TODO list to compile a list of used functions in a selection of picked extensions to use that as a starting point of an "API".

    Regards,

    Álvaro


Clearly we have the same goal in mind. I don't know how I missed seeing your thread.

 

Re: Soliciting Feedback on Improving Server-Side Programming Documentation

От
Craig Ringer
Дата:
On 16 March 2016 at 01:02, Corey Huinker <corey.huinker@gmail.com> wrote:
 
#1 git grep is a helpful reflex for discovering examples on my own, but it requires that I have a term to search on in the first place, and too often I don't know what I don't know.

Yep. This can be painful when you're trying to figure out what macro to use to access fields in some struct, the PG_GETARG macro for a type, etc.
 
#2 is the gold standard in terms of correctness (the code had to have worked at least up to the last checkin date), and in terms of discoverability it often gave me names of new macros to search for, coding patterns, etc. However, I was always left with the questions: How would I have figured this out on my own? How is the next person going to figure it out? Why doesn't anybody document this?

Indeed. In particular, it's not always obvious when you're new to the codebase which files relate to what. The codebase is fairly well structured but you have to get a decent understanding of how the bits of the server fit together before you really know where to look.

A src/README docs with brief descriptions of the tree structure and key files would be helpful, or additions to src/DEVELOPERS .
 
#3 Often answers the last question in #2: It *is* documented, but that documentation is not easily discoverable by conventional means.

Particularly when it's a comment on a function that you have to know about before you can find the comment.
 
So what I'd like to do is migrate some of the helpful information in the header files into pages of web searchable documentation, and also to revamp the existing documentation to be more relevant.

I'm not convinced by that part. Rather than moving stuff into the SGML docs, which are frankly painful to maintain and less visible when editing the relevant code, I'd be much happier to see README-style docs located close to the code they're relevant to, and/or cross-referencing comments in headers and sources to help tell you where to look.

Along the way, I collected a list of things I wished I'd had from the start:
  • A list of all the GETARG_* macros. It would have been especially great if this were in table form:   Your Parameter Is A / Use This Macro / Which Gives This Result Type / Working example.
Yes, though that's an area where "git grep" does a reasonable job it's a bit awkward.

This probably *should* be in the SGML docs, in the C extensions section, along with the related DatumGet and PG_RETURN_ functions and macros.

... at least where the C define is different to the plpgsql constant. Which is occasonally the case.
  • The SPI documentation mentions most/all of the SPI functions, but I couldn't find documentation on the SPI variables like SPI_processed and SPI_tuptable.
 
  • Examples and explanation of how PG_TRY()/PG_CATCH work. How to add context callbacks.
... and warnings about their limitations. In particular, that PG_TRY / PG_CATCH doesn't imply a savepoint and you can't just merrily carry on after (say) an SPI error. 
  • Direct Function Calls
Yeah, with a few examples, including one showing caching of the fmgr info for a FunctionCall by info not oid.
  • A comparison of the two modes of writing SRF functions (Materialize vs multi-call)
Worthwhile, yeah. 
  • Less explanation of how to do write V0-style functions. That was called the "old style" back in version 7.1. Why is that information up front in the documentation when so much else is sequestered in header files?
I'd just like to delete that entirely. 

Some of these things may seem obvious/trivial to you. I would argue that they're only obvious in retrospect, and the more obvious-to-you things we robustly document, the quicker we accumulate programmers who are capable of agreeing that it's obvious, and that's good for the community.

I still remember them being very non-obvious, so I agree.
 
Because I'm still going through the learning curve, I'm probably the least qualified to write the actual documentation.

You're *extremely* qualified to make notes of what's hard, though, which is something people who've worked on the codebase for a while tend to forget.
 
I've been trying to write little  bits of docs as I go and as I learn. Going to write one on how timelines work soon.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services