Обсуждение: Vectorization of some functions and improving pg_list interface

On 2023-08-24 10:07, Maxim Orlov wrote:
> 1) Why do I need to specify the number of elements in the list in the
> function name?

This is reminding me of something someone (Tom?) worked on sort of
recently.

Ah, yes: 
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=1cff1b9

I wasn't following closely, but that and the discussion link
may answer some questions.

Regards,
-Chap

24.08.2023 17:07, Maxim Orlov wrote:
> Hi!
> 
> Recently, I've been playing around with pg_lists and realize how 
> annoying (maybe, I was a bit tired) some stuff related to the lists.
> For an example, see this code
> List     *l1 = list_make4(1, 2, 3, 4),
>           *l2 = list_make4(5, 6, 7, 8),
>           *l3 = list_make4(9, 0, 1, 2);
> ListCell *lc1, *lc2, *lc3;
> 
> forthree(lc1, l1, lc2, l2, lc3, l3) {
> ...
> }
> 
> list_free(l1);
> list_free(l2);
> list_free(l3);
> 
> There are several questions:
> 1) Why do I need to specify the number of elements in the list in the 
> function name?
> Compiler already knew how much arguments do I use.
> 2) Why I have to call free for every list? I don't know how to call it 
> right, for now I call it vectorization.
>      Why not to use simple wrapper to "vectorize" function args?
> 
> So, my proposal is:
> 1) Add a simple macro to "vectorize" functions.
> 2) Use this macro to "vectorize" list_free and list_free_deep functions.
> 3) Use this macro to "vectorize" bms_free function.
> 4) "Vectorize" list_makeN functions.
> 
> For this V1 version, I do not remove all list_makeN calls in order to 
> reduce diff, but I'll address
> this in future, if it will be needed.
> 
> In my view, one thing still waiting to be improved if foreach loop. It 
> is not very handy to have a bunch of
> similar calls foreach, forboth, forthree and etc. It will be ideal to 
> have single foreach interface, but I don't know how
> to do it without overall interface of the loop.
> 
> Any opinions are very welcome!

Given use case doesn't assume "zero" arguments, it is possible to 
implement "lists_free" with just macro expansion (following code is not 
checked, but close to valid):

#define VA_FOR_EACH(invoke, join, ...) \
    CppConcat(VA_FOR_EACH_, VA_ARGS_NARGS(__VA_ARGS__))( \
        invoke, join, __VA_ARGS__)
#define VA_FOR_EACH_1(invoke, join, a1) \
    invoke(a1)
#define VA_FOR_EACH_2(invoke, join, a1, a2) \
    invoke(a1) join() invoke(a2)
#define VA_FOR_EACH_3(invoke, join, a1, a2, a3) \
    invoke(a1) join() invoke(a2) join() invoke(a3)
... up to 63 args

#define VA_SEMICOLON() ;

#define lists_free(...) \
    VA_FOR_EACH(list_free, VA_SEMICOLON, __VA_ARGS__)

#define lists_free_deep(...) \
    VA_FOR_EACH(list_free_deep, VA_SEMICOLON, __VA_ARGS__)

There could be couple of issues with msvc, but they are solvable.

------

Regards,
Yura

06.09.2023 13:24, Yura Sokolov wrote:
> 24.08.2023 17:07, Maxim Orlov wrote:
>> Hi!
>>
>> Recently, I've been playing around with pg_lists and realize how 
>> annoying (maybe, I was a bit tired) some stuff related to the lists.
>> For an example, see this code
>> List     *l1 = list_make4(1, 2, 3, 4),
>>           *l2 = list_make4(5, 6, 7, 8),
>>           *l3 = list_make4(9, 0, 1, 2);
>> ListCell *lc1, *lc2, *lc3;
>>
>> forthree(lc1, l1, lc2, l2, lc3, l3) {
>> ...
>> }
>>
>> list_free(l1);
>> list_free(l2);
>> list_free(l3);
>>
>> There are several questions:
>> 1) Why do I need to specify the number of elements in the list in the 
>> function name?
>> Compiler already knew how much arguments do I use.
>> 2) Why I have to call free for every list? I don't know how to call it 
>> right, for now I call it vectorization.
>>      Why not to use simple wrapper to "vectorize" function args?
>>
>> So, my proposal is:
>> 1) Add a simple macro to "vectorize" functions.
>> 2) Use this macro to "vectorize" list_free and list_free_deep functions.
>> 3) Use this macro to "vectorize" bms_free function.
>> 4) "Vectorize" list_makeN functions.
>>
>> For this V1 version, I do not remove all list_makeN calls in order to 
>> reduce diff, but I'll address
>> this in future, if it will be needed.
>>
>> In my view, one thing still waiting to be improved if foreach loop. It 
>> is not very handy to have a bunch of
>> similar calls foreach, forboth, forthree and etc. It will be ideal to 
>> have single foreach interface, but I don't know how
>> to do it without overall interface of the loop.
>>
>> Any opinions are very welcome!
> 
> Given use case doesn't assume "zero" arguments, it is possible to 
> implement "lists_free" with just macro expansion (following code is not 
> checked, but close to valid):
> 
> #define VA_FOR_EACH(invoke, join, ...) \
>      CppConcat(VA_FOR_EACH_, VA_ARGS_NARGS(__VA_ARGS__))( \
>          invoke, join, __VA_ARGS__)
> #define VA_FOR_EACH_1(invoke, join, a1) \
>      invoke(a1)
> #define VA_FOR_EACH_2(invoke, join, a1, a2) \
>      invoke(a1) join() invoke(a2)
> #define VA_FOR_EACH_3(invoke, join, a1, a2, a3) \
>      invoke(a1) join() invoke(a2) join() invoke(a3)
> ... up to 63 args
> 
> #define VA_SEMICOLON() ;
> 
> #define lists_free(...) \
>      VA_FOR_EACH(list_free, VA_SEMICOLON, __VA_ARGS__)
> 
> #define lists_free_deep(...) \
>      VA_FOR_EACH(list_free_deep, VA_SEMICOLON, __VA_ARGS__)
> 
> There could be couple of issues with msvc, but they are solvable.

Given we could use C99 compound literals, list contruction could be 
implemented without C vaarg functions as well

     List *
     list_make_impl(NodeTag t, int n, ListCell *datums)
     {
         List       *list = new_list(t, n);
         memcpy(list->elements, datums, sizeof(ListCell)*n);
         return list;
     }

     #define VA_COMMA() ,

     #define list_make__m(Tag, type, ...) \
        list_make_impl(Tag, VA_ARGS_NARGS(__VA_ARGS__), \
          ((ListCell[]){ \
            VA_FOR_EACH(list_make_##type##_cell, VA_COMMA, __VA_ARGS__) \
          }))


     #define list_make(...) list_make__m(T_List, ptr, __VA_ARGS__)
     #define list_make_int(...) list_make__m(T_IntList, int, __VA_ARGS__)
     #define list_make_oid(...) list_make__m(T_OidList, oid, __VA_ARGS__)
     #define list_make_xid(...) list_make__m(T_XidList, xid, __VA_ARGS__)

(code is not checked)

If zero arguments (no arguments) should be supported, it is tricky 
because of mvsc, but solvable.