Endless loop calling PL/Python set returning functions

Поиск
Список
Период
Сортировка
От Alexey Grishchenko
Тема Endless loop calling PL/Python set returning functions
Дата
Msg-id CAH38_tnKLFvCCbiaaQmVhsvboher4Vqv680FqR8QEkWfSO2CXQ@mail.gmail.com
обсуждение исходный текст
Ответы Re: Endless loop calling PL/Python set returning functions  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-hackers
Hello

There is a bug in implementation of set-returning functions in PL/Python. When you call the same set-returning function twice in a single query, the executor falls to infinite loop which causes OOM. Here is a simple reproduction for this issue:

CREATE OR REPLACE FUNCTION func(iter int) RETURNS SETOF int AS $$
return xrange(iter)
$$ LANGUAGE plpythonu;

select func(3), func(4);

The endless loop is caused by the fact that PL/Python uses PLyProcedure structure for each of the functions, containing information specific for the function. This structure is used to store the result set iterator returned by the Python function call. But in fact, when we call the same function twice, PL/Python uses the same structure for both calls, and the same result set iterator (PLyProcedure.setof), which is being constantly updated by one function after another. When the iterator reaches the end, the first function sets it to null. Then Postgres calls the second function, it receives NULL iterator and calls Python function once again, receiving another iterator. This is an endless loop

In fact, for set-returning functions in Postgres we should use a set of SRF_* functions, which gives us an access to function call context (FuncCallContext). In my implementation this context is used to store the iterator for function result set, so these two calls would have separate iterators and the query would succeed.

Another issue with calling the same set-returning function twice in the same query, is that it would delete the input parameter of the function from the global variables dictionary at the end of execution. With calling the function twice, this code attempts to delete the same entry from global variables dict twice, thus causing KeyError. This is why the function PLy_function_delete_args is modified as well to check whether the key we intend to delete is in the globals dictionary.

New regression test is included in the patch.

--
Best regards,
Alexey Grishchenko
Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Robert Haas
Дата:
Сообщение: Re: Parallel Aggregate
Следующее
От: David Steele
Дата:
Сообщение: Re: [PROPOSAL] Client Log Output Filtering