missing NOTIFY events in async mode
От | Jan Urbański |
---|---|
Тема | missing NOTIFY events in async mode |
Дата | |
Msg-id | 53A85FA2.50605@wulczer.org обсуждение исходный текст |
Список | psycopg |
Hi, I got a bug report for txpostgres about NOTIFY events being missed if a large number of events were sent at the same time. After investigating, I believe it's a psycopg bug causing the behaviour. Commit d9c0b8166f9ba17eacb09d5861409ab862447331 removed conn_notifies_process from pq_fetch for what I guess were good reasons. The problem is that the async execution patch will call pq_fetch after an asynchronous query is finished and any notifies received at that point will not be exposed to the application until the next poll() cycle. What actually happens is that conn_poll calls _conn_poll_query, which then calls pq_is_busy. That will check for notifies, but only the ones that have already been received on the socket at that time. With enough notifies to span several TCP packets, some of them will remain unread by the client. Note that the result of _conn_poll_query will be PSYCO_POLL_OK anyway. conn_poll then continues to call pq_get_last_result, which calls down to PQgetResult, causing the remainder of the data to be read from the socket, along with the pending notifies. However, they won't be parsed and made available on the Python Connection object. Attached is a patch that solved the problem for me and a test case that demonstrates how the connection is missing notifies. I'm not 100% sure if that's the correct place to put a conn_notifies_process() call - perhaps it should be in conn_poll inside the block starting with: if (res == PSYCO_POLL_OK && self->async && self->async_cursor) { In any case, it should have a C comment attached, but as I wasn't sure if the fix is in the right place, I haven't added one yet. Credit for reporting the bug and preparing the initial test case goes to Max Walton, which I have taken the liberty of CC'ing. As an aside: what's the correct process for sending patches to Psycopg? I see that tickets are tracked in Lighthouse, but the project is still taking pull requests on GitHub? Best, Jan
Вложения
В списке psycopg по дате отправления: