Re: Fw: Problem with psycopg2, bytea, and memoryview

On Thu, Aug 1, 2013 at 6:37 AM, Frank Millman <frank@chagford.com> wrote:
> Hi all
>
> I recently posted about a problem I was having with psycopg2 returning a
> 'memoryview' for bytea objects. Daniele Varrazzo very kindly gave me an
> effective workaround.
>
> I had previously asked the question on the python mailing list. Among other
> replies, I received the following comment. I felt that I should forward it
> to this list, but I am passing it on for information only. I have no
> personal opinion on the matter.

Hi Frank, thank you for forwarding the thread.


> ----- Original Message ----- From: "Terry Reedy" <tjreedy@udel.edu>
> Newsgroups: gmane.comp.python.general
> To: <python-list@python.org>
> Sent: Wednesday, July 31, 2013 7:59 PM
> Subject: Re: Problem with psycopg2, bytea, and memoryview
>
>
>> On 7/31/2013 9:07 AM, Antoine Pitrou wrote:
>>>
>>> Frank Millman <frank <at> chagford.com> writes:
>>>>
>>>>
>>>> Thanks for that, Antoine. It is an improvement over tobytes(), but i am
>>>> afraid it is still not ideal for my purposes.
>>>
>>>
>>> I would suggest asking the psycopg2 project why they made this choice,
>>> and
>>> if they would reconsider. Returning a memoryview doesn't make much sense
>>> IMHO.
>>
>>
>> I agree.
>> "memoryview objects allow Python code to access the internal data of an
>> object that supports the buffer protocol without copying."
>> Example: the binary image data of an image object.
>> They are not intended to be a standalone objects when there is an obvious
>> alternative (in this case, bytes).

Until a not very long time ago, this was exactly the case: upon
reading bytea we were handed over some data to pass to a libpq
function to decode. The resulting decoded string was to be released by
PQfreemem, and the memoryview was the right object to reconcile the
lifetime of the python object with this deallocation requirement.
Later things have changed: because of the change in bytea format in
PostgreSQL 9.0 (the new format was the default and wasn't handled by
libpq < 9.0, which created widespread problems) we wrote our own
parser. As a consequence the memoryview is not really needed anymore,
but we preferred to avoid breaking programs already using the current
interface.

So yes: bytes (and str on Py2) is definitely a better object to get in
Python. It is already on the list of what I want in a version that
would break compatibility in order to gain feature or improve the
adapter in other ways (e.g. psycopg3).

-- Daniele