Re: psql copy command - 1 char limitation on delimiter

On 10/12/2010 08:28 PM, Bruce Momjian wrote:
> Steve Crawford wrote:
>
>> On 09/25/2010 07:03 AM, Tom Lane wrote:
>>
>>> rey<reywang@optonline.net>   writes:
>>>
>>>
>>>> Why limit this to a single character?
>>>>
>>>>
>>> Performance.  Believe it or not, breaking fields at the delimiter is
>>> a significant factor in COPY speed.
>>>
>>>             regards, tom lane
>>>
>>>
>>>
>> I agree that that multi-character (or even regex) delimiters would be
>> useful. Would it be reasonable for the copy process to differentiate
>> between single character delimiters which could be processed in
>> "high-speed" mode and multi-character or regex delimiters which would be
>> available as needed albeit at the expense of a performance hit?
>>
> I am not sure you are aware but Postgres never confuses delimiters from
> data because it uses a backslash before literal data that matches
> delimiters.
>
>
Yes, I am. But the discussion was about using multi-character strings as
delimiters.

But while I have encountered files using multiple-character delimiters,
I'm finding myself leaning toward the camp that says that such cases are
better processed externally by Perl/Python/sed/awk/Ruby/ETL/etc.
Especially given the "fun" of defining how to properly escape a
regex-matching string in a regex delimited file.

Cheers,
Steve