Re: Different length lines in COPY CSV

Tom Lane wrote:

>>Where would you want this file conversion 
>>utility? bin? contrib? pgfoundry?
>>    
>>
>
>I'd say pgfoundry for starters --- there's no reason to tie it down
>to server release cycles.  Maybe when the thing is fairly mature and
>doesn't need frequent releases, we could think about whether it ought
>to be brought into the core distro.
>
>However, it likely won't ever be a candidate to become part of core
>unless it's written in C, and offhand I would judge C to not be the
>best choice of implementation language for such a thing.  This is surely
>going to be mostly a string-pushing type of problem, so something like
>perl might be a better bet.
>
>    
>  
>

You are probably right. The biggest wrinkle will be dealing with various 
encodings, I suspect. That at least is one thing that doing CSV within 
the backend bought us fairly painlessly. Perl's Text::CSV_XS module for 
example simply handles this by declaring that only [\x09\x20-\x7f] are 
valid in its non-binary mode, and in either mode appears to be MBCS 
unaware. We should try to do better than that.

cheers

andrew