Re: multiline CSV fields

On Nov 11, 2004, at 10:07 PM, Andrew Dunstan wrote:

>
>
> Patrick B Kelly wrote:
>
>>
>>
>>
>> My suggestion is to simply have CopyReadLine recognize these two 
>> states (in-field and out-of-field) and execute the current logic only 
>> while in the second state. It would not be too hard but as you 
>> mentioned it is non-trivial.
>>
>>
>>
>
> We don't know what state we expect the end of line to be in until 
> after we have actually read the line. To know how to treat the end of 
> line on your scheme we would have to parse as we go rather than after 
> reading the line as now. Changing this would be not only be 
> non-trivial but significantly invasive to the code.
>
>

Perhaps I am misunderstanding the code. As I read it the code currently 
goes through the input character by character looking for NL and EOF 
characters. It appears to be very well structured for what I am 
proposing. The section in question is a small and clearly defined loop 
which reads the input one character at a time and decides when it has 
reached the end of the line or file. Each call of CopyReadLine attempts 
to get one more line. I would propose that each time it starts out in 
the out-of-field state and the state is toggled by each un-escaped 
quote that it encounters in the stream. When in the in-field state, it 
would only look for the next un-escaped quote and while in the 
out-of-field state, it would execute the existing logic as well as 
looking for the next un-escaped quote.

I may not be explaining myself well or I may fundamentally 
misunderstand how copy works. I would be happy to code the change and 
send it to you for review, if you would be interested in looking it 
over and it is felt to be a worthwhile capability.



Patrick B. Kelly
------------------------------------------------------                              http://patrickbkelly.org