Обсуждение: Trouble w/ COPY command

Поиск
Список
Период
Сортировка

Trouble w/ COPY command

От
Wells Oliver
Дата:
I have CSV files blowing up because they have double quotes around numeric values: " 1.34" yields ERROR:  invalid input syntax for type numeric: " 1.34"

And occasionally these are empty, and you see invalid syntax for type numeric: " "

Are there flags I can pass to COPY to avoid this? There are properly quoted string values elsewhere in the final, so I'm reticent to strip out double quote characters or anything.

Thank you.

--
Wells Oliver
wellsoliver@gmail.com

Re: Trouble w/ COPY command

От
Greg Williamson
Дата:
Wells Oliver asked:


>
>I have CSV files blowing up because they have double quotes around numeric values: " 1.34" yields ERROR:  invalid
inputsyntax for type numeric: " 1.34" 
>>
>And occasionally these are empty, and you see invalid syntax for type numeric: " "
>
>Are there flags I can pass to COPY to avoid this? There are properly quoted string values elsewhere in the final, so
I'mreticent to strip out double quote characters or anything. 
>


There's no easy command line method that I know of. Perhaps if you can detect patterns some sed / awk might help.

I'd probably load the CSV file into a temporary work table with the columns that are giving you issues defined as type
"text."Then use SQL to remove the offending quotation marks from the afflicted columns. Create new columns of type
numericin the table, and copy the values from the text variant to the numeric ones with the proper cast one at a time;
ifit blows up you go back to resolving issues. Then copy the table using the numeric columns to the desired
destination.

Hopefully others might have better ideas ... best o' luck!


Greg Williamson


Re: Trouble w/ COPY command

От
Tom Lane
Дата:
Wells Oliver <wellsoliver@gmail.com> writes:
> I have CSV files blowing up because they have double quotes around numeric
> values: " 1.34" yields ERROR:  invalid input syntax for type numeric: "
> 1.34"

Is that an exact copy of the error message?  Because there are double
quotes in that error message's text; what you are showing does not
suggest that they are part of the data value.  I'm wondering about
non-visible characters myself.

> And occasionally these are empty, and you see invalid syntax for type
> numeric: " "

Well, that you're going to have to do something else about.  Possibly
you could treat the empty string as null?  (See the 'null string'
option to COPY.)

            regards, tom lane