Re: Unicode normalization SQL functions

On 1/28/20 9:21 PM, Peter Eisentraut wrote:
> You're right, this didn't make any sense.  Here is a new patch set with 
> that fixed.

Thanks for this patch. This is a feature which has been on my personal 
todo list for a while and something which I have wished to have a couple 
of times.

I took a quick look at the patch and here is some feedback:

A possible concern is increased binary size from the new tables for the 
quickcheck but personally I think they are worth it.

A potential optimization would be to merge utf8_to_unicode() and 
pg_utf_mblen() into one function in unicode_normalize_func() since 
utf8_to_unicode() already knows length of the character. Probably not 
worth it though.

It feels a bit wasteful to measure output_size in 
unicode_is_normalized() since unicode_normalize() actually already knows 
the length of the buffer, it just does not return it.

A potential optimization for the normalized case would be to abort the 
quick check on the first maybe and normalize from that point on only. If 
I can find the time I might try this out and benchmark it.

Nitpick: "split/\s*;\s*/, $line" in generate-unicode_normprops_table.pl 
should be "split /\s*;\s*/, $line".

What about using else if in the code below for clarity?

+        if (check == UNICODE_NORM_QC_NO)
+            return UNICODE_NORM_QC_NO;
+        if (check == UNICODE_NORM_QC_MAYBE)
+            result = UNICODE_NORM_QC_MAYBE;

Remove extra space in the line below.

+    else if (quickcheck == UNICODE_NORM_QC_NO )

Andreas