Re: Fixed length data types issue

On Fri, Sep 08, 2006 at 04:49:22PM -0400, Andrew Dunstan wrote:
> mark@mark.mielke.cc wrote:
> >Only ASCII values store more space efficiently in UTF-8. All values
> >over 127 store more space efficiently using UTF-16. 
> This second statement is demonstrably not true. Only values above 0x07ff 
> require more than 2 bytes in UTF-8. All chars up to that point are 
> stored in UTF-8 with greater or equal efficiency than that of UTF-16.  
> See http://www.zvon.org/tmRFC/RFC2279/Output/chapter2.html

You are correct - I should have said "All values over 127 store
at least as space efficiently using UTF-16 as UTF-8."

From the ICU page: "Most of the time, the memory throughput of the
hard drive and RAM is the main performance constraint. UTF-8 is 50%
smaller than UTF-16 or US-ASCII, but UTF-8 is 50% larger than UTF-16
or East and South Asian scripts. There is no memory difference for
Latin extensions, Greek, Cyrillic, Hebrew, and Arabic.

For processing Unicode data, UTF-16 is much easier to handle. You get
a choice between either one or two units per character, not a choice
among four lengths. UTF-16 also does not have illegal 16-bit unit
values, while you might want to check or illegal bytes in UTF-8.
Incomplete character sequences in UTF-16 are less important and more
benign. If you want to quickly convert small strings between the
different UTF encodings or get a UChar32 value, you can use the macros
provided in utf.h and ..."

I didn't think of the iterators for simple uses.

Cheers,
mark

-- 
mark@mielke.cc / markm@ncf.ca / markm@nortel.com     __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   | 
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada
 One ring to rule them all, one ring to find them, one ring to bring them all                      and in the darkness
bindthem...
 
                          http://mark.mielke.cc/