Text encoding.

[Alex Tweedly]

Sorry to drag us off the interesting topic of licensing :-) into some 
Livecode question.

I know little or nothing about Unicode, text encodings, etc. - so my 
question is indeed naive.

I have a text file (War & Peace from Project Gutenberg), about 3.4Mb. 
The Mac describes it simply as "Plain text".

When I read that into a variable, and then do
     replace tChar by SPACE in tWholeText
it takes between 1000 and 4000 millisecs - versus the 8-10 msecs I had 
expected from other samples.

If I put in
     put textEncode(tWHoleText, "UTF8") into tWholeText
before the replace then it does indeed tae 8-10 msecs.

Q1. What (if anything) am I losing by doing that ?

Q2. Is this the best alternative ?

Additional info - I just discovered that according to 'more' command 
line, the file start with :

<U+FEFF>The Project ....

if that is useful.

Many thanks,

Alex.