Character encoding is driving me insane

[Fredrik Andersson]

Hi there,

I'm writing a small application that's supposed to keep track of  
different tasks for certain people at work. I'm using a datagrid to  
present the data and simple text files for storage. The problem is  
that I tend to lose the characters åäö and ÅÄÖ (which we use a lot  
here in Sweden).

At first I simply loaded a tab-delimited file into the dgText of the  
datagrid but that didn't work.

After searching around on the forums/mailing list I found some code  
for handling UTF-encoded text and thus I changed the "Open file"-code  
to something like this:

set the unicodetext of fld "importedData" of card "Data" to   
uniencode(tFile,"UTF8")
     put the fld "importedData" of card "Data" into tData

And then I set the dgText to tData. This worked fine until I started  
messing around with making some columns into presenting calculated  
values only. Since I didn't need to save the data in those columns to  
any file I started using the dgData-property to only save & load  
specific fields instead.

The problem is that now I lose my special characters. They simply do  
not appear in the text. They're still there in the text file, just not  
in the "importedData"-field and thus neither in the datagrid. I have  
no idea why my first solution suddenly stopped working. The text file  
is in the "UTF-8"-format.

Does anyone out there have any good ideas of how to work with text  
with special characters? Or perhaps you can point me toward a previous  
thread with useful info. Right now I know nothing about character  
encoding except that they make my life more difficult.. sigh.

Thanks,

Fredrik Andersson