FIlter unicode variable with variable

[Dar Scott]

On May 7, 2005, at 8:50 AM, Thomas McGrath III wrote:

> I guess the question/problem is

> 1) does the message box truly show unicode in a variable?

No.  It effectively does 'put into', not 'set the uncodeText of'.

> 2) does the filter command work with unicode?

The short answer is No.

> 3)if it does then do I need both the variable and the expression to be 
> in unicode?

You will always need both strings in your operations to be unicode.

This might work 90% of the time:  Make sure there is a unicode line end 
after both strings.  Then use replaceText to replace with empty.

Remember.  You have access to host UTF16 of field text and you can set 
it.  You can make conversions.  You can create unicode characters 
(UTF16 encodings) from codes and the inverse.  However, beyond that you 
have to consider your unicode data as simply a string of bytes, each 
pair representing a Unicode character.  (Unless you have surrogates, 
but that is unlikely.)  You are essentially processing raw binary data.

That means...

1.
You can't use = to test for equality of unicode strings.
That might work in some quick tests, but will fail in general.
Use length equality and contains.

2.
You usually can't use any of the text processing commands in Rev.
Build a loop for processing each character in a unicode string.
Use that template lots of places.
Use that to build your template for processing each line.

3.
Exploit setUseUnicode.
Use this with numToChar() to create special characters such as lf.
Use this with charToNum() to dump the encodings for debugging.
This takes into account the order of the two bytes for each computer.

4.
Conversion to anything except UTF8 can lose information.

5.
You don't have unicode characters or strings in Transcript,
only binary data that represents those.

Dar
-- 
**********************************************
     DSC (Dar Scott Consulting & Dar's Lab)
     http://www.swcp.com/dsc/
     A Sponsor of RevCon West
**********************************************