curlyquotes, character sets, livecode, and english

[Richmond]

On 05/26/2013 08:30 PM, Dr. Hawkins wrote:
> I can't believe that this one flummoxed me as long as it did.
>
> SQLite, like the honeybadger, just don't care . . .
>
> I got hung up on a curlyquote.
>
> I'll definitely have people pasting in from whatever sources, possibly the
> wrong one for the platform (ever read from mac or unix a web page written
> by someone who thinks that MS word is a standard?).
>
> There is no possibility of my application ever being used in an application
> other than English.

Really?

>
> For that matter, there is no possibility of it being used in a non-US
> country.

Missouri German.

Texas German.

Amish Swiss German (steam-driven computer perhaps).

Mexican Spanish.

Louisiana French.

Haight-Ashbury LSD-driven gobbledegook (pace Tom Sharp's "All Electric 
Kool-Aid Acid Test - wow, that book really rocked my socks).

Gullah.

Yiddish.

US English with 'funny names' such as Nöel (and, just in case !!!! that's
a capital 'n' followed by an 'o' with 2 dots over the top).

And as someone who speaks 'Scots' as well as various types of 'English'

[to the English 'Scots' is a very odd dialect of English; to the Scots 
Scots is either 'Hame Inglis', 'Lallans' or 'Scots']

I would like to remind you that what is one (wo)man's English is another 
(wo)man's foreign tongue.

An wi that scrieve, aiblins thou wilt ken quhat Ahm speirin eftir!

Richmond.

>
> postgres chokes on character 213, as an illegal utf8.
>
> The simple solution seems to be to filter any input based on the host OS,
> turning it into utf8.
>
> But what about the mac user who pastes from an ms data, or the ms who
> pastes from a webpage.
>
> Is there a "sane" way to turn english text into a single usable format?
>
> I'd be tempted to go pure 7 bit ascii, but there's enough names with extra
> characters it doesn't support to make this a non-starter.
>