OMG text processing performance 6.7 - 9.5

Ralph DiMola rdimola at
Thu Jan 30 17:41:04 EST 2020

I found this as well. Another thing, it's faster to truncate the string and
search from the beginning than using a "start at" on the entire string when
searching for all occurrences of a string . This was counter intuitive to me
until Mark explained that skipping chars requires more work because
repetitive skipping of Unicode chars is slower than many "memcpy"s on very
long strings.

Ralph DiMola
IT Director
Evergreen Information Services
rdimola at

-----Original Message-----
From: use-livecode [mailto:use-livecode-bounces at] On Behalf
Of Neville via use-livecode
Sent: Thursday, January 30, 2020 4:49 PM
To: use-livecode at
Cc: Neville
Subject: Re: OMG text processing performance 6.7 - 9.5

Are you perchance using lineOffset searches? I have found that lineOffset
performance on utf8 text degrades exponentially with the length of the file,
presumably as it searches for line breaks. Use offset instead which remains
fast (and much faster still if you can search on the raw text before
textencoding, then utf8 encode the found chunks)
use-livecode mailing list
use-livecode at
Please visit this url to subscribe, unsubscribe and manage your subscription

More information about the use-livecode mailing list