OMG text processing performance 6.7 - 9.5

[Ralph DiMola]

I found this as well. Another thing, it's faster to truncate the string and
search from the beginning than using a "start at" on the entire string when
searching for all occurrences of a string . This was counter intuitive to me
until Mark explained that skipping chars requires more work because
repetitive skipping of Unicode chars is slower than many "memcpy"s on very
long strings.

Ralph DiMola
IT Director
Evergreen Information Services
rdimola at evergreeninfo.net


-----Original Message-----
From: use-livecode [mailto:use-livecode-bounces at lists.runrev.com] On Behalf
Of Neville via use-livecode
Sent: Thursday, January 30, 2020 4:49 PM
To: use-livecode at lists.runrev.com
Cc: Neville
Subject: Re: OMG text processing performance 6.7 - 9.5

Are you perchance using lineOffset searches? I have found that lineOffset
performance on utf8 text degrades exponentially with the length of the file,
presumably as it searches for line breaks. Use offset instead which remains
fast (and much faster still if you can search on the raw text before
textencoding, then utf8 encode the found chunks)
_______________________________________________
use-livecode mailing list
use-livecode at lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode