Slow stack problem

[Neville Smythe]

In my last epistle I mentioned the repeat loop had only 32 iterations. Much more relevant of course is the inner loop on the number of lines of the data variable fff. In this case fff had 1760 lines. So the total possible number of iterations was around 30000 to 50000, getting up there but still well within LC capability.

I tried operating the loop on just the first 100 lines of fff. The repeats took 0.006 seconds (impressive).

Then just the first 300 lines. Timing was 0.016 seconds, approximately linear increase as expected

Then the first 500 lines. Timing is now 6.43 seconds. Something very odd there, that’s beyond exponential increase.

And on the last 500 lines, timing was 0.135 seconds (Aha !!!)

This would seem to point to matchChunk having indigestion over something in the middle of the text data. The data is 98% plain ascii, but quite likely it has a few Unicode characters: I have taken a whole lot of time to convert my legacy app to accept Asian and European Unicode personal names.

Gives me something to work on.  If I am right, it points to either a bug or a severe limitation in matchChunk if it cannot work with Unicode.

And lo…

Put textEncode(fff,”ascii”) into fff

And now the whole 1760 lines take 0.073 seconds to complete. 

I have a solution or at least a workaround, and LC has an impending bug report [does anyone know if the latest versions of  regexp searches have a performance problem with Unicode in other implementations?].

Neville Smythe