Encountering slow navigation to a card containing very large fields? Do this one simple trick.

[David V Glasgow]

> On 17 Apr 2022, at 12:30 am, Bob Sneidar via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> Yes, I think you have it when you say that displaying text in a field is processor intensive. Fields are notoriously inefficient at storage. Do all your processing in memory and then display your results in whatever object you desire.

Yup, I do that.  processing takes time, but honestly it is amazingly fast.  Dumping found lines into the results field takes a surprising amount of time, but even that doesn’t bother me too much.  The issue is purely navigating back to the card containing the 600lb fields from any card that doesn’t.

> Datagrids can provide an efficient intermediary display mechanism because the data is already in a form that is easy to work with, namely arrays. An array is merely a handle to a position in memory. A datagrid is a window on that object. As you scroll through the datagrid, you are only manipulating the data appearing in the window, and not the data itself. 
> 
> A field is different in that how the data appears in the interface must be calculated in advance. Think of it like the old “How do you eat an Elephant” allegory. The Datagrid eats the Elephant, “One Bite At a Time.” The field tries to eat the Elephant all at once. 
> 
> Sent from my iPhone
> 
>> On Apr 16, 2022, at 01:40, David V Glasgow via use-livecode <use-livecode at lists.runrev.com> wrote:
>> 
>> Thanks for this, Bob.  I have followed various discussions about datagrids, but didn’t appreciate the property your “window on the data” metaphor describes.  I have also noticed that processing text in a variable is amazingly fast but then the results being put into a field takes a lot of time.  I assume the linking of a text variable to a datagrid would also avoid that bottleneck?
>> 
>> That is very interesting and I will explore the functions you describe.  Thanks very much.
>> 
>> Best Wishes,
>> 
>> David Glasgow
>> 
>> 
>>> On 15 Apr 2022, at 8:35 pm, Bob Sneidar via use-livecode <use-livecode at lists.runrev.com> wrote:
>>> 
>>> Not sure why you are having this issue, but I will say that this is one of the primary reasons Datagrids were created. Table fields with large amounts of data were causing issues, and especially if the limits of lines or characters were exceeded. 
>>> 
>>> With a Datagrid however, only the data that is visible is actually being displayed. For example, you can have a datagrid with 10 visible rows and containing 10,000 lines of text (or array keys if you prefer) but only 10 lines of data will actually be in fields in the datagrid. There is wizardry in play when you scroll. It *appears* like there is data above and below the visible data, but trust me there is not. You can think of a datagrid as a moving window above the data. Anything not showing throught the window is not being processed by your vision, if you take my clumsy analogy. 
>>> 
>>> Once the data is in the datagrid, there are a number of ways to work with it. I like using the dgData of the datagrid which returns a numbered array, because I have written functions like filterArray and arrayToSQLite, cursorToArray, arrayToText, mergeArray etc. which allow me to work efficiently with these numbered arrays. 
>>> 
>>> Of particular interest to you might be the arrayToSQLite, bcause once you have data in an sqLite table, querying is a much more powerful (and efficient) way of getting just the data you want, such as the information displayed in your summary page. 
>>> 
>>> To use datagrids, you only need to know about a few properties, particularly the dgData, the dgText, the dgHilitedLine / Index, the dgDataOfLine / Index and handling the selectionChanged message. 
>>> 
>>> Bob S
>>> 
>>> 
>>>>> On Apr 15, 2022, at 10:27 , David Glasgow via use-livecode <use-livecode at lists.runrev.com> wrote:
>>>> 
>>>> Hi folks,
>>>> 
>>>> 
>>>> MacOS 10.13.6 and Livecode 9.6.0
>>>> 
>>>> It may be that this is well known to everyone but me.  But just in case...
>>>> 
>>>> I have a card (A) with two fields in separate groups.  Field 1 contains many many thousands of lines of text which are filtered for keywords and the results displayed in field 2 (which can also be many thousands of lines).  The user can navigate to a separate card (B) to view summary statistics and a bar chart.
>>>> 
>>>> I found that worked fine, but navigating back from B to A always  took many seconds and looked like a hang.  The delay seemed to be in some way proportional to the number of lines in field 1 and field 2.
>>>> 
>>>> By a slip of the finger I discovered that this lag disappears completely if the groups containing field 1 and 2 are also placed on the summary stats card B.  There is no real need for either to be there otherwise , and it seems they can even be invisible.  I tried to see if there is a significant memory overhead to doing this, but there doesn’t seem to be.
>>>> 
>>>> Is this a known thing?  And either way, is there a better approach, or a downside I haven’t considered?
>>>> 
>>>> Best Wishes,
>>>> 
>>>> David Glasgow
>>>> 
>>>> _______________________________________________
>>>> use-livecode mailing list
>>>> use-livecode at lists.runrev.com
>>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>> 
>>> _______________________________________________
>>> use-livecode mailing list
>>> use-livecode at lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode