Array Split vs Combine

[Sean Cole (Pi)]

The code for 'Split':

void MCArraysExecSplit(MCExecContext& ctxt, MCStringRef p_string,
MCStringRef p_element_delimiter, MCStringRef p_key_delimiter, MCArrayRef&
r_array)
{
if (MCStringSplit(p_string, p_element_delimiter, p_key_delimiter, ctxt .
GetStringComparisonType(), r_array))
return;

ctxt . Throw();
}


vs

The code for 'Combine' (// comments added by me):

void MCArraysExecCombine(MCExecContext& ctxt, MCArrayRef p_array,
MCStringRef p_element_delimiter, MCStringRef p_key_delimiter, MCStringRef&
r_string)
{
bool t_success;
t_success = true;  // Create a register to check progress success

uindex_t t_count;  // Create a new (t)emp counter for indices
t_count = MCArrayGetCount(p_array);  // Find out how many arrays there are

MCAutoStringRef t_string;  // Create (t)emp string to store the result
if (t_success)

t_success = MCStringCreateMutable(0, &t_string); // t_success is always
true here initially and is reset as true if t_string is now mutable, false
if not


combine_array_t t_lisctxt;  // create a new array object
t_lisctxt . elements = nil; // initialise t_lisctxt array
if (t_success)

t_success = MCMemoryNewArray(t_count, t_lisctxt . elements); // make sure
the array was created


if (t_success)
{

t_lisctxt . index = 0;

MCArrayApply(p_array, list_array_elements, &t_lisctxt);

qsort(t_lisctxt . elements, t_count, sizeof(array_element_t),
compare_array_element); // sort the elements

for(uindex_t i = 0; i < t_count; i++)

{ // Loop through all indices

MCAutoStringRef t_value_as_string; // create a (t)emp string for element
value



t_success = ctxt . ConvertToString(t_lisctxt . elements[i] . value,
&t_value_as_string); // convert array value to string

if (!t_success)

break; skip if unable to convert to string


t_success =

(p_key_delimiter == nil ||

(MCStringAppend(*t_string, MCNameGetString(t_lisctxt . elements[i] . key))
&&

MCStringAppend(*t_string, p_key_delimiter)))&&

MCStringAppend(*t_string, *t_value_as_string) &&

(i == t_count - 1 ||

MCStringAppend(*t_string, p_element_delimiter)); // t_success is true if
the array element and values are added correctly


if (!t_success)

break; // skip if unable to add value

}

}

if (t_success)

t_success = MCStringCopy(*t_string, r_string);  // Copies the (t)emp string
into the (r)eturn string


MCMemoryDeleteArray(t_lisctxt . elements);

if (t_success)

return;


// Throw the current error code (since last library call returned false).
ctxt . Throw();
}


Following on from Bob's VM comment, there is reference to
'MCMemoryNewArray(t_count,
t_lisctxt . elements)' which does highlight that some memory management for
the arrays is necessary in the combine command. This only creates a
temporary copy of the array for working through. How this plays out
differently for Windows vs Mac/Linux and why this would be increasing the
time necessary by a factor of about 4:1 I can't see.

I've tested as far back as LC7
(Times - Read into memory, Split to array, Combine from array)
LC9.5.0 Win64 - 0.437s, 0.516s, 3m 1.378s
LC9.0.5 Win32 - 0.446s, 0.547s, 3m 27.9s
LC8.2.0 DP2 - 0.543s, 0.577s, 3m 30.208s
LC8.0.0 - 0.542s, 0.545s, 3m 30.815s
LC7.0.0 - 0.827s, 0.460s , 3m 37.896s

On mac all times are less than 1sec, 3 sec total.

Sean


On Wed, 10 Mar 2021 at 17:08, Bob Sneidar via use-livecode <
use-livecode at lists.runrev.com> wrote:

> Now THAT is fascinating, considering the Windows performance issues with
> file access reported in the past. Could it be that combine is somehow
> caching data to virtual memory?
>
> Bob S
>
>
> On Mar 9, 2021, at 1:05 PM, Sean Cole (Pi) via use-livecode <
> use-livecode at lists.runrev.com<mailto:use-livecode at lists.runrev.com>>
> wrote:
>
> It's looking to be a Windows only issue. I need to see how far this goes
> back and then I'll post a bug report. It's making a process that should
> only take 30s max on a single thread 2GHz remote Win server take 16mins to
> process 2 of these files, so it will be good to find a solution for this.
>
> Thanks everyone for confirming and providing your input.
>
> Regards
> Sean
>
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>