Performance Problem with large file

[Jim Ault]

On 2/24/07 4:18 PM, "Len Morgan" <len-morgan at crcom.net> wrote:
> Ask and ye shall receive.  Here's the "Convert" mouseUp script:

Hi, Len,
First idea is to get text from a field, then work with data in variables (or
if you need something saved with the stack, use custom properties)
Variables will evaporate when quitting, customproperties get saved with the
stack.

>  on mouseUp
     put xmlFileHeader() into vXmlOutput --***
     put field txtInput into vTxtInput

     REPEAT for each line curLine in vTxtInput
>         put the item 1 of curLine into idnum
>         put item 2 of curLine into lst_n
>         put item 3 of curLine into fst_n
>         put item 4 of curLine into day_in
>         get xmlWriteNewInmateRecord(idnum, lst_n, fst_n, day_in, tCount)
--I am assuming the previous line writes to the output field
--   you should change this to a variable (or custom property)
eg.                 put curLineToWrite & cr after vXmlOutput

>         get setPercentDone(tCount, ttlLines)
>         put "Record " & tCount & " of " & ttlLines into field "progressText"
>         put tCount + 1 into tCount

>     END REPEAT
>      
>     # Write the closing tag
--write the variable to the output field as the last step.
     put vXmlOutput & "</RCATS>" & cr INTO field "xmlOutput"
>     set the visible of field "xmlOutput" to true


If both of the handlers are in the same script container then do:
--
local vXmlOutput  -- at the top of the script container

on mouseup
  put the short date into vXmlOutput  --for other handlers to use
end mouseup
--    now this variable is the same to all the handlers in the same
container.  Each handler can read, write, append, clear the variable


If the handlers are not in the same script container, then you can use a
global.  Just remember, a global is actually a variable owned by Rev and
will retain its value until you quit Rev, clear it, or delete it.  A global
is available to all scripts in all stack at all times.

This should get you started.  Using variables for data mungering instead of
fields is a standard practice in Rev.  Fields by their nature are much, much
slower.

Jim Ault
Las Vegas


On 2/24/07 4:18 PM, "Len Morgan" <len-morgan at crcom.net> wrote:

> Paul Looney wrote:
> 
>>> If it is taking 3 seconds now, processing from a variable should take
>>> (far) less than one second. If that is so, then you will not need to
>>> worry about covering the "hole".
>>   
> If 3 seconds was the case when the field is displayed, I would not have
> even brought it up.  It's 3 seconds if the field is made invisible.
> It's over a minute (about 3 seconds PER RECORD) if the field is showing.
>>> I do second Richard's suggestion about moving from XML; it is really
>>> slow, takes more memory for code, takes more memory for data, etc. -
>>> worth asking the vendor if it is necessary.
>>   
> 
>> Some vendors won't listen. WebMerge works with a lot of merchant data
>> feeds, and some of the vendors are so fixated on the fashionability of
>> XML that even the argument about the impact in their service usage
>> doesn't sway them.
> 
> In my case, a) the vendor WON'T change and b) I only have to generate an XML
> file.  I don't have to read or process it.  To me, I just take a csv text file
> and then add all the XML bits and spit it out.  Other than the declarations at
> the top and bottom of the file, every record simply takes three items off each
> line and wraps XML tags around them before outputing them.  Really pretty
> simple.
> 
>>> Also, are you using "repeat with" where you could be using "repeat
>>> for"? The latter is about ten times faster.
>>   
> 
>> Paul raises a very important point here; "repeat for each" is a
>> performance godsend.
> 
>> One great thing about this list is that if you'll post the code you'll
>> have a small army of seasoned pros rewrite it for you for free.  :)
> 
>> Len, can you post that handler?
> 
>     
> ##############################################################################
> #######
> 
>     # Clear the output field we're going to use and turn if off (for
> performance reasons)
>     put empty into field "xmlOutput"
>     put xmlFileHeader() after field "xmlOutput"
>     set the visible of field "xmlOutput" to false
>     
>     REPEAT for each line curLine in field "txtInput"
>         put the item 1 of curLine into idnum
>         put item 2 of curLine into lst_n
>         put item 3 of curLine into fst_n
>         put item 4 of curLine into day_in
>         get xmlWriteNewInmateRecord(idnum, lst_n, fst_n, day_in, tCount)
>         get setPercentDone(tCount, ttlLines)
>         put "Record " & tCount & " of " & ttlLines into field "progressText"
>         put tCount + 1 into tCount
>     END REPEAT
>      
>     # Write the closing tag
>     put "</RCATS>" & cr after field "xmlOutput"
>     set the visible of field "xmlOutput" to true
> 
> ##############################################################################
> ###############
> 
> That's the "meat" of the handler.  Hopefully most of it will be
> self-explanitory.  The only part that might not be obvious is the
> setPercentDone function which is given a record number and total records and
> update a progress bar.  Hope that sheds some light.
> 
> Thanks again,
> 
> Len
> 
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-revolution