Working with csv files that are 5000 lines or more

[Jim Schaubeck]

Mark,
Thank you for the quick response.  I have not tried your idea but i will work on it now.  Your approach seems to be very close to what I was doing (as you suspected, I used the repeat with x = 2 to the number of lines etc

If your approach is many times faster, that could be the trick

Thank you
Jim...

 
On Wednesday, April 09, 2008, at 05:52PM, "Mark Smith" <mark at maseurope.net> wrote:
>Jim, are you using a 'repeat for' loop? In this kind of situation,  
>they can be many, many times faster than 'repeat with n = 1 to the  
>number of lines'.
>
>Another option might be to split the data into an array:
>
>put url ("file:" & someCSVfile) into tData
>split tData by return  -- now you have an array, one line per  
>element, the elements numbered 1 to 7000
>
>repeat for each line L in the keys of tData
>   if whatYourSearchingFor is in tData[L] then
>     either do what you need to with tData[L] or
>     put L & comma after listOfLinesThatMatchTheSearchCriteria
>   end if
>end repeat
>
>Best,
>
>Mark
>
>On 10 Apr 2008, at 01:35, Jim Schaubeck wrote:
>
>> My next project is reading in a csv file as large as 7000 lines  
>> with 60 items per line.  The "read file..." and "put it into  
>> tempvar" command work very quickly.  When I search the data in  
>> tempvar, the repeat command works quickly for the first 1000 lines  
>> or so then things slow down dramatically.  I tested it with a  
>> scrollbar being updated for every line and 1000 seems to be the  
>> break point.  Using "Find..." in excel works very quickly but the  
>> same file in my stack is slow.  I'll be loading my app onto 20 or  
>> so other users so a database may not be an option unless I can load  
>> it into the stack (never tried to include an actual database in my  
>> apps).
>>
>> Any ideas on how to search through csv files that are 2000 to 7000  
>> lines?
>>
>> Thanks in advance
>> Jim...
>> _______________________________________________
>> use-revolution mailing list
>> use-revolution at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your  
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-revolution
>
>_______________________________________________
>use-revolution mailing list
>use-revolution at lists.runrev.com
>Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>http://lists.runrev.com/mailman/listinfo/use-revolution
>
>