Sort IP List

Bob Sneidar bobsneidar at iotecdigital.com
Fri Jun 29 12:45:54 EDT 2018


A realistic expectation of IP addresses for a given network might only be around 16,535 (class B network) assuming every address was in use, an unlikely scenario. I thought of way to do this for an extremely large number of addresses by reading a large file in 1000 line chunks into 4 columns of a SQL memory database, then querying using concatenation and sorts on the 4 columns, and using limit 1,1000 1001,1000 2001,1000 etc. and writing back to another file. The time to do this of course would be much longer, but it would avoid any memory constraints. 

Bob S


> On Jun 29, 2018, at 09:35 , Mike Bonner via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> ## was writing this when your response showed up..
> Just did some tests.  For reasonable size lists of IP numbers, your method
> is clearly faster. As lines increase the disparity shrinks (due to 4
> processings of the same lines rather than 1) until around 35000 lines at
> which point positions reverse and the disparity grows in favor of the
> function(each) method.
> 
> After just a bit more testing, the fastest method (for anything over 20000
> lines) is to run through the whole list first converting it to numeric and
> then do a single simple sort, but that leaves you with a straight list of
> numbers that would then have to be re-divided into triads, so the post
> processing needed kills the whole idea.
> 
> As for your response,
> Yes  the function is called by sort for each line and returns the numbers
> to use to give that line its sort value.  It isn't as fast as I had hoped
> most likely because it has to call a function for each line in the list.
> With the first method I posted, on my machine, the crossover point is right
> around 35000 lines (at least on my system, at 35000 lines it takes 475
> millisec to 612 millisec vs the 4xsort)   At 200,000 lines the disparity
> grows to over 3 seconds difference, but i'm unsure what max length list
> might be reasonably expected.  The 4 sort method is by far the simplest to
> code and is plenty fast for list under 100000.
> The nicest part of your method is that if processing a huge list, its easy
> to give visual feedback between each sort if need be.  But again, all this
> is likely moot unless the ip list is huge.





More information about the use-livecode mailing list