Speed

[Alex Tweedly]

On 22/08/2014 21:17, Beat Cornaz wrote:
> At the moment I use :
>
> --
> function BC2_RemoveDuplicate_Lines pData
>    
>     repeat for each line rLine in pData
>        add 1 to TestArray[rLine]
>     end repeat
>     
>     return the keys of TestArray
> end BC2_RemoveDuplicate_Lines
> --	
>
>
> I hope someone knows a faster way :-)
>
>
I'll read the rest of the email carefully and (hopefully) have more 
comments later.
But for now, I'll try this one part.

I don't know a way to do this much faster. You'd get a small (perhaps 
very small) improvement by replacing the central line by

       put 1 into TestArray[rLine]


but (without testing it), I suspect it will be a small change.

However, if you can change your input lines slightly, you can do 
something that should be better.  If these are permutations that you are 
generating, then as you generate each one, you might be able to add an 
extra character onto each line (let's say, a colon ":"). Then your data 
would look like
123:
125:
152:
231:
etc.

With this input, you could do
    split pData by CR and ":"
    combine pData by CR and ":"
and all your duplicates have disappeared.

It might be even faster to do
   split pData by CR and ":"
   put the keys of pData into pData
and that would remove those extra ":"s for you at the same time.

-- Alex.