[Gambas-user] Get distinct array from large array

[Brian G]

----- On Dec 10, 2021, at 8:35 AM, Safiur Rahman <isafiur at gmail.com> wrote: 

> Hi Tobias

> Your solution is at least twice faster than uniq command and much faster than
> checking each item in loop in my case. Thank you.

> ----[ http://gambaswiki.org/wiki/doc/netiquette ]----
I would like to add just one or two line to Tobias's solution, about 4 percent faster. As it does not allocate the value storage, or change it. 
This becomes larger the slower the cpu being used. 

Strange this is that making the whole thing fast unsafe actually makes it slower... ummmm. 

Start with :1080000 strings 
original solution 10000 time=24.1813569869992 
Tobias's 10000 time=0.264946608003811 
with added line 10000 time=0.253471721996903 

Public Function Deduplicate(xxx As String[]) As String[] 
Dim stash As New Collection 'Collection(gb.ignorecase) use this if case does not matter 
stash.default = False ' sets the default value in the collection to false 
For Each x As String In xxx 
If stash[x] Then Continue ' adding this line still does the lookup, but not the assignment about 4% faster for 108000000 records 
stash[x] = True 
Next 
#If gambas >= "3.16.90" 
Return stash.Keys ' this turns out to be the same time as the below loop 
#Else 
Dim resultset As New String[] ' use this if not supported keys 
For Each x As String In stash 
resultset.Add(stash.key) 
Next 
Return resultset 
#Endif 
End 

"Failure is the key to success; 
each mistake teaches us something" .. Morihei Ueshiba 
Brian G 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gambas-basic.org/pipermail/user/attachments/20211211/007acbb1/attachment-0001.htm>