Re: program to remove duplicates

On 9/21/2024 3:18 PM, fir wrote:

Chris M. Thomasson wrote:

On 9/21/2024 11:53 AM, fir wrote:

>
>
i think if to write a simple comandline program
that remove duplicates in a given folder

[...]
>
Not sure if this will help you or not... ;^o
>
Fwiw, I have to sort and remove duplicates in this experimental locking
system that I called the multex. Here is the C++ code I used to do it. I
sort and then remove any duplicates, so say a threads local lock set was:
>
31, 59, 69, 31, 4, 1, 1, 5
>
would become:
>
1, 4, 5, 31, 59, 69
>
this ensures no deadlocks. As for the algorithm for removing duplicates,
well, there are more than one. Actually, I don't know what one my C++
impl is using right now.
>
https://groups.google.com/g/comp.lang.c++/c/sV4WC_cBb9Q/m/Ti8LFyH4CgAJ
>
// Deadlock free baby!
void ensure_locking_order()
{
   // sort and remove duplicates
>
   std::sort(m_lock_idxs.begin(), m_lock_idxs.end());
>
   m_lock_idxs.erase(std::unique(m_lock_idxs.begin(),
     m_lock_idxs.end()), m_lock_idxs.end());
}
>
Using the std C++ template lib.

 im not sure what you talking about but i write on finding file duplicates (by binary contents not by name).. it is disk thing and i dont think mutexes are needed - you just need to read all files in folder and compare it byte by byte to other files in folder of the same size

It's just that there are many different ways to sort and remove duplicates. That sometimes, it is required...