Re: program to remove duplicates

Chris M. Thomasson wrote:

On 9/21/2024 11:53 AM, fir wrote:

>
>
i think if to write a simple comandline program
that remove duplicates in a given folder

[...]
>
Not sure if this will help you or not... ;^o
>
Fwiw, I have to sort and remove duplicates in this experimental locking
system that I called the multex. Here is the C++ code I used to do it. I
sort and then remove any duplicates, so say a threads local lock set was:
>
31, 59, 69, 31, 4, 1, 1, 5
>
would become:
>
1, 4, 5, 31, 59, 69
>
this ensures no deadlocks. As for the algorithm for removing duplicates,
well, there are more than one. Actually, I don't know what one my C++
impl is using right now.
>
https://groups.google.com/g/comp.lang.c++/c/sV4WC_cBb9Q/m/Ti8LFyH4CgAJ
>
// Deadlock free baby!
void ensure_locking_order()
{
   // sort and remove duplicates
>
   std::sort(m_lock_idxs.begin(), m_lock_idxs.end());
>
   m_lock_idxs.erase(std::unique(m_lock_idxs.begin(),
     m_lock_idxs.end()), m_lock_idxs.end());
}
>
Using the std C++ template lib.

im not sure what you talking about but i write on finding file duplicates (by binary contents not by name).. it is disk thing and i dont think mutexes are needed - you just need to read all files in folder and compare it byte by byte to other files in folder of the same size