Re: program to remove duplicates

fir wrote:

fir wrote:

Bart wrote:

On 22/09/2024 11:24, fir wrote:

Paul wrote:

>

The normal way to do this, is do a hash check on the
files and compare the hash. You can use MD5SUM, SHA1SUM, SHA256SUM,
as a means to compare two files. If you want to be picky about
it, stick with SHA256SUM.

>
>

the code i posted work ok, and if someone has windows and mingw/tdm
may compiel it and check the application if wants
>
hashing is not necessary imo though probably could speed things up -
im not strongly convinced that the probablility of misteke in this
hashing is strictly zero (as i dont ever used this and would need to
produce my own hashing probably).. probably its mathematically proven
ists almost zero but as for now at least it is more interesting for me
if the cde i posted is ok

>
I was going to post similar ideas (doing a linear pass working out
checksums for each file, sorting the list by checksum and size, then
candidates for a byte-by-byte comparison, if you want to do that, will
be grouped together).
>
But if you're going to reject everyone's suggestions in favour of your
own already working solution, then I wonder why you bothered posting.
>
(I didn't post after all because I knew it would be futile.)
>
>

i wanta discus nt to do enything that is mentioned .. it is hard to
understand? so i may read on options but literally got no time to
implement even good idead - thsi program i wrote showed to work and im
now using it

>
also note i posted whole working program and some other just say what
can be done... in working code was my main goal not quite starting in
contest of what is fastest (this is also interesting topic but not the
main goal)

interesting thing is yet how it work in system ...
im used to write cpu intensive application and used to controll frame times and usage of cpu.. bet generally never t vrite disk based apps
here above is the first..i use sysinternals on windows ind when i run this prog the it has like 3 stages
1) read directory info (if ts big like 30k files it mat take soem time)
2) square part that read file contents and compares and sets flags of duplicates on list
3) the rename part - i mean i cal "reneme" function on duplicates
the most tiem it takes the square part and indicator of disk usage is full , cpu usage is 50% it means probably one core usage is full
the disk indicator in tray shows (in square phase) something like
R: 1.6 GB
O: 635 KB
W: 198 B
dont know what it is, R is for read for sure and W is for write but what it is exactly?
there is also a question if closing or killing program in those phases may generate some disc dameges? - as ror most time in square phase it takes reads i quite sure that closing in read phase may not incur anny errors - but im not sure as to renaming phase