The VG Resource

Anyone has an automated way to eliminate duplicate pictures in a file? Im checking at some pc games that have thousands of pictures in a single file, but there are a lot of duplicates.
Edit: 15,158 pictures to be exact.

in what kind of file?

This makes a difference in how to handle it, if at all.

If it's an archive, I heard CCleaner has a duplicate-checking function, although I don't know how to use it.
You'd just extract and then run CCleaner on it, however exactly that is supposed to work, as I said, I don't understand CCleaner, as I do not use it.

If it's a single image file or something like that, hard to say.
If they're all the same size, and arranged uniformly with 0 pixel gaps, you could use the tile ripper module of Shoebox to treat the sheet as a tilemap and create a tileset and a .tmx map file to go with it (You can delete the latter, in this case) Depending on what level of precision you use, though, and even the size of the input file, this could either take a while or crash Shoebox completely.

That's all I can think of.

Good luck!

Basically as DG said, depends on the file. For an archive there should be a program that can detect duplicate files. For an image, something could be made for it but it would be very difficult/slow if they're not uniformly spaced.

Each pictures is in a singles files, 15k of them. Tried CCleaner and he doesnt see any differences. I bet they really are different but the differences isnt something i can see.
I made this example with SpriteSheeter with the fist few:
Edit: Alright, found an app thats works.

With the duplicates removed, im at 12k files. Whats the maximum size of .zip accepted on tsr?

I'm not sure (I think it was 50MB or so, unless that was removed?) but in general you probably don't want to upload massive files anyway. I'd recommend splitting them into reasonable chunks, no more than around 50MB apiece. This is just because downloading huge files can be a bit problematic, and it's usually nicer to get it in smaller chunks (especially if the person is looking for one or two specific pictures, and those are in the first or second files).

I started working on a sheet of Grand Knight History, the town of Avalon. Its pretty impressive the number of layer it has:

This looks great! Hope to see more!