Castlevania: Harmony of Despair data file info

[John4300]

EDIT: Post now includes raw sprite data on the bottom, including DLCs.
EDIT2: Added Puggsoy's sprite ripper to bottom.
EDIT3: Updated download link for the program - PM me if the link isn't working

I have been checking around the datafiles in Castlevania HD, and I was able to figure out the way they work, using TiledGGD.
Yes, I know most of the sprites are 1 to 1 copies from DS games or other source material, but the files are pretty nicely organized, and once I figured out the palette files, they should have correct colors aswell, which is hell to figure out on DS files.

The game sprite data, which aren't surfaces, consist of three files.

.col files are the palettes. These files are too dark, and every value needs to be multipled by 8 in order for them to work.
.csr files are the main sprite files
.opd files are most probably files that align and set what needs to be read. Doesn't seem to contain sprites/palettes.

Anyhow, to get stuff working. Load any csr file on tiledGGD, and set stuff like this:

Panel size needs to be 128 x Anything
Format needs to be 4 bits/pixel
Endianness needs to be little endian

Then load up the palette from a .col file in same folder, it needs to be:

4 bytes/colour
Big endian
RGB
The palettes start at offset 0x20 , containing 16 colors each.

In the end, you will get out an image looking something like this:


The palettes, for some RANDOM reason are too dark. I made a little program, which starts reading the palettes at offset 0x20, and multiplies every color by 8. The result looks like this:



I would say this is close to the DS palette. The colors are off ~8 values, since the values needed to be multiplied by 8. Especially noticable on Jonathan, his sprite's outline looked dark gray on DS, black on HD.


And yes, rest of the palettes work as well. Here are few examples of the other, in-game palettes for Julius.


I might do some rips from the game soon, as using these techniques, ripping is pretty easy. Everything that is made in the CSR files should be readable, as the .col files containing the palettes are in the same folder, only too dark.

As xb360 doesn't have an emulator, I can't confirm these too well, but the player palettes seemed to be correct looking on real console. I wonder if this is close enough to make a spritesheet for the site?


Here is the raw sprite data to be used from the game. I don't want to reserve making the sprites or sprite sheets any way myself, so if anyone wants to make the sprite sheets to the site, feel free to do so.

There are two variations: The first one is 11mb, only with sprites. The second one also includes stage and menu surfaces, taking 125mb.
PASSWORD : thesprite

Sprites only: https://mega.co.nz/#!AEtGkAqD!Pkqwwf4EE3...4lVSAjWjKw
Sprites + Surfaces: https://mega.co.nz/#!kQ1n3Ioa!1t5Rju498o...m3y7wlayrc

To open the files, Puggsoy was really nice to make a program to use for ripping the sprites of the game! Give it a go!
https://dl.dropbox.com/s/wijs0kt0652bi8c/cashdconv.zip

Code:

Usage: cashdconv palFile imgFile [outDir]
  palFile: The .col file containing palettes you want to use.
  imgFile: The .csr file you want to convert. Can alternatively be a folder multiple .csr files
  outDir: The folder to save the converted files to. If ommitted will create a subfolder called 'converted'

I hope this info would help people rip sprites and make some new, cool castlevania fangames. Have fun! =)

[puggsoy]

I know you don't need help, but would you be able to upload a couple of these files? I just want to take a look at them out of curiosity more than anything, especially those strange palette files.

[John4300]

Which of the files would you like to take a look at, in specific? Ripped files, or the source files? And does the character/boss/enemy matter? I can also include "expanded" palette files if needed. I'm looking into making a little program with C that would be a bit more usable than what I did with Game Maker quickly. I got all the files from 360 version, all DLC included.

And I might need some help with the ripping, there is quite a lot of files to look at, and I'm not too good at arranging good sheets that would be posted online.

[puggsoy]

Just the source files, doesn't matter whether it's enemy or boss or whatever, but a bit of variety would be nice. I don't need the expanded palette, just the original files. Relevant .opd files would also be nice.

I can maybe help with ripping, but I've never played the game myself so I'm probably not the best person. I'm just interested in the format. I can give suggestions or feedback on sheets though.

[John4300]

Alright, I sent you a PM about them.

The opd files seem to have all the sprite and palette files in plain text for loading, also some text like "houki00", which are most probably variables where the sprites are arranged to for easier use. So if one were to find out how these files are used for loading, one might get easy access to all the sprites.

[puggsoy]

I took a look at the .opd files but it's not at all clear how to read them. They probably do have some sort of frame or piece grabbing info but these are sheetable without that, it'll just take some manual work.

Just a note about the .csr files, they have the image dimensions at 0x20 as two 2-byte integers, most of the time it's 0x80 times 0x80 (so 128x128). The image data also starts at 0x230.

The palette files are indeed weird, I thought there might be a more reasonable method for getting the right colours but as you said, it just seems to be that you have to multiply it by 8 (or, shift 3 bits left). The only reason I can think of for storing it like that is possibly so that it's easier for when sprites need to look darker (like in a shadow or dimly lit room, I dunno). Shifting bits left might be easier/better than shifting them to the left for the game's engine, so they store the palettes as dark and just light them up when needed. That's just an idea though, as I said I haven't played the game so I don't know if they ever need to be dark.

[Dazz]

I think even just a simple conversion script instead of manually ripping from GGD would probably make this a lot easier to handle too - would that be possible?

I recall looking into this game before, the maps were stored in a really simple manner but without animations, so I wasn't sure whether to rip them or not.

[puggsoy]

It's doable, and I was actually considering making something like that. The only issue is that for every .csr you'd have to convert it multiple times for every single palette of its corresponding .col. There are some empty palettes that I've seen, those would be skippable, but apart from that there's no way to detect which .csr works with which palettes, so it'd just make one image for each. That said, doing that and then deleting the ones you don't want is still more efficient than checking each one manually in TiledGGD.

If that's OK, then I can probably whip something up tomorrow. Unless John would prefer to, I don't mind either way.

[John4300]

I don't really mind, it's not like I want to "reserve" ripping these sprites under my name, I just wished to shed some light on the data format since there isn't really any good rips of the game.
It probably isn't too hard of a job to make a little program which loads the sprites and writes them to files. With my current knowledge of programming that is rather hard for me sadly, so if anyone else wants to do it they are welcome to.

I can send rest of the data files as PM if needed, I don't think I should post them here as they are. I'm not sure if that is against the forum rules.

Also, about the darker areas in the game, I don't remember there being anything like shadows or light sources. Some stages might be darker than others (Chapter 8, underground area from Symphony of the Night comes to mind), but it's a bit hard to confirm, as at least I can't get clear enough screenshots from Xbox 360 to compare.

[puggsoy]

I've been working on a program. It's almost 100% working, there are just some annoying kinks that I have to work out. I should get it sorted out tomorrow.

And yeah, it's totally fine to post the files here. As long as you don't link to a ROM/ISO or pirated software, you should be fine. You can check out the rules using the link near the top-right area of the page.

[puggsoy]

Okeydoke, here you go. It's command-line but fairly easy to use:

Code:

Usage: cashdconv palFile imgFile [outDir]
   palFile: The .col file containing palettes you want to use.
   imgFile: The .csr file you want to convert. Can alternatively be a folder multiple .csr files
   outDir: The folder to save the converted files to. If ommitted will create a subfolder called 'converted'

You give it a single .col file, a .csr file or a folder containing the .csrs you want to convert, and an output folder (or not). The output folder doesn't have to exist.

However, as I mentioned before, there's no way to know which of the palettes in a .col work with the .csr, so it has to go through and make an image for every single palette. I did manage to make it skip empty palettes, so with some .col files there are only 4 palettes (like c_axe_pl.col), which isn't a big deal. However, files like c_tpm.col have 253 non-empty palettes. That's the maximum amount (256) minus 3. This palette file corresponds to all the .csrs in the tpm folder except for f_wp75.csr. That's 27 files. So each of these is converted using 253 different palettes. 27 * 253 = 6,831 files total, which takes a while to convert. Not hours, but those 27 files took a solid 10 minutes for me. My computer isn't that great but regardless, you'll probably want to do something else while you wait.

Anyway, within the output folder will be subfolders for each .csr. Each of these will have within it .pngs of the .csr with every palette. They'll have the name of the .csr + "_pal" + the palette number, e.g. f_tpm00_pal0.png, f_tpm00_pal1.png, and so on. You can probably see pretty easily any nonsense palettes and remove those to leave you with the ones you want.

By the way, if I happened to have somehow screwed some of the colours up just tell me. I'm pretty sure I got it right but I don't know exactly what the colours are supposed to look like so I might have done something a bit wrong. And of course tell me if anything else goes wrong.

[John4300]

Great, that should make ripping the sprites easier. One way is to make it to accept palette ranges, but for example Alucard's sprite is one crazy playground probably used as a test, having well over 20 working palettes. Some of the unused palettes seem just to be gradients.

Thanks for the help making the program!

Edit: Also with a quick test, the colors seem to be correct and the program working.

Edit 2: ACTUALLY, there seems to be some problems. When calling stuff from "common" folder containing effects, texts and stuff, it gives an error like this, even when trying to load a single file:

Converting f_com00.csr
Called from ? line 1
Called from Main.hx line 31
Called from console/Begin.hx line 31
Called from Main.hx line 70
Called from Main.hx line 119
Called from Main.hx line 220
Called from C:\HaxeToolkit\haxe\std/haxe/io/Output.hx line 256
Uncaught exception - Overflow

Here is the folder for tests: https://mega.co.nz/#!pAknkYSI!dt3r5vOhoN...Hdt726kaoE

I should probably upload the full sprite data, I'm not sure are these any different from the others. I did try checking these files with TiledGGD, and they seemed to work fine.

[puggsoy]

It appears the error was due to it trying to write a 28-bit integer using a function for writing 24-bit integers. This is because the colours in c_com00.col need to be shifted only 2 bits (multiplied by 4). Shifting them by 3 made the colour too large.
I found a byte at 0x1F of each .col file, which is the number of bits to shift minus 1. For most of the files it's 2, but for c_com00.col and c_com01.col it's 1. So basically to fix this I just made my program read in that value and add 1, and then shift that number of bits.

I also improved it a little bit by not just removing palettes that are all 0s, but just any palette in which all the colours are the same. c_gimick.col has heaps of palettes that are all completely filled with green, which seem just as pointless, so it ignores those as well.

Anyway yeah, just download from the same link as before.

[John4300]

Hmh, weird that the values they change between files, but it's great if it's stored in the palette file so it can easily be multiplied in the fly.
Is it okay if I'll add your program to the main post?

Also, I uploaded the sprite files and linked them to the main post, in two variations. One having the stage and menu surfaces included which take 100mb on their own, and the other only having the sprites which only takes about 10mb. If there are problems with the fact that I upload the sprites (Not allowed to this site), please let me know. I didn't include music, or other crucial data to launch the game, so the zipped files cannot be run on the system/emulator.

Edit: Hmh, still some file oddities. Seemingly some files, possibly ones that are direct copies or from older developement time, give errors still, this time the CSR files. See DRA3 from boss folder or arlau from enemy folder as an example.

Converting DRA3/F_DRA3E.CSR
Invalid .csr header for DRA3/F_DRA3E.CSR!

[puggsoy]

Ah, right. I was checking what appeared to be a consistent magic ID in the header of the file to make sure it was a proper .csr. This seems to have backfired because for some reason not all .csrs have that same ID.
I've just gone ahead and made it check the extension instead, so just make sure it has a proper extension. Same with .col files. Again, same link as before.

And yeah, feel free to link it in the main post