The VG Resource

Hi! I'm doing RE work on Luigi's Mansion 2: Dark Moon. This post, as well as a little C# program I created, is dedicated to documenting file formats of this game.

1. data and dict files
Data contains actual data and dict files are like archive headers and contain some basic info about the files. 
The basic structure of dict is this(put it in a spoiler because it's too long and messy):
Data structure is very simple. It just contains all the files "glued" together with padding to closest 8-byte boundary every time a file starts.

Dicts have some sort of mystery in that they seem to have dummy and null entries for many files. Even though we can kinda extract them, we should figure out why they exist in the first place.

2. Files inside data/dict archives
(Disclaimer: the subfiles have no file names whatsoever, so the names are used are the same as in my code)

file000 contains information about subfiles in file003, containing offset, length and type This data is usually preceded with 0x01130002 with const length of 0x18, purpose of these is unknown.

file002 probably contains meta info on files stored in file003. For now only purpose of PÓwé(or Powe as I'll be referring to these, because I don't have a way to input diacritic marks on my keyboard) sections is known, they contain info about textures(like resolution, format, mip levels)

The structure of Powe is like this:
Here are a few textures I've dumped with my tool.
3. Models
Models are stored in file003. Stuff can be organised into so called "model groups"(basically models which share the same vertex data, submesh info and other stuff). Each model group contains vertex data section (contains vertex stuff and vertex indices), submesh vertex start pointers (relative to vtx data start), submesh info (has stuff like index count/offset, vertex count, data format, etc.) and some other stuff(bones probably, but I haven't figured those out yet).

About vertex data formats: there are a whole bunch of them...and I really mean A LOT(just look at my code)...
A lot of them are kinda similar. Like most of character models use ShortFloat(IDK how this is properly called) for vertex encoding, while levels use float32. I sadly can't figure out much more than vertex coordinates (can't even find UVs in some cases), so if anyone is willing to and can actually help, please do. 

I was able to write a working exporter to wavefront obj for some of the formats, here are a few models I've ripped:
Screenshot of Luigi model just as an example:
In the attached archive you can find this model and E. Gadd's in obj (and Luigi in collada, because wavefront doesn't support multiple textures per material). I didn't bother adding other stuff I've tried to rip, mainly because UVs are all over the place and a lot of stuff doesn't work properly.

[attachment=9739]

4. NLOC
NLOC - Next-level LOCalization(?) files contain text strings. You can extract these with RoadrunnerWMC's python script(look for it in his gihub gist)

5. NMLB
NMLB - Binary Luigi's Mansion N-something (reversed because little endian). IDK what it is, but I just simply came across this signature while browsing some(probably cutscene) of the files in the hex editor

6. FENL
FENL - IDK what this stands for. Looks like layout data based on where it's found

7. Audio
The audio is stored in wwiseaudio folder. You might thing that wwise-unpacker might extract these, but it fails. The reason is that despite there files being the standard .bnk and .pck variety. The reason wwise-unpacker fails is that these contain audio compressed with the same compression as BCSTM files (DSP ADPCM in case of LM2).

This is a brief writeup on what I've found. If you want to go a bit more in-depth, then go look at the code of LM2L, a tool I made for extracting files from this game. You can find the source code on GitHub. (To anyone wondering, the OGL viewer is probably not gonna be a working thing because I know nothing about OpenGL and how to program it)
P.S. Bonus: in the code.bin there is a string referring to "DS Horror". Seems like they didn't have ideas on how to name the thing or just used this name as a placeholder until they came up with "DualScream"

*bump* I guess nobody cares...
Anyway, I've reversed quite a few things, updated the post with that info and made a tool which extracts some stuff!
Btw, @Twilighter, I know you've said that you don't remember anything about what you've found in your post, but maybe you have some more notes in your files? I've read through your findings but didn't manage to make sense of it, the adresses just seemingly pointed into the middle of index table or into nowhere.

I care, namely cause Luigi's Mansion 2 Dark Moon has been needing some good rips for quite a while now.


Out of curiosity, would your research work with Punch-Out Wii? They seem to be developed by the same studio, so you think that would result in working with both.

(05-19-2019, 06:45 PM)The Prawn Wrote: [ -> ]I care, namely cause Luigi's Mansion 2 Dark Moon has been needing some good rips for quite a while now.


Out of curiosity, would your research work with Punch-Out Wii? They seem to be developed by the same studio, so you think that would result in working with both.

Good to see that someone cares! I'm not familliar with Punch-Out on the Wii and the series in general, but I'll see if I can figure anything out.
Edit: Oh wow. I just opened the files, and I saw basically the same directory structure! Also, as I've suspected from a string in each dict in LM2, .debug files do exist! Now I'm interested if the structure's the same just in big endian...

Edit 2: just for the sake of not creating a separate thread(let's be honest, LM2 and Punch-Out Wii do have too many things in common, so this makes sense), I'll dump anything on punch-out that I find in this post.

1. dict structure isn't totally the same(though has similarities to LM2), but features the same "signature" just in BE. Unlike LM2, this game doesn't use any compression on the data whatsoever.
And also this time we have a magical file \art\hashid.bin, which contains strings asociated with IDs! I should note that even if that file has ID reference, it doesn't neceserely mean that ID corresponds to your data(like texture hashIDs just correspond to some random stuff)

2. Haven't figured it out completely yet, but something Powe-alike seems to be here too. The section is 0x60 in size, starts with hashID in uint, followed by two ushorts that are widths and height. For not I wasn't able to find where in that whole structure the texture format is stored, but through analyzing this game in Dolphin emulator, seems to me like it uses two texture types: CMPR and RGBA8(following the same trend with using just two texture types like in LM2, aren't we NLG?). For now I have no way to autoextract those(cause IDK where texfmt is specified), but bodging pure texture data to a tpl file and altering it some, then loading in brawlbox resulted in success! Have this texture I randomly extracted from \art\fe\febundlefrontend(and I also very much noticed that the naming scheme is exactly the same between LM2 and Punch-Out Wii, even going as far that quite a few files are in the same place with the same names)
3. NLOC is exactly the same as LM2 format-wise, except for it being in BE this time. Can be extracted with RoadrunnerWMC's python script, you just have to state endian='>' when calling readNLOC() and it'll read it just fine! Also his assumption of NLOC standing for "Next level game LOCalization" is confirmed, since these files have ".loc" file extension in this game.

4. FENL is here too, I'm now pretty confident that it's layout data

5. Sound is still wwiseaudio, again using DSPADPCM for encoding, so no program will read it without transplanting the audio data into a compatible container(like .brstm, or putting the whole .wav into .genh)

6. "flow.data" seems to be a script file of some sort. Would be interesting to figure out the bytecode and decompile it

I actually went back and remembered enough to rip some more models and even submitted one to the site. It was fine but was suggested that I go back and check for redundancies or duplicates in model data...or something like that. Here is the model I uploaded: 

[attachment=9642]

I don't think I can really help much but I know that every model file has a vertex pointer list and an index pointer list. The pointers are respective to the start of the vertex data meaning they ignore all of the texture data that comes before it so it'll point to a spot in the file that seems off. The pointers would be accurate if you separated the model and texture data (that's what most people do who know what they're doing, not me). Also, If it seems like and address I noted points to an arbitrary location, consider that It may be pointing to the start of a vertex or index buffer for a submesh... It sounds like you know what you are talking about. Hopefully we can see some progress in ripping these models.

(05-20-2019, 04:28 PM)Twilighter Wrote: [ -> ]I actually went back and remembered enough to rip some more models and even submitted one to the site. It was fine but was suggested that I go back and check for redundancies or duplicates in model data...or something like that. Here is the model I uploaded: 



I don't think I can really help much but I know that every model file has a vertex pointer list and an index pointer list. The pointers are respective to the start of the vertex data meaning they ignore all of the texture data that comes before it so it'll point to a spot in the file that seems off. The pointers would be accurate if you separated the model and texture data (that's what most people do who know what they're doing, not me). Also, If it seems like and address I noted points to an arbitrary location, consider that It may be pointing to the start of a vertex or index buffer for a submesh... It sounds like you know what you are talking about. Hopefully we can see some progress in ripping these models.

Thanks so much for the info, this will be really useful(epecially that model info pointers are relative to model data, cause I didn't realize that either)! And I think I've just found the something about the first file in data (always starts with 0x01130002), which clearly has pointers to vertex and index pointer lists you talked about!

Btw, that ghost is from Scarescraper(or whatever the multaiplayer mode is called, IDK anything about it since I've never played multiplayer in this game)?

And another thing I'd like to ask(sorry if I'm asking too many questions) is which data type are indecies/vertecies? I found out that indecies can be a few different formats(some are bytes, others are ushorts) and vericies are stored as signed shorts, but like how do you turn a short into a floating point number?(I'm so confused about this "short" format, mainly because hex2obj and alike are all closed source and you can't really see how they decode it...and tutorials for all of those tend to purpousely skip specifics of short vertex coordinates)

For music, if it's like Luigi's Mansion 1, the DSP is being worked on by a guy at hcs, who is designing a custom, sequenced format to be read by vgmstream. It's in development, but he plans for it to work with other games like Super Mario Sunshine and Luigi's Mansion eventually. (Saw this talked about in their discord, but don't know if Luigi's Mansion 2 uses this same format).

(05-21-2019, 10:59 AM)Pingu! Wrote: [ -> ]For music, if it's like Luigi's Mansion 1, the DSP is being worked on by a guy at hcs, who is designing a custom, sequenced format to be read by vgmstream. It's in development, but he plans for it to work with other games like Super Mario Sunshine and Luigi's Mansion eventually. (Saw this talked about in their discord, but don't know if Luigi's Mansion 2 uses this same format).

I think I saw some of that, BMS has been a format that hasn't been reversed in such a long time(or atleast reversed to a good degree)...but music from LM1 3DS remake could probably be extracted even easier, since it's uses sequenced format very similar to sdat on DS(which is quite well RE'd if I remember correctly). But Luigi's Mansion 2 is pretty lame in this aspect. All audio is streamed(and I don't think wwiseaudio(which is an off the shelf commercial sound engine btw) even supports any kind of sequenced audio), and furthermore audio for every cutscene is premixed(which is strange considering that all cutscenes are rendered in real time and are not FMVs, which is not too common on the 3DS) meaning that background music is mixed with sfx(so were not getting a clean version of the elevator theme or any other cutscene music unfortunately). Actually LM2 wasn't developed in-house by Ninty, so this game has way more similarities to another Next-Level Games game Punch-Out Wii(and that also uses wwiseaudio btw) as I've discovered a few days ago (thanks once again The Prawn for suggesting me to look at it!)