Puzzle & Dragons (Mobile)

[jkid101094]

Hey, guys. I know I haven't posted much on this forum yet but I figured this would be the best place to get help with my problem. I want to use the assets from Puzzle & Dragons and figured I'd just easily take them from the .apk. No biggie, right? Well turns out it's actually a big...uh..."biggie" because the sprites themselves aren't actually stored in the .apk; or at least not in the same form as most mobile games.

See most mobile games seem to have similar file structures with sprites generally being stored in the "res" folder. Unfortunately in the case of P&D that folder is filled with Twitter, Google+ and Facebook icons with not a single monster sprite to be found. A bit of Hex viewing via HxD, however, has lead me to one of two conclusions:

1). The Sprites are stored within' the .bin files in the assets folder. The .nam folder refrences most of these files by name implying that they're on the .apk.

or the more likely scenario:

2). When the game is first booted up there is a mandatory in-game download. At first I believed this to simply be a patch but this may very well be where the sprites are actually stored. The trouble is that I haven't been able to find these files.

It's also entirely possible that the sprites are located in the resources.arsc file but nothing I've seen seems to point to that conclusion. I know I'm not the only one having this problem as I can't find them anywhere online and even the two wikis for the game only have screenshots. It would be too tedious to remove the background of each sprite one by one so I'm hoping someone with more mobile knowledge will be able to lend a hand here. Of course, I'm certain the internet would appreciate the resources.

[puggsoy]

OK, so DATA000.BIN is kind of a weird format, can't tell what a lot of it means but I can see file offsets and sizes. At the very least they help with extracting by hand. DATA000.BIN seems to contain corresponding filenames for each file.

The other 3 .BIN files contain the data. DATA003.BIN has images, some of which I can manage to get:

A bit big

However, others are in a different format. Doesn't look to hard to find out (possibly 16-bit ARGB or something) but isn't supported by TextureFinder. I can maybe make a program to test it out later on.
The biggest issue is that I'm unsure how to tell which images are in which format. Some files contain multiple images with different formats, but I can't see any kind of format flag.

DATA002.BIN is weird, it just contains 3 small images, which are all 100% white. Probably negligible. DATA003.BIN contains sound effects in WAV format. Not terribly hard to extract.

You might be right about some sprites being externally downloaded, but the menu assets at least seem to be stored here. There is a lot of stuff to go through that I've only lightly scratched though, so it could be that everything is here.

I'll make an extraction script for the files anyway, which should get you the WAV files.

EDIT: Here's a QuickBMS script for them:

Code:

goto 0x20
getdstring FILE1 0x10
getdstring FILE2 0x10
getdstring FILE3 0x10

open FDSE FILE1 1
#open FDSE FILE2 2
open FDSE FILE3 3
open FDSE "DATA000.NAM" 4
savepos POS
set NMPOS POS
math NMPOS -= 0x50
get FSIZE asize 4

for NMPOS = NMPOS < FSIZE
    get v1 short
    get NUM short
    math NUM += 1
    get SIZE long
    get OFFSET long
    get DUMMY long
    
    goto NMPOS 4
    getct NM string 0x2E 4
    #get NM string 4
    
    if v1 == 0x8000
        if NUM == 1
            set NAME FILE1
            string NAME -= 4
            string NAME += "/"
            string NAME += NM
            
            goto OFFSET 1
            getdstring EXT 4 1
            
            if EXT == "TEX1"
                string NAME += ".btex"
            elif EXT == "BFNT"
                string NAME += ".bfnt"
            elif EXT == "DMSG"
                string NAME += ".bin"
            else
                string NAME += ".unk"
            endif
            
            log NAME OFFSET SIZE 1
        #elif NUM == 2
        #    set NAME FILE2
        #    string NAME -= 4
        #    string NAME += "/"
        #    string NAME += NM
        #    string NAME += ".btex"
            
        #    log NAME OFFSET SIZE 2
        elif NUM == 3
            set NAME FILE3
            string NAME -= 4
            string NAME += "/"
            string NAME += NM
            
            goto OFFSET 3
            getdstring EXT 4 3
            
            if EXT == "RIFF"
                string NAME += ".wav"
            else
                string NAME += ".unk"
            endif
            
            log NAME OFFSET SIZE 3
        endif
    endif
    
    savepos POS
    math NMPOS = POS
    math NMPOS -= 0x50
next

To use it all the files need to be in the same folder, and the file you select should be DATA000.BIN. In the output folder, you'll get two folders, DATA001 and DATA003. The latter contains the sound effects. The files in DATA001 are (mostly) textures, but not yet converted into a usable format.

[jkid101094]

Thank you very much! I'm glad I decided to ask here for help.
Unfortunately I'm not much help in the actual conversion department, but my inspection of the .btex files seems to imply that each (or at least the ones I checked) contain .png files. BannerPH.btex, for example, seems to be a very small blank image. I've also noticed the lack of Monster images (or even allusions to) in these files, though I haven't looked through every file yet.

alltex.bin seems to contain all the "hint" screen text displayed at the end of each dungeon which might be useful information to anyone wanting to edit those.


ngWords.bin also contains text. Let's see what we can fi-

Oh...OH...Oh my sweet Jesus. I'd post some of the text from here but I'm afraid it'd get me banned. I'm hoping this is the list of banned words for account names but it doesn't make it any less shocking to me...let's move on.

Looking through a few other files lead me to finding .jpg files as well. I'm almost positive that these .btex files are just compressed images at this point.

[Daxar]

Care to post a few for those of us who don't own the game?

[jkid101094]

Care to post a few for those of us who don't own the game?

So long as you're referring to the .btex files rather than their contents. Extracting the images is beyond my skillset.

https://www.dropbox.com/s/s1d2g5kqifetf3...01.7z?dl=0

I'm sorry I couldn't attach them to the post but I kept getting errors about the file types and sizes so I figured this way would be more convenient for everyone.

[puggsoy]

I just downloaded the APK to get the files (it's a free game so that shouldn't have any legal issues).

The .btex files are uncompressed raw pixel data, I don't know why there are .png images referenced within them (maybe some sort of reference to the files they were originally stored in or something). As I mentioned before, some of them are easy to convert (BGR565 and 32-bit BGRA, I think), while others are in some other kind of 16-bit format; I still need to make something to test them since TextureFinder doesn't have whatever format they're in (I'm guessing some sort of 16-bit BGRA or the like).

[Daxar]

There's actually... six kinds of images here. Complicated much?

Anyhoo, have a program to extract the .btex files: https://dl.dropboxusercontent.com/u/3181...BTexEx.zip
(works for all but one image type, but that image doesn't look interesting anyway for the files given, and I need sleep)

Oh, and for testing image types, I usually just use http://rawpixels.net/ . Simple and gets the job done.

[puggsoy]

Nice Also rawpixels.net looks really useful, thanks!

[jkid101094]

Is it possible that perhaps some palettes are mixed up? I've noticed the DC Rare Egg Machine sprite is green where in the game it should be blue.




If you plan on expanding on that extraction tool to incorporate the last image type, would it be possible to fix the palettes of some of the sprites too; or would you rather I do that manually myself with Photoshop?

The sprites I know for certain have palette errors are the DC Rare Egg Machine (GATYA_LJ_BODY) and the Neon Genesis Evangelion Rare Egg Machine (GATYA_EVA_BODY). Looking at a few other sprites it seems the palettes may simply have been assigned to the wrong images as the Pink Rare Egg Machine's palette looks to have been assigned to GATYA_GAKUSEI_BODY. I really hope you plan on fine tuning the tool as P&D is a regularly updated game meaning have these tools around will make further expansions easier to rip from in the future.

[Daxar]

My bad; I forgot to shift some of the bits over when extracting that kind of image. That's what I get for late-night coding! Kinda hard to tell what the images are supposed to look like sometimes. Colors, how do they work?

Try this one instead: https://dl.dropboxusercontent.com/u/3181...BTexEx.zip

This should extract the 8bpp images more correctly (maybe?) also; the 8bpp images here are miniscule and it's hard to tell what they're supposed to look like. Let me know if anything looks funky!

[jkid101094]

My bad; I forgot to shift some of the bits over when extracting that kind of image. That's what I get for late-night coding! Kinda hard to tell what the images are supposed to look like sometimes. Colors, how do they work?

Try this one instead: https://dl.dropboxusercontent.com/u/3181...BTexEx.zip

This should extract the 8bpp images more correctly (maybe?) also; the 8bpp images here are miniscule and it's hard to tell what they're supposed to look like. Let me know if anything looks funky!

Huh. Seems my last post didn't go through. Must have accidentally closed the tab without sending it.

I'm no Puzzle & Dragons expert by any means but from what I do know I can tell you your converter is working like a dream. Unfortunately this does leave us with the unfortunate fact that the Monster sprites aren't actually contained within' DATA001.bin. It seems likely at this point that my idea about them being in the initial "update" was correct; meaning we would need to get ahold of that file as well. Not to mention the Japanese version of the .apk which should contain the Dragonball Rare Egg Machine among other Japan exclusives. Thankfully with a little digging and research coming across those two files shouldn't be too difficult.

If you guys would like to take this chance to start uploading what we have to The Spriter's Resource that's fine by me. The way I see it you both deserve more credit in the endeavor than I do.

Edit 1:
I don't know why I assumed puggsoy's QuickBMS script would work for the more updated Japanese .apk. If I were to supply it's DATA001.bin would either of you be willing to give it a look?

[puggsoy]

I can try modify my script to work with the Japanese version, but I would need all the BIN files (as well as the NAM, if any).

[jkid101094]

I can try modify my script to work with the Japanese version, but I would need all the BIN files (as well as the NAM, if any).

http://www.dropbox.com/s/ah0l64f712y6veq...s.zip?dl=0
These are the files from the version of the Japanese .apk I was able to find (version 8.1.1 according to the filename). Hopefully adding compatibility for it is just a quick fix.

[puggsoy]

They seem to work fine for me. However I did fix it up a bit to handle the different possible file extensions in any of the files. E.g. it handles .wav files in DATA001.BIN now, in the other version WAVs were only present in DATA003.BIN and it assumed that. It also handles .btex or whatever else in DATA003.BIN, even though they're not there they might be in future versions.

Code:

goto 0x20
getdstring FILE1 0x10
getdstring FILE2 0x10
getdstring FILE3 0x10

open FDSE FILE1 1
#open FDSE FILE2 2
open FDSE FILE3 3
open FDSE "DATA000.NAM" 4
savepos POS
set NMPOS POS
math NMPOS -= 0x50
get FSIZE asize 4

for NMPOS = NMPOS < FSIZE
    get v1 short
    get NUM short
    math NUM += 1
    get SIZE long
    get OFFSET long
    get DUMMY long
    
    goto NMPOS 4
    getct NM string 0x2E 4
    
    if v1 == 0x8000
        if NUM == 1
            set NAME FILE1
            string NAME -= 4
            string NAME += "/"
            string NAME += NM
            
            goto OFFSET 1
            getdstring EXT 4 1
            
            if EXT == "TEX1"
                string NAME += ".btex"
            elif EXT == "BFNT"
                string NAME += ".bfnt"
            elif EXT == "DMSG"
                string NAME += ".bin"
            elif EXT == "RIFF"
                string NAME += ".wav"
            else
                string NAME += ".unk"
            endif
            
            log NAME OFFSET SIZE 1
        /*elif NUM == 2
            set NAME FILE2
            string NAME -= 4
            string NAME += "/"
            string NAME += NM
            
            goto OFFSET 2
            getdstring EXT 4 2
            
            if EXT == "TEX1"
                string NAME += ".btex"
            elif EXT == "BFNT"
                string NAME += ".bfnt"
            elif EXT == "DMSG"
                string NAME += ".bin"
            elif EXT == "RIFF"
                string NAME += ".wav"
            else
                string NAME += ".unk"
            endif
            
            log NAME OFFSET SIZE 2*/
        elif NUM == 3
            set NAME FILE3
            string NAME -= 4
            string NAME += "/"
            string NAME += NM
            
            goto OFFSET 3
            getdstring EXT 4 3
            
            if EXT == "TEX1"
                string NAME += ".btex"
            elif EXT == "BFNT"
                string NAME += ".bfnt"
            elif EXT == "DMSG"
                string NAME += ".bin"
            elif EXT == "RIFF"
                string NAME += ".wav"
            else
                string NAME += ".unk"
            endif
            
            log NAME OFFSET SIZE 3
        endif
    endif
    
    savepos POS
    math NMPOS = POS
    math NMPOS -= 0x50
next

It's mostly the same though. Make sure you're running it on DATA000.BIN specifically, not the whole folder all the files are in.

[jkid101094]

D'oh. I feel kind of silly now. The Japanese files didn't work for me because I was accidentally running them on DATA001.bin instead of DATA000.bin. Sorry about that. ^^;

The big difference between what's in this .apk compared to the last one (besides the obvious Japanese text) is the addition of Puzzle & Dragons World assets. I've never played the game so I can't vouch for everything but nothing I've seen looks off in the slightest which is a good sign seeing as it means that your current toolset should support all past and future versions of the game.

Oddly enough but still worth mentioning is that the only .wav file in DATA001.bin is a miscellaneous sound effect. I wonder why it isn't with the others?

I've had no luck getting that extra download data, however. Neither of you would happen to know where Android stores game updates, would you? I've tried to find the P&D apk on the family tablet but my sister has so much stuff on it that I might as well be looking for a needle in a haystack.