UV coordinates in Digimon Story: Cyber Sleuth

[Toastline]

Greetings! I hope this is the right forum for this.
 I've wanted someone to convert the DS:CS models for a while, but since nobody seemed to be doing it, I took a stab at it.

However, I haven't been able to figure out how the texture coordinates are stored in the model files (or if they're even stored in the model files at all)!
User Ploaj  has started on a program to convert the game's image files, but apparently has not yet released a version that can do the model textures.
My findings are accessible here.
The .geom files I have used so far are here.
My converter that will give you textureless models, in obj format, is here.

Honestly, I would like someone to take this project off my hands altogether, but if that's not going to happen, I will try to continue for a little while longer. Can anyone help?

[Jinzo-Advance]

May I request the Renamon obj file with the textures and mtl file if you have them, please? I'd like to take a look at it to see what's up.

[Toastline]

May I request the Renamon obj file with the textures and mtl file if you have them, please? I'd like to take a look at it to see what's up.

You may, but it won't do you much good as far as discovering the texture coordinates goes. If my converter could put the uv coords in when it turned .geom files into .obj files, it would mean that I had already solved this problem!
https://dl.dropboxusercontent.com/u/1467...esting.zip
I can give you the obj and a texture (Lucky for you, this is the one texture Ploaj did manage to pull out before apparently vanishing!). My converter does not produce a working .mtl file yet.

[Jinzo-Advance]

I'll take a look. If it gives out even wrong UV coordinates, I might be able to fix it.

Edit: First off, I might have made a break through. Despite the lack of UV maps, I was able to figure out how to get the tail to look right. Blender will not import the obj file as it was, so I had to run it through another program convert it to something more useful. If you want, I could give you the blend file with some notes on what you need to do if you want to work on the model yourself. I'm not very familiar with the character design as I never watched the show or played the games, so any progress I make is based on guess work at best.

[xdaniel]

Hi there, a guy who's been working on a couple Vita-related things over the last few months here. For one, I've written GXTConvert, which is a GXT-to-PNG converter compatible with Cyber Sleuth and other Vita games using GXT textures: https://github.com/xdanieldzd/GXTConvert - In fact, CS (along with SAO: Hollow Fragment and Senran Kagura: Shinovi Versus) was one of the first games whose GXTs I worked with

Next up, after having seen some posts by Toastline about CS' .geom models a couple of days ago, I went ahead and took a stab at reversing them to render, and possibly export. Here's my preliminary results (in the spoiler, because tall images!):

Spoiler

Your initial findings about the file structure gave me a small headstart, but more research was needed (and, frankly, still is!) to get to texture coordinates, normals, primitive types, texture IDs, etc... Most important for these are the vertex attribute definitions, which are the block of data you had down, I think, as "???B" in your notes. These define what each element of data inside one vertex is used for, like - hypothetical example - "at offset 0x0 inside the vertex are 3 32-bit floats, which are used as the vertex position", or "at offset 0xC are 2 16-bit floats, which are used as the texture coordinates". And yes, there are 16-bit floating point numbers - I had to guess how to read them, failed multiple times, then ended up finding some code on Stack Overflow or somewhere like that that happened to work.

Anyway, my personal TODO list for this:

• Fix and improve the Collada exporting code (current code has been ripped from an earlier project of mine and just crammed into this one...)
  
• Generally figure out more about the format, ex. how the data blocks work that contain ex. texture IDs
  
• Generally improve the program's code and post a build for people to test
  
• After the above, eventually put the source for this and Cobalt (3D rendering helper library) up on GitHub
  
• Pipe dream time! Skeletal animation! Brrrr... <.<
  

I should be able to get my viewer and exporter far enough along so that you can easily render and export static models, but I'm unlikely to be able to figure out and export animations and the like; 3D animation has always been my weakness.

So, yeah, hope that helps somehow already?

[TGE]

I had casually started writing down my own specification on the format but I suppose there's no need for it now.
Of course if I could help figure out some of the remaining some of the unknown stuff I'd be up for it.

[xdaniel]

I had casually started writing down my own specification on the format but I suppose there's no need for it now.
Of course if I could help figure out some of the remaining some of the unknown stuff I'd be up for it.

I'll start writing down some actual documentation in a bit, as I'm one of those people who, for better or worse, just implement stuff in code directly instead of documenting what they do. I'll post that here once I'm done with it, then you could add whatever information you have that I'm missing, or what you might be able to figure out? I'm making a bunch of assumptions right now, which are kinda working out so far, but it's still guesswork that should eventually be confirmed or disproved.

[Toastline]

I'll take a look. If it gives out even wrong UV coordinates, I might be able to fix it.

Edit: First off, I might have made a break through. Despite the lack of UV maps, I was able to figure out how to get the tail to look right. Blender will not import the obj file as it was, so I had to run it through another program convert it to something more useful. If you want, I could  give you the blend file with some notes on what you need to do if you want to work on the model yourself. I'm not very familiar with the character design as I never watched the show or played the games, so any progress I make is based on guess work at best.

Ah, whoops! I must have sent an incorrect version of the model. Sorry about that! The converter in my post, if used on renamon's .geom, should output something that blender can load without needing to be adjusted first.



xdaniel:
Aha, I had suspected that half-precision floats might be involved somehow. Excellent work so far! It is highly fortuitous that you have a converter for the images already, too. I eagerly await your next update!

[Avenger_Seraphim]

Im no expert or something like that, but they used the same models for the mobile game Digimon Linkz. Althought the ripper said that the bigger models from Cyber Sleuth had better textures and maybe more polygons. You can check the models that he ripped (no bones though)
here http://withthewill.net/threads/15915-3D-...ext-0rder)

[xdaniel]

Im no expert or something like that, but they used the same models for the mobile game Digimon Linkz. Althought the ripper said that the bigger models from Cyber Sleuth had better textures and maybe more polygons. You can check the models that he ripped (no bones though)
here http://withthewill.net/threads/15915-3D-...ext-0rder)

If one just wants the models to rig them, use in some other game, etc., then these models there are likely just fine. However, I am personally more interested in figuring out Cyber Sleuth's specific model file format, i.e. the .geom files, - which I'm quite sure is unrelated to Unity - for curiosity's sake as well as potential unused content, instead of actually using the models for anything. If that means I'll be alone in trying to research and document the format, then that's fine by me.

Edit: Plus, I assume Linkz doesn't have Cyber Sleuth's player characters, NPCs, level maps, etc., which are all in the .geom format as well. Totally forgot about these, heh

[Jinzo-Advance]

What you can do in blender is clean up the mesh to delete loose vertices, edges, and faces in edit mode. From there click the lower left corner of the 3d view window and drag it down to get rid of the time frame window. Click again and drag it to the right to create another window. On the left 3D view Windows, click the cube at the bottom and select UV editor. After that, it comes down to doing these things:

In the 3D view:
selecting vertices/edges/faces
aligning your field of view so you can see the from a perspective that looks similar to the texture
unwrapping UVs from your FOV

In Uv editor:
Seclecting the vertices/edges/faces
Scaling, rotating, and translating them into their proper spot.

[xdaniel]

In case there's still interest in cracking the .geom format, here's my work-in-progress notes about it:

Code:

#####################
Geom Header Structure
#####################
0x00 -> Uint32: Unknown (always 0x00000064/100?)
0x04 -> Uint16: Number of meshes
0x06 -> Uint16: Number of "unknown material data"
0x08 -> Uint32: Unknown (always zero?)
0x0C -> Uint32: Number of "unknown data 1"
0x10 -> Uint32: Size of texture name data (divide by 0x20 for number of texture names)
0x14 -> Vector3: Unknown (maybe 3 float32s instead)
0x20 -> Vector3: Unknown (maybe 3 float32s instead)
0x2C -> Uint32: Unknown (always zero?)
0x30 -> Uint32: Pointer to meshes
0x34 -> Uint32: Unknown (always zero?)
0x38 -> Uint32: Pointer to "unknown material data"
0x3C -> Uint32: Unknown (always zero?)
0x40 -> Uint32: Unknown (always zero/potentially pointer?)
0x44 -> Uint32: Unknown (always zero?)
0x48 -> Uint32: Unknown (always zero/potentially pointer?)
0x4C -> Uint32: Unknown (always zero?)
0x50 -> Uint32: Pointer to "unknown geom data 1"
0x54 -> Uint32: Unknown (always zero?)
0x58 -> Uint32: Unknown (always zero/potentially pointer?)
0x5C -> Uint32: Unknown (always zero?)
0x60 -> Uint32: Pointer to texture names (0x20 bytes per name, padded with null bytes)
0x64 -> Uint32: Unknown (always zero?)
0x68 -> Uint32: Pointer to "unknown geom data 2" (only in character models, zero in map "f" models?)
0x6C -> Uint32: Unknown (always zero?)
(Total: 0x70 bytes)

++++++++++++++
Mesh Structure
++++++++++++++
0x00 -> Uint32: Pointer to vertex data
0x04 -> Uint32: Unknown (always zero?)
0x08 -> Uint32: Pointer to vertex indices (one Uint16 per index; might be variable like primitive type?)
0x0C -> Uint32: Unknown (always zero?)
0x10 -> Uint32: Pointer to "unknown indices" (one Uint32 per index?)
0x14 -> Uint32: Unknown (always zero?)
0x18 -> Uint32: Unknown (always zero/potentially pointer?)
0x1C -> Uint32: Unknown (always zero?)
0x20 -> Uint32: Pointer to vertex attributes
0x24 -> Uint32: Unknown (always zero?)
0x28 -> Uint16: Number of "unknown indices"
0x2A -> Uint16: Number of vertex attributes
0x2C -> Uint32: Size of single vertex
0x30 -> Byte: Unknown (ex. 0x02)
0x31 -> Byte: Unknown (ex. 0x05, 0x01)
0x32 -> Enum: Primitive type
0x34 -> Uint32: Unknown (ex. 0x478EE4F0, 0x9487C9D4)
0x38 -> Uint16: Index into "unknown material data"
0x3A -> Uint16: Unknown
0x3C -> Uint32: Number of vertices
0x40 -> Uint32: Number of vertex indices
0x44 -> Vector3: Unknown (maybe 3 float32s instead)
0x50 -> Vector3: Unknown (maybe 3 float32s instead)
0x5C -> Vector3: Unknown (maybe 3 float32s instead)
(Total: 0x68 bytes)

-----------------------------
Enum (Uint16): Primitive Type
-----------------------------
0x0000: Triangle Strip
0x0001: Triangles
[... more? ...]

++++++++++++++++++++++++++
Vertex Attribute Structure
++++++++++++++++++++++++++
0x00 -> Enum: Attribute usage
0x02 -> Uint16: Number of components
0x04 -> Enum: Component data type
0x06 -> Uint16: Offset in vertex data
(Total: 0x08 bytes)

------------------------------
Enum (Uint16): Attribute Usage
------------------------------
0x0001: Position
0x0002: Normal
0x0005: Texture coordinates
0x0006: Unknown
0x0009: Color (unsure)
0x000A: Bone IDs (unsure)
0x000B: Bone weights (unsure)
[... more? ...]

----------------------------------
Enum (Uint16): Component Data Type
----------------------------------
0x0000: Byte (unsigned byte)
0x0001: Sbyte (signed byte)
0x0002: Uint16 (unsigned short)
0x0003: Int16 (signed short)
0x0004: ByteN (unsigned byte N?)
0x0005: SbyteN (signed byte N?)
0x0006: Uint16N (unsigned short N?)
0x0007: Int16N (signed short N?)
0x0008: Float16 (half-precision float)
0x0009: Float32 (single-precision float)
(Note: same as SceGxmAttributeFormat)

+++++++++++++++++++++++++++++++++
"Unknown Material Data" Structure
+++++++++++++++++++++++++++++++++
0x00 -> Uint32: Unknown
0x04 -> Uint32: Unknown (ex. 0x088100C1)
0x08 -> Uint16: Unknown (ex. 0x0111)
0x0A -> Byte: Unknown (ex. 0x08, 0x00)
0x0B -> Byte: Unknown (ex. 0x00, 0x02)
0x0C -> Uint16: Unknown (always zero?)
0x0E -> Uint16: Unknown (ex. 0x0004)
0x10 -> Uint16: Unknown (always zero?)
0x12 -> Uint16: Unknown (ex. 0x0004, 0x0005)
0x14 -> Byte: Number of "material data entry" (1)
0x15 -> Byte: Number of "material data entry" (2)
0x16 -> Uint16: Unknown (ex. 0x0003, 0x0000, 0x0001)
(Total: 0x18 bytes)

+++++++++++++++++++++++++++++++++++++++
"Unknown Material Data Entry" Structure
+++++++++++++++++++++++++++++++++++++++
0x00 -> Byte[]: Raw data [0x10 bytes]
0x10 -> Enum: "Data usage"
0x11 -> Byte: Unknown (ex. 0x00, 0x04, 0x64)
0x12 -> Uint16: Unknown (ex. 0xFF00)
0x14 -> Uint32: Unknown
(Total: 0x18 bytes)

-------------------------
Enum (Byte): "Data Usage"
-------------------------
0x32: Texture ID
>> Raw Data: 0x00 -> Byte: Index into texture names
0x33: Unknown
0x48: Unknown
0xA4: Unknown
0xA8: Unknown
[... more? ...]
(Note: possibly PVR registers, common shader uniforms, or similar?)

+++++++++++++++++++++++++++++++
"Unknown Geom Data 1" Structure
+++++++++++++++++++++++++++++++
0x00 -> Vector3: Unknown (maybe 3 float32s instead)
0x0C -> Vector3: Unknown (maybe 3 float32s instead)
0x18 -> Vector3: Unknown (maybe 3 float32s instead)
0x24 -> Vector3: Unknown (maybe 3 float32s instead)
(Total: 0x30 bytes)

+++++++++++++++++++++++++++++++
"Unknown Geom Data 2" Structure
+++++++++++++++++++++++++++++++
[... unknown ...]

[Avenger_Seraphim]

Im no expert or something like that, but they used the same models for the mobile game Digimon Linkz. Althought the ripper said that the bigger models from Cyber Sleuth had better textures and maybe more polygons. You can check the models that he ripped (no bones though)
here http://withthewill.net/threads/15915-3D-...ext-0rder)

If one just wants the models to rig them, use in some other game, etc., then these models there are likely just fine. However, I am personally more interested in figuring out Cyber Sleuth's specific model file format, i.e. the .geom files, - which I'm quite sure is unrelated to Unity - for curiosity's sake as well as potential unused content, instead of actually using the models for anything. If that means I'll be alone in trying to research and document the format, then that's fine by me.

Edit: Plus, I assume Linkz doesn't have Cyber Sleuth's player characters, NPCs, level maps, etc., which are all in the .geom format as well. Totally forgot about these, heh

Hehe then you're right ^ I Totally forgot about the character models etc!! Im really interested to see them too : Its too bad that i cant help you :/ I will send my positive power thought xD  Keep it up

[Jinzo-Advance]

Been a little busy. I'll show you what I got. mainly just guess work on my part, but here's a picture to show:

   

Again, mapping the UV's is based on guesswork for me, I'm not familiar with the character.

[xdaniel]

I released a pre-release build and the source code (which is still missing the Cobalt library, so it's currently more of a public reference than something buildable) of my viewer/exporter on GitHub: https://github.com/xdanieldzd/CSGeom/

Do note that is is still somewhat buggy, it's incomplete, model exports might have texturing issues, etc., etc. - Some of these issues will be fixed, some might not be, I'll keep working on it so we'll see how far this goes.

Screenshots below:

Spoiler