The VG Resource
Switch BNTX research - Printable Version

+- The VG Resource (https://www.vg-resource.com)
+-- Forum: The Resources (https://www.vg-resource.com/forum-109.html)
+--- Forum: The Textures Resource (https://www.vg-resource.com/forum-113.html)
+---- Forum: Ripping Help (https://www.vg-resource.com/forum-117.html)
+---- Thread: Switch BNTX research (/thread-31389.html)

Pages: 1 2

Switch BNTX research - gdkchan - 07-21-2017

This thread is an attempt to collect information about the new texture format used on the Nintendo Switch, BNTX (Binary Texture). It's different from formats used on previous nintendo consoles (like the WiiU and the 3DS, at least afaik).

I started to write a tool to extract textures from the bntx container. Currently, it supports the following formats (list not guaranteed to be up-to-date):

- BC1 (DXT1)
- BC2 (DXT3)
- BC3 (DXT5)
- BC4
- BC5
- RGBA8888
- RGB565

BNTX texture tool: https://github.com/gdkchan/BnTxx

I also made a tool to extract the Swich RomFS. Discussion about said container is out of the scope of this thread, but if anyone is interested, tool can be found here: https://gist.github.com/gdkchan/635187f50a18539b36860a53493f275f.

Overview of the format:

BNTX is basically a texture container. It can contain multiple textures, have a PATRICIA-trie based dictionary that allows quick access to textures using names as key, and also a relocation table that allows the binary to be loaded anywhere in memory and the addresses can be easily converted from relative offsets to absolute pointers.

Sections:

- BNTX Main header, contains pointers to the other sections, and also some lengths
- _STR String table section. First name is always an empty string "\0", used by the root node of the tree.
- _DIC Dictionary using the PATRICIA tree, each node have 16 bytes.
- BRTI Texture information. Contains one for each texture on the file.
- BRTD Texture data. BNTX contains only one of this section with all textures inside. Textures are aligned into 0x800 bytes blocks, and the 16 bytes header comes before the data.
- _RLT Relocation table, it's the last section on the file and contains the addresses for all pointers inside the file.

All the sections that starts with _ can be ignored if one just whiches to extract textures, because all data can be obtained from other sections too. The BRTD header can also be ignored because the only information it contains is the length of the data section (which is only useful if you're going to read it into memory and use the buffer directly).

Swizzling:

Switch textures uses swizzling, the DXT compressed textures have swizzling applied to the address of the 4x4 tiles, and on non-compressed textures, it is used on the address of each pixel. On the tile address, the bits from the X and Y coordinates are distributed using this pattern: yyyy x yy x y. However, after certain point it seems to use linear addressing, and this point is when either the numbers of available bits are over, or when the biggest tile size (which is 4/8/16/32x128, see below for details on the width/X pad) is hit. Take this information with a grain of salt, since it's not guaranteed (and most likely isn't) accurate.

Anyway, here is a real example (from a 512x512 dxt5 texture) that maybe can help you better understand the address format:

x x x x x y y y y x y y x y 0 0 0 0

Note that the entire address have 18 bits, which is the size of 512 * 512 - 1 = 0x3ffff. Since we're talking about dxt5 textures here, the lower 4 bits are the address inside the 16 bytes tile data block, this one is linear and you don't need to worry about it.

It will keep the pattern for the biggest tile size that can still fit inside the texture (see exception below). Which is 4x128 on a 2048x2048 (512x512 tiles) dxt5 texture for example, or 4x16 on a dx5 texture with something like 30x21 tiles. It decides whenever to round up or down based on the wasted height. Below you can find pseudo-code that shows how it's calculated:

Code:
//Note: Perform rounding only if number is NOT a power of 2 already, otherwise the code below can be ignored entirely.
height_rounded_up = pow2_round_up(height)
height_rounded_down =  pow2_round_down(height)

IF height <= height_rounded_down + height_rounded_down / 3 THEN
    height = height_rounded_down
ELSE
    height = height_rounded_up
END IF

For a 30x21 textura for example, a 4x16 block is used, while a 30x22 texture uses 4x32 blocks, and the texture data needs to be padded accordingly.

It's currently unknown if this only applies to compressed textures or all types, or even if this is accurate to what the hardware does, but this worked for all observed textures so far.

The "real swizzling" only seems to take place starting at bit 4. So, for example, dxt5 have 4 bits on the address for addressing inside the 16 bytes tile. On dxt1 on the other hand, each tile only uses 8 bytes, so only the lower 3 bits are used for addressing inside the tile. The extra "0" is filled with x. So, following the above swizzle, we have for dxt1:

... y y y y x y y x y x 0 0 0

And, for rgba8888, each pixel uses 4 bytes, so only the lower 2 bits are used for addressing inside the pixel color. We therefore have:

... y y y y x y y x y x x 0 0

For rgb565/rgb5551 and 16 bits formats:

... y y y y x y y x y x x x 0

and so on...

My current theory is that this was done to make hardware implementation simpler, maybe, since swizzling takes place at the same position, but since this is not my field I could be totally wrong here.

You can find a most likely shite implementation of the above swizzle here:

https://github.com/gdkchan/BnTxx/blob/master/BnTxx/SwizzleAddr.cs

Note that the upper bytes of the address uses linear addressing. So you need to calculate it as x + y * remaining_width, and shift the result to place it at the top bits. This is necessary for non-power of 2 textures.

Observed texture data width seems to be padded. On tiled textures, it seems to be padded so that the width is always a multiple of 4 (and 8 for 64 bits formats?). For RGBA8888, it seems to be padded to be a multiple of 16, and on RGB565/L8A8 a multiple of 32.

Some textures:

Those are some textures extracted from Puyo puyo tetris, the game I'm using to do this research, and also one of the few games that interests me on the Switch currently:

[+]Spoiler
Any suggestion for improvement, correction or new information is welcomed.

TODO list:

- Figure out how non-compressed textures are swizzled (they seems to be encoded into 8x8 tile blocks but i'm not sure yet).
- Add support for more formats
- Support cubemap textures

RE: Switch BNTX research - Random Talking Bush - 07-22-2017

Nice to see someone else working on the format and seeing what crap I'm having to go through right now with my own script. I'm pretty much in the same spot swizzling-wise (with non-tiled textures being incorrectly extracted), but allow me to give you a little push in the right direction while I'm getting my own QuickBMS script finished off. Here's a list of the formats I've come across -- the first byte's the actual format, and the second's for the variant:

Code:
0x02XX - R8
0x07XX - B5G6R5
0x09XX - R8G8
0x0AXX - R16
0x0BXX - R8G8B8A8
0x0FXX - R11G11B10
0x14XX - R32
0x1AXX - BC1 / DXT1
0x1BXX - BC2 / DXT3
0x1CXX - BC3 / DXT5
0x1DXX - BC4 / ATI1
0x1EXX - BC5 / ATI2
0x1FXX - BC6H
0x20XX - BC7

0xXX01 - _UNORM
0xXX02 - _SNORM
0xXX05 - _FLOAT
0xXX06 - _UNORM_SRGB
0xXX0A - _UF16

And as for some of the unknowns you have listed:

Code:
Unknown14 = Actually two values. First two bytes I'm trying to figure out (and range from 0-3 and 5), but the second two are the amount of mipmaps (normally 0x01 for UI stuff).
Unknown30 = The amount of "faces" the image has (used for font files and cubemaps).
Unknown5C = Not sure if this is meant to be a long value or just a short, but 0x01 = Default texture, 0x03 = Cubemap, and 0x08 = CubemapFar.

Unknown0C, Unknown64 and Unknown6C (as well as the next four bytes after DataPtrAddress) are always 0 because they're 64-bit offsets.

I'm sure the finished product will be a lot better than anything I'll come up with, so keep it up! Tongue

RE: Switch BNTX research - gdkchan - 07-22-2017

(07-22-2017, 09:41 AM)Random Talking Bush Wrote: Nice to see someone else working on the format and seeing what crap I'm having to go through right now with my own script. I'm pretty much in the same spot swizzling-wise (with non-tiled textures being incorrectly extracted), but allow me to give you a little push in the right direction while I'm getting my own QuickBMS script finished off. Here's a list of the formats I've come across -- the first byte's the actual format, and the second's for the variant:

Code:
0x02XX - R8
0x07XX - B5G6R5
0x09XX - R8G8
0x0AXX - R16
0x0BXX - R8G8B8A8
0x0FXX - R11G11B10
0x14XX - R32
0x1AXX - BC1 / DXT1
0x1BXX - BC2 / DXT3
0x1CXX - BC3 / DXT5
0x1DXX - BC4 / ATI1
0x1EXX - BC5 / ATI2
0x1FXX - BC6H
0x20XX - BC7

0xXX01 - _UNORM
0xXX02 - _SNORM
0xXX05 - _FLOAT
0xXX06 - _UNORM_SRGB
0xXX0A - _UF16

And as for some of the unknowns you have listed:

Code:
Unknown14 = Actually two values. First two bytes I'm trying to figure out (and range from 0-3 and 5), but the second two are the amount of mipmaps (normally 0x01 for UI stuff).
Unknown30 = The amount of "faces" the image has (used for font files and cubemaps).
Unknown5C = Not sure if this is meant to be a long value or just a short, but 0x01 = Default texture, 0x03 = Cubemap, and 0x08 = CubemapFar.

Unknown0C, Unknown64 and Unknown6C (as well as the next four bytes after DataPtrAddress) are always 0 because they're 64-bit offsets.

I'm sure the finished product will be a lot better than anything I'll come up with, so keep it up! Tongue

Thanks for the info. Would be nice if you could send me some sample files for each format so I can try to implement them on the tool. The only ones I could find on puyo puyo files was bc1, bc3 and rgba8888. I will try to implement the changes you suggested tomorrow.

Also, i'm not sure what you mean with non-tiled textures, but if you mean textures with non power of 2 sizes, I already fixed that and updated the source/post info to reflect the changes. I also added support for rgba8888 (swizzling is prety much the same).

Edit: Well, what you mean with non-tiles textures are the ones that doesnt uses compression Tongue Anyway I took a look at your format list, and for some formats (like dxt for example), storing data as float doesn't make sense (the only thing that can change on a dxt encoded block is the endianness of the data). So I guess some formats will always have this byte set to 1. Also not sure what UF16 (16 bits float a.k.a. Half Float I guess?).

RE: Switch BNTX research - Random Talking Bush - 07-23-2017

(07-22-2017, 08:28 PM)gdkchan Wrote: Thanks for the info. Would be nice if you could send me some sample files for each format so I can try to implement them on the tool. The only ones I could find on puyo puyo files was bc1, bc3 and rgba8888. I will try to implement the changes you suggested tomorrow.

Also, i'm not sure what you mean with non-tiled textures, but if you mean textures with non power of 2 sizes, I already fixed that and updated the source/post info to reflect the changes. I also added support for rgba8888 (swizzling is prety much the same).

Edit: Well, what you mean with non-tiles textures are the ones that doesnt uses compression Tongue Anyway I took a look at your format list, and for some formats (like dxt for example), storing data as float doesn't make sense (the only thing that can change on a dxt encoded block is the endianness of the data). So I guess some formats will always have this byte set to 1. Also not sure what UF16 (16 bits float a.k.a. Half Float I guess?).
Check your PMs, I sent you a bundle of 'em. And yeah, I guess I used the wrong term for those non-compressed textures, there, whoops!

And also yeah, UF16 is for half-floats (Unsigned Float, 16-bit). There's also SF16 which would be for signed half-floats, but I don't think it's used, or rather I haven't encountered any with it (as it is, there's barely any that I've seen with BC6H_UF16!).

Here's an updated list with all of the type combinations I've seen used in files:
Code:
0x0201 = R8_UNORM
0x0701 = B5G6R5_UNORM
0x0901 = R8A8_UNORM
0x0A01 = R16_UNORM
0x0B01 = R8G8B8A8_UNORM
0x0B06 = R8G8B8A8_UNORM_SRGB
0x0F05 = R11G11B10_FLOAT
0x1405 = R32_FLOAT
0x1A01 = BC1_UNORM / DXT1
0x1A06 = BC1_UNORM_SRGB
0x1B01 = BC2_UNORM / DXT3
0x1C01 = BC3_UNORM / DXT5
0x1C06 = BC3_UNORM_SRGB
0x1D01 = BC4_UNORM / ATI1
0x1D02 = BC4_SNORM
0x1E01 = BC5_UNORM / ATI2
0x1E02 = BC5_SNORM
0x1F0A = BC6H_UF16
0x2001 = BC7_UNORM
0x2006 = BC7_UNORM_SRGB
0x2D06 = ASTC (thanks, gdkchan!)
0x2F01 = UNKNOWN!
The last one in the list is one that's in the samples I've sent you. I have no idea what format it's supposed to be, everything I've tried just comes up with garbage (unless the texture itself's supposed to look like that, hrm).

(EDIT: Sent you another example with another unknown format that I found.)

RE: Switch BNTX research - gdkchan - 07-23-2017

Thanks for the files. I managed to fix some stuff and discover what some of the unknowns means with them. Extraction of non power of 2 textures is still hit or miss unfortunately, I still didn't totally figured out how the swizzle works, but its better at least. Tomorrow I plan to work to fix the remaining swizzling issues.

About the unknown formats you sent me, I took a look on the last one, and it's using ASTC compression. Heres one of them decoded (tc_MiiSuit_39^w):
[Image: WL38pDz.png]

This compressed format is still not supported on my tool (and i'm not on the mood to write a ASTC decompressor :I but I will end up doing it sooner or later anyway).

RE: Switch BNTX research - gdkchan - 07-24-2017

Well the other unknown format is also ASTC (but with a different block size). ASTC supports different block sizes, so I imagine that a bunch of formats starting at 0x2D are ASTC with different block sizes.

Trying to decode as if it used 5x5 blocks gives me this (SetMoonPhone_00^y):

[Image: iaGAwoZ.png]

So I believe that:

0x2D = ASTC 4x4 block
0x2F = ASTC 5x5 blocks

Btw did you have any luck figuring out how the swizzle works on non pow2 textures?

Edit: Updated because I was able to decode the texture properly. Also, I believe that starting from 0x2D, those are the texture formats:

0x2D ASTC 4x4 block
0x2E ASTC 5x4 block
0x2F ASTC 5x5 block
0x30 ASTC 6x5 block
0x31 ASTC 6x6 block
0x32 ASTC 8x5 block
0x33 ASTC 8x6 block
0x34 ASTC 8x8 block
0x35 ASTC 10x5 block
0x36 ASTC 10x6 block
0x37 ASTC 10x8 block
0x38 ASTC 10x10 block
0x39 ASTC 12x10 block
0x3A ASTC 12x12 block

Of course, I was only able to test two of those. If you find the some of the other possible ASTC formats, we can confirm this theory.

RE: Switch BNTX research - KillzXGaming - 07-26-2017

Nice work on the tool! I'm not sure if you've looked into BC4 textures yet, though i've taken a look at some BC4 textures myself and it seems that they're missing the green and blue channels.
Here is DK's texture in MK8D
http://i.imgur.com/Pl9l9r9.png

And in original MK8 for wii u.
http://i.imgur.com/RcL7Hat.png

RE: Switch BNTX research - Random Talking Bush - 07-26-2017

(07-26-2017, 11:15 AM)KillzXGaming Wrote: Nice work on the tool! I'm not sure if you've looked into BC4 textures yet, though i've taken a look at some BC4 textures myself and it seems that they're missing the green and blue channels.
Here is DK's texture in MK8D
http://i.imgur.com/Pl9l9r9.png

And in original MK8 for wii u.
http://i.imgur.com/RcL7Hat.png
Well, BC4 is just a greyscaled image anyway, so the green and blue would be identical to the red channel.

(07-24-2017, 10:22 PM)gdkchan Wrote: Btw did you have any luck figuring out how the swizzle works on non pow2 textures?
The way I've got it for my QuickBMS script right now is incredibly hacky and frankly I'm not even sure how it's working, so I can't say for sure. Unsure

RE: Switch BNTX research - KillzXGaming - 07-26-2017

Ah yes true. Does appear to be identical to original if i copy to the other channels.

Also another note, BC5 textures in this case used as normal maps seem to look quite strange.
http://i.imgur.com/j9DzRIV.png

Original
http://i.imgur.com/RVVIw0T.png

RE: Switch BNTX research - gdkchan - 07-26-2017

(07-26-2017, 12:15 PM)KillzXGaming Wrote: Ah yes true. Does appear to be identical to original if i copy to the other channels.

Also another note, BC5 textures in this case used as normal maps seem to look quite strange.
http://i.imgur.com/j9DzRIV.png

Original
http://i.imgur.com/RVVIw0T.png

Well, normals are unit vectors, so on BC5 the Z component is not stored on the texture, only X and Y. Then Z is calculated based on X and Y (since X² + Y² + Z² are always 1 on unit vectors, Z can be computed as sqrt(1 - (X² + Y²)). Those blocks does seems weird through, its probably because the sign [-1, 1] is not being interpreted correctly. The way fragment shader expects is usually 0 being -1, and 255 being 1, but on a signed byte it's mapped as 128 = -1, 0 = 0 and 127 = 1. Not sure if this is the problem through.

Anyway I didn't looked into this yet, My focus currently is getting the swizzle correct for all textures, and then I'll start working on each pixel format. But thanks for pointing out this issue, I'll try to fix it as soon as I have the swizzle fully working.

(07-26-2017, 11:39 AM)Random Talking Bush Wrote: The way I've got it for my QuickBMS script right now is incredibly hacky and frankly I'm not even sure how it's working, so I can't say for sure. Unsure

This seems to be quite a interesting piece of code. I kind of isolated the problem I have with the swizzle currently, and also managed to understand a little better how it is supposed to work.

The easiest way I found to understand the ordering was this:

We first we have a 2x2 block. I will call it the "small block". Each small block contains 4 (2x2) tiles, and is ordered like this:

Code:
+--+--+
|0 |2 |
+--+--+
|1 |3 |
+--+--+

Then, we have a 2x4 block. Let's call it the "medium block". Each medium block contains 8 (2x4) small blocks, and is ordered like this:

Code:
+----+----+
|0   |4   |
|    |    |
+----+----+
|1   |5   |
|    |    |
+----+----+
|2   |6   |
|    |    |
+----+----+
|3   |7   |
|    |    |
+----+----+

And then we have a 2x16 block, a "large block". Each large block contains 32 (2x16) medium blocks. The ordering follows the same pattern of the previous blocks, and this is the biggest swizzled block that a texture can contain (at least on the files I observed). When the image is smaller than the block size (for example, images with less than 128 pixels of height (or 512 for 4x4 tiles compressed textures), then the block is ignored, and swizzle is performed just on the block sizes that does "fit" inside the texture.

So, basically my current problem is - what happens when the image size is not a multiple of those block sizes? Or when they don't have a power of 2 size?

The following textures can be decoded fine:
- Size is power of 2
- Width and height are greater than 128 (or 512 for compressed textures)

All other cases may or may not work properly.

I also generated mapping for some of the texture sizes for analysis.

[+]Spoiler
Both are extracted correctly with those patterns, but they were generated using different code, and they are also different. So I basically, don't know why this difference exists, and I'm currently trying to make some sense out of it. I spent some time yesterday trying different solutions, but none of them worked for all samples.

Also, another thing to take into account:  Unused values = space waste. Basically, for a 64x66 textures, this would mean 64x62 pixels worth of wasted space on the file, however this is not what it does, it never wastes so much space. I would expect it to round the texture down to 64x32 then address the top bits linearly, but that doesn't seems to be the case either - rounding it down fixes some textures but break others.

RE: Switch BNTX research - aboood40091 - 07-26-2017

I made a BNTX extractor (in which the swizzling is based on your algorithm, but with slight improvements) and every texture I tried works fine.

RE: Switch BNTX research - gdkchan - 07-26-2017

(07-26-2017, 03:23 PM)aboood40091 Wrote: I made a BNTX extractor (in which the swizzling is based on your algorithm, but with slight improvements) and every texture I tried works fine.
Nice one! I took a look on the repo, and the readme says "format used in Wii U games" but I guess you meant "Switch games", just a note.

Also, the swizzling issue only affected a few textures. But well, good news is that I (think) that I managed to fix it. At least it's working on all samples I have here, yay! What I discovered using trial and error, was that the wrong textures was always on the same height range. Basically, if we have for example, a texture with height 68. We will need to round it up to calculate the number of bits for each coord value (X and Y), so we get 128. If we subtract 128 - 68 we get the wasted space height, so basically 60. Looks like that it uses some threshold to decide whenever to round up or down based on that. If the wasted height is greater than 2/3 of the rounded down height (in this case 64), it seems to round down, otherwise it rounds up. Now tbh, this seems a bit weird, but all the tests I did so far indicates that this is the right thing to do. With this code in place, all the swizzle issues I knew of are now gone. The code on github is already updated, and if anyone find any swizzle issues just let me know.

I will try to fix the issues pointed by KillzXGaming tomorrow.

RE: Switch BNTX research - gdkchan - 07-27-2017

(07-26-2017, 12:15 PM)KillzXGaming Wrote: Ah yes true. Does appear to be identical to original if i copy to the other channels.

Also another note, BC5 textures in this case used as normal maps seem to look quite strange.
http://i.imgur.com/j9DzRIV.png

Original
http://i.imgur.com/RVVIw0T.png

This should be fixed now.

RE: Switch BNTX research - Random Talking Bush - 08-05-2017

Found another unknown format in Splatoon 2, 0x3106, which seems to be ASTC-based. I sent it to you to check out at your own leisure.

RE: Switch BNTX research - gdkchan - 08-05-2017

(08-05-2017, 01:48 AM)Random Talking Bush Wrote: Found another unknown format in Splatoon 2, 0x3106, which seems to be ASTC-based. I sent it to you to check out at your own leisure.
Thanks for the files. They are already supported by the tool actually (sort of), my assumption seems to be correct.
[Image: VPwIfiD.png]
[Image: DhOnV1p.png]