Converting VDC memory dumps into an editable PNG?

[elmer]

Is there already a tool somewhere to take binary dumps from the VDC and VCE memory and interpret them as sprites/tiles, and then convert them into nice 256-color PNG files (with 16 x 16-color palettes) that can be edited?

It's time to rip the Japanese text-in-graphics out of the Xanadu games and redraw them in English.

[NightWolve]

http://www.pcenginefx.com/forums/index.php?topic=19670.msg423955#msg423955

Isn't PCX the preferred format ?

[dshadoff]

Viewing them is only half of the work; you need to know where they started from, so that the modified graphics can be reinserted.

I've used TMOD2 (with plugins I wrote for PC Engine) to go browsing in the binary for both sprite and background graphics.  Of course, the palette isn't known until runtime.

There were still a few issues I faced:
- How to extract/re-import the graphics in a useable format
- How to stitch together a series of neighbouring sprites into a visible canvas, to be edited
- How to deal with the fact that palette information isn't available until runtime

I can be lazy when it comes to graphics formats - and, I know that any graphic artist who would want to edit the file could likely play with colors in a way that violates the original palettes... so I wrote an extractor which extracts it as ASCII art.  And of course, the ASCII art characters are definable, for contrast.  Easier to keep track of the palette limitations (but less appealing to artists to edit).

I'm happy to share my code for this.  You could (if so motivated) extend it to write/read graphic file formats instead of textfiles.

And as a special bonus, I can send out the information I have for extracting virtually all of the graphic text from Bazaru de Gozaru.

Interested ?

Dave

[ccovell]

My aforementioned program can dump all 64K of VRAM itself as BG/SPR tiles in a .PCX, keeping all 256 palette entries intact, which is more or less what you wanted.  It can also dump the current BAT as a 256-colour .PCX with each tile palette intact.

As you have probably seen here:
http://www.chrismcovell.com/PCEmon/PCEmon_Commands.html#helpers

But yeah, this is a part of PCEmon and not some tool that reads in emulator savestates or ROMs...

[MooZ]

You can use yy-chr to view/edit the VDC dump and save it to BMP.

[elmer]

Viewing them is only half of the work; you need to know where they started from, so that the modified graphics can be reinserted.

I've used TMOD2 (with plugins I wrote for PC Engine) to go browsing in the binary for both sprite and background graphics.  Of course, the palette isn't known until runtime.

There were still a few issues I faced:
- How to extract/re-import the graphics in a useable format
- How to stitch together a series of neighbouring sprites into a visible canvas, to be edited
- How to deal with the fact that palette information isn't available until runtime

These are all exactly the kind of issues that I have in the Xanadu, together with the complication that everything is compressed into small chunks of data inside the META_BLOCKs.

Just "browsing" through the decompressed data files is of limited help when I don't know just how the graphics data is used.

I can use Mednafen to dump RAM/VDC contents when something is used, and see exactly how it is used, and then find out where it is loaded from.

Then I want to be able to reassemble that data into an "editable" format, with color indexes intact.

I do not want an artist changing those palette indexes and screwing up the re-insertion.

That's a bit different to a normal game-production workflow where the artist has a lot more freedom (within limits, of course)
 
 

I'm happy to share my code for this.  You could (if so motivated) extend it to write/read graphic file formats instead of textfiles.

My aforementioned program can dump all 64K of VRAM itself as BG/SPR tiles in a .PCX, keeping all 256 palette entries intact, which is more or less what you wanted.  It can also dump the current BAT as a 256-colour .PCX with each tile palette intact.

You can use YY-CHR to view/edit the VDC dump and save it to BMP.

Thanks guys, but I guess that my "requirements" and "desired workflow" are different-enough that I'm better-off just writing a dedicated tool to do what I need.

YY-CHR is the closest to what I want, but even it is missing a few crucial things that I want.

Isn't PCX the preferred format ?

I already have code for reading various types of BMP/IFF/PCX/PNG/TGA and writing BMP/PCX/PNG, so I've got options.

The source code for writing BMP and PCX is tiny, but code-size usually isn't a huge problem these days on a computer.

PNG just seems like the modern "universal" format, and the source-code is easy to include in a project without invoking any GPL nonsense, or requiring annoying DLLs.

Support for PCX just seems to be dying out, so I'm not sure if I get any benefit in using it for this.

[NightWolve]

Just seems in general over the years, beyond recent apps/tools like pcemon, they were sticking with the PCX format for whatever reasons (easy to translate to/from PCE graphic formats, I dunno?). I could swear I've seen several other PCE tools going with it around here, is all. As a format, yeah, I too noticed it declined some time after Win9X (late 90's) or whatever, but I still saw it used for retro hacking purposes and whatnot. Anyway, whatever works out for you.

[Bonknuts]

I had one, for medanfen vram dumps (and palette block dumps), not sure where the source is now. Should be trivial to write one up though (BMP format is so easy, you don't even need a lib for it). The only thing I caution, is when you parse the palette block and create a 256 palette picture, is use an embedding scheme for the subpalettes.

 It works like this:
 
- Use 36 base steps for the colors (to scale to full RGB on PC side)
 - Since there are ~36 values between each step, you can encoded a +1 or +2 difference into the color channels
- Think of RGB as 6bits; R=2bits, G=2bits, B=2bits. 00/00/00 would for the first 16 colors, 00/00/01 would be the next 16 colors.
 - This eliminates the issue of redundant colors being repacked down in a photoediting app (or even something like Mappy). It basically makes each color unique, but the difference is soo slight that it looks the same visually. Now you can edit graphics within the guidelines of the original palette definitions.
- For re-encoding back from the large pic, you can easily identify the subpalette for that 8x8 block be looking at the embedded difference scheme (I do this with my FMP conversion app, so I can work with subpalettes indirectly)

 Hopefully that makes sense.

[elmer]

I had one, for medanfen vram dumps (and palette block dumps), not sure where the source is now. Should be trivial to write one up though (BMP format is so easy, you don't even need a lib for it).

Yep, the basic print-to-bitmap for both the sprites and the backgrounds was only a couple of pages of code (and I like to use lots of white-space).

Both BMP and PCX are pretty simple to write ... but my BMP writer is somewhat complicated by handling both 24bpp and 32bpp as well as 8bpp (for historical reasons).

The only thing I caution, is when you parse the palette block and create a 256 palette picture, is use an embedding scheme for the subpalettes.

Now this is a really clever idea! 

Are you storing the original palette index in those bottom bits, or a palette number, or the repeat-count of the particular RGB value?

With the 1st, you can reconstruct the original palette order from just the bitmap file, but with the other 2, you still need access to the original palette in order to fix a mis-ordered palette.

But I'm just not sure how much help it's going to be to an artist if they're doing any reasonable editing on the file. 

If they're using something like Photoshop that has lousy palette handling, isn't it going to re-order the palettes on load/save and make it almost-impossible for the artist to actually edit the bitmap and keep to within a single palette in an 8x8 tile anyway?

Isn't the better "solution" just to get the artist to use an editing program that doesn't f**k up the bitmap's palette ordering (like "grafx" or "ProMotion")?

Are there some particular image-editing scenarios that I'm missing here?

OTOH ... if this really is a major benefit in practical use, then it sounds like it should be a separate utility that can be run at any time to either "fix" or "protect" a bitmap's palette by checking or creating these guard-bits.

****************

BTW ... the last time that I was working on a game that used tilemaps, we didn't use a map-editor like Mappy, we just did everything on a large bitmap in a bitmap editor.

Then the bitmap would get carved up into tiles by my converter program.

Pretty much like Mappy's "Create Map From Big Picture", or what ProMotion allows you to do (but theirs is much, much better).