Researchers have recovered a lost game for NES from 30-year-old floppy disks
Hi, I’m the founder of the Video Game History Foundation, Frank Sifaldi. Today, Rich Whitehouse and I will tell you a story about how we restored and reassembled Days of Thunder , an unreleased, previously unseen game co-authored by Chris Obert of Mindscape.
After leaving us in 2012, programmer and designer Chris Oberth made a long and diverse career in the video game industry. I think for most of us the main game of Chris Obert was Anteater - an arcade project that he invented and developed while working at Stern (or, perhaps, you know a clone of this game called Ardy the Aardvark , which Chris himself wrote for home computers, or another clone, Oil's Well , created by other people). For the rest, such a game could be Winter Games for Commodore 64 or, perhaps, its latest projects for arcade machines like Time Killers or World Class Bowling , which he developed at Incredible Technologies; or such iconic games for you could be several projects that he developed on his Apple II in the late 70s and early 80s (as he himself liked to show off, his Apple II had serial number 201).
In early 2020, a friend of the Obert family contacted the Video Game History Foundation, asking for help to deal with the materials left after him. In the basement of his house, where he often worked, for years he kept untouched heaps of old computers, backups on CD-Rs, floppy disks, records, tapes, EPROMs and data on tape drives, accumulated since his work with Apple II at the end of the 70's x His family agreed to temporarily give us these archives for analysis.
As often happens with such archives, the most important thing for us was to try to recover all of Chris’s lost works - the games he developed, which for one reason or another remained unreleased. Unfortunately, we almost did not find the remains of his work from the time of cooperation with Stern, that is, the unreleased arcade game Crypt , the source of inspiration for which was Hunt the Wumpus , most likely, forever lost. But the rest of the collection gave us hope for preserving his subsequent work for home computers and consoles.
The enigmatic "Hot Rod Taxi" was the first sign that an unpublished work could be stored in the Oberth code archive.
Especially curious was the 5.25-inch floppy disk with a handwritten note “NINTENDO HOT ROD TAXI FINAL”. The name of this game is unknown, but we knew that Chris had developed at least one game for the Nintendo Entertainment System - American Gladiators . We did not know the details of his work of that time, but, fortunately, later, Obert gave at least two interviews to the first sites of fans of retro games - one site Derek's Basement Arcade , and the second interview in three parts in 2006 to Retrogaming Times (it is published in releases 21, 24 and 27 of“ Retrogaming Times Monthly ”). In the latter, Chris briefly mentioned an unreleased game for NES - an unpublished version of Days of Thunder developed for Mindscape (eventually another project created by Beam Software was published instead). Was Hot Rod Taxi the same lost Days of Thunder ? Perhaps the name was removed from it under film license?
Hot Rod Taxi for NES, compiled from Obert's source code. This is not a game, it is more like the first technology demo. Days of Thunder slipped away from us again.
Probably. Maybe. Having made a blank disk rip and compiled a binary file from the source code and graphic data, we got a playable game for NES, which, although it was interesting from a historical point of view, was not very much like a game. It looked more like the first test of the concept; probably with her help, Obert first became acquainted with a completely unfamiliar platform.
We had another object for our searches: a huge pile of hard drive backups (mostly signed) that were created over the years and stored on approximately 40 floppy disks. These were not just files stored on disks - the backup was carried out by various tools for copying hard disks, that is, the files were split into parts and encrypted. The only way to recover them was to digitize the entire heap and collect all the data.
Now I pass the word to Rich.
Part of the hundreds of discs left over from Chris Obert. Is it possible that somewhere in this heap there are Days of Thunder ?
Our first task in the process of searching for Days of Thunder residues was to recover the maximum possible amount of data from various media, mainly represented by old 5.25-inch disks. Some of these discs had unreadable sectors, but many remained intact. In addition, some of these disks were immediately counted after the creation of FAT12 file systems under MS-DOS. However, the most promising part was a set of 21 disks containing backups of four separate sections of the hard disk in a proprietary format generated by PC Tools 5.1 . The inscriptions “PCTools 5.10. Backup ”on disc labels.
Our first attempt to recreate the 1989 Oberth hard drive using the backup tool failed - emulation via DOSBox was not able to complete this task.
First I tried running the PC Tools backup program through DOSBox , hoping that I could restore the backups using emulation. I was not able to get the program to recognize disk images under DOSBox, and I decided that to move along this path, it might be necessary to make changes to the emulator itself. At this point, I decided to go the other way to understand the data directly, and set about exploring the raw sector data in these backup disk images.
The data structure was not particularly complex. Each record about folders and files was stored in a packed stream, in which command bytes were used to indicate one of the three options. 0xDA means that a “folder entry” will be added to the current folder stack. 0xDB means “file record”, immediately followed by the file data itself. 0xDC means “extract folder”, that is, delete the very last folder from the folder stack. In the records themselves, I managed to identify the date/time words in MS-DOS style, which allowed us to determine the exact dates of each of the files in the backup.
The beginning turned out to be promising, it allowed us to generate a list of the files contained in the backup copies. We quickly found a bunch of source code and data that most likely represented our big prize: Days of Thunder . Unfortunately, we still had a problem: even though I managed to generate a complete list of paths from backups, most of the data we were interested in was compressed by a proprietary algorithm. It would be easy to understand the compression used here, but encryption was also used on top of the already compressed data.Fortunately, the encryption applied to the data was standard and did not depend on the key entered by the user - otherwise I would still restore the data, and not write this article!
Next, I considered the possibility of using several approaches. The first was to create your own DosBox build to make the PC Tools backup program work properly with the original copy images through emulation. The second approach was to reverse-engineer the encryption and compression used in PC Tools, for this it would take a lot of time to disassemble (and, ideally, debug) the program itself. I can disassemble systems that are larger and more complex than this, but I decided that my own build of DosBox would be a useful tool anyway. You can use it until it becomes clear that the backup data is somehow corrupted. In this case, I will have to parse every bit of them to collect everything useful from the wreckage.
A VGHF volunteer with the nickname Foone came to our aid. To correctly restore the backup copy of Obert's hard disk, he recreated the hardware environment corresponding to that era.
The third approach was much simpler than the two previous ones: use real "hardware"! Unfortunately, I didn’t have the right equipment at hand, but we were immediately able to take advantage of the help of the one who had it . The Foone volunteer was able to build and run the hardware system, and although we had to rip one of the backup disk sets so that PC Tools understood them, we were ultimately able to recover every bit of data from each of these four hard disk partitions.
Since the third solution worked, I never had to plunge into the jungle of reverse engineering of the compression algorithm used in PC Tools. However, according to rumors, it was most likely something like LZSS . I did some simple tests with the pcsecure.exe program from the PC Tools suite; it seems to use compression (although not exactly the same, judging by the results obtained by compressing the corresponding data), very similar to the compression used by the backup program. I also noticed that in some cases the compression failed and larger files were created than would be expected in the worst case for LZSS. For example, this situation occurred when compressing sequences of completely unique bytes, which increased the file size; I will assume that this is caused by the addition of control bytes to the data.
Source code build
After a brief study, I did not find anything resembling an already assembled ROM image. Of course, because it would be too simple. So, I had data located on four sections of the hard drive. I started with a folder named C: \ ROMX \ BTR . We were most glad to see her when we studied the resulting list of file names. It seemed that this folder contains all the source code and some data for Days of Thunder . The C: \ UTIL folder contained backup copies of many of the tools used to develop the game, including assembler/linker. In development, the X6502 assembler was used - the 6502 cross-assembler created by 2500 A.D. Software, Inc. Both assembler and linker were version 4.03b.
I compiled the sources and compiled all the objects, while 6 separate binaries were created. This brings us to the next part of the process - to a program called ROMX. This program is part of the ROMulator package created by GTEK, Inc. Notably, the company’s company’s website is still working . In our case, the ROMX program was used to transfer data through the RS232 computer port directly to the memory PRG/CHR NES development cartridge.It allowed developers to quickly update and test changes in the code and graphics on a real hardware. Even more convenient, they could selectively update the desired set of banks in order to reduce the iteration time when updating a specific piece of code or a set of tiles. There were quite a lot of batch files in the source code that ran ROMX commands to load specific files into specific areas in PRG/CHR. Here is the batch file from which you can understand the full PRG ROM schema:
The above commands tell us where to put 6 binary code files (those of which I spoke about the layout above) along with two pre-generated binary sound files. When one of the developers wanted to create a simple binary file from PRG or CHR memory, instead of copying all this information from a PC, he could just run ROMX with parameters allowing to dump all the data directly from the hardware and save it to a file.
At this stage of the game’s assembly, starting from the same number of PRG banks, it became obvious that we needed some kind of mapper, so I looked at the sources and came across a code that explicitly worked with registers MMC1 . I made the iNES header with my 128-kilobyte PRG ROM, set the mapper bits accordingly, and then just threw random garbage into the CHR ROM for testing. The game successfully loaded in the emulator and began to play music, displaying random data from random tiles. As I understand it, they should have been a splash screen.
After starting the game with a set of random data in the CHR ROM, I realized that we saw quite unique tile data. Some traces of this data were saved in the source code. In particular, there was a samopisny tool designed to parse blocks of tiles from the CLA files of the Pictor , recording them into a temporary file and load it using ROMX to explicitly specified CHR addresses. This process was called for small sets of tiles with a machine and adjustable wrench. However, the bulk of the CHR data was not mentioned at all, and there was no indication of how the CHR data was normally downloaded via ROMX.
In another folder, C: \ ROMX \ CHEDIT , I found a makeshift NES tile editor with remnants of data that seemed to belong to Days of Thunder . I managed to match some of the.map and.pal data exported here with the data collected in the PRG, and since they coincided, I assumed that the.chr files located here would also correspond to something related to CHR. I started with a file named CAR.CHR and went into the analysis of where exactly it should be in the CHR memory. It turned out that this is the address 0x3000. I was able to verify this by looking at the splash screen, even though the rest of the CHR was filled with random data:
We found and compiled these programs, but most of the graphics were missing! Unfortunately, we have come a long way, but still could not cross the finish line.
We made sure that this specific data belonged to CHR ROM, but many data were still missing. I found in the scripts of some utility for animating links pointing to images of the road and tiles in places that were not in the backup we restored. I was afraid of the worst - that we lost a large set of CHR data along with source data to restore them.
Knowing that I already have a likely candidate for finding most of the CHR ROM data, I took a unique byte string from CAR.CHR and performed a binary search on each of the backup files, hoping that I would be lucky and I will find the already collected CHR ROM fragment. The first attempt was unsuccessful. After that, I decided to apply a “nuclear” solution - I extracted each archive in every known format from each backup copy of the hard disks, and I did the same with every disk image that I was able to recover. I did another search on this entire array of information, and there was only one match in the 128-kilobyte binary. Oh yes!
Putting it all together
I was very careful, because it’s not a fact that this binary with such a convenient size of 128 KB will actually coincide with the sources and tiles with which I work. I was especially worried that the string match was found in another set of disks. However, at this point I already had no other options, so I took the file and used it to fill the entire CHR ROM in the NES ROM I created. It turned out that everything is in those places where it should be!
Finally, for the first time in thirty years, we saw the long-lost game Days of Thunder !
On this, our mission can be considered successfully completed. Unfortunately, it seems that we are missing a lot of initial data, and I still suspect that this data is stored somewhere on other sections of the hard drive that were not included in the main backup we restored. If we could not find the CHR binary from an seemingly independent dataset, then we would never be able to fully restore the game or end up creating our own tiles to preserve some kind of original game. This proves that every floppy disk is important!
Build the game and play it yourself!
With permission from the Chris Obert family, we uploaded to GitHub the source code of Days of Thunder together with native build tool and pre-compiled binaries.
Video game programmer Chris Obert (1953-2012) .