DS2x86 progress

March 18th, 2012, 22:53 Posted By: wraggster

For the past week I have been working on getting Jazz Jackrabbit running. It uses Borland RTM DOS Extender instead of the more common dos4gw extender. It has been a while since I last worked on it, and it has never progressed very far, so I thought that it was about time that I really looked into supporting the Borland RTM extender properly. Thus, I began debugging the game. Here is a list of the changes and improvements I have so far needed to do for that game.

1. Pretty soon after I began working on it, I noticed that it does not progress even as far as it did when I last worked on it a long time ago. It crashed with a "Page Fault in InitPage!" message when running a "repe stosw" opcode. The segment it tried to write to had a base address of 0xF0000004, which obviously was not within the 16MB of emulated memory. I spent some hours debugging it, comparing the behaviour to DOSBox, and finally I noticed that the game runs some memory allocation stuff twice in DS2x86, but only once in DOSBox. It seemed to enable various memory allocation handlers depending on the memory managers present on the system, and DS2x86 reported that both HIMEM.SYS and extended memory is available. I looked at DOSBox sources, and it reports that no extended memory is available if HIMEM.SYS features are enabled. I made DS2x86 also report that no extended memory is available, and then Jazz progressed up to the same crash location as it did a while ago. A bit silly that the game enables several memory managers at the same time, but of course on the real system only one method of handling extended memory is available at any one time.
2. The next problem was that the DS segment got an invalid value 0x15B9 in a "mov ds,ax" opcode. The segment descriptor tables only had valid values in the range 0x0000..0x011F, so that value was far outside that range. This problem was a bit weird, as the code looked like this:
   00E7:48C2 mov ax,15B9 inc bp push bp mov bp,sp push ds00E7:48CA mov ds,axThat is, the invalid value was loaded into AX register as an immediate value from the code segment! I spent a couple of days trying to debug this, but could not find the cause. Finally I decided to run the game in DOSBox, writing to a log file every single opcode and the resulting register values, up to the point where the game crashes in DS2x86. This log file was around 30 megabytes in size and had 483957 rows (for 161319 opcodes). Of course I did not need to check the result of every single opcode in DS2x86 and compare it with the log, but even checking the situation only after various function calls meant quite a lot of searching.
   
   Pretty soon I discovered that the game allocates some DOS RAM using a "LAST FIT" memory allocation strategy, that is, it requests a block of memory from the end of the DOS 640KB memory area. DS2x86 has never supported this strategy, it always allocates the next sufficiently large free block. This has not caused problems before, but this was obviously a potential problem. I wanted to make sure that this was the cause, though, so I actually spent a couple of days comparing the behaviour of the game in DS2x86 to the DOSBox log file. Finally I found the problem. The game frees a block of memory that still contains a part of the RTM DOS extender data, then allocates the LAST FIT memory block, clears it, and then copies the already freed RTM DOS extender data to the extended memory. Since DS2x86 allocated the new block over the freed RTM DOS extender data, it was cleared when the game cleared the newly allocated memory block. This cleared block was then copied to the extended memory, and when it was later used to set up various segment selectors, the selectors got invalid values because the block did not contain the data it should have contained. I implemented the "LAST FIT" DOS memory allocation strategy into DS2x86, and this enabled Jazz Jackrabbit to progress further.
3. The next problem was that the game crashed in a "les di,[bp+06]" opcode. This opcode loads both the ES segment selector and a DI register from memory. The selector was 0x01A7, which was in the correct range, but the corresponding descriptor had a base value of 0x545404AA! This was obviously again far outside of the supported 16MB emulated memory area. I also noticed that the descriptor did not have the "Present" bit on. This bit is used to cause a Page Fault when a segment selector is loaded, if the page is not present in memory. I checked in DOSBox, and indeed the game wants to cause a Page Fault at that point. I did not support Page Faults in that opcode in DS2x86 yet, and adding Page Fault handling to that opcode again allowed the game to progress further.
4. After fixing the previous problem, the game simply exited to DOS with a message "Loader error (0000): unrecognized error". This seemed to mean that all the work I had done so far were only for the Borland RTM loader, and that the actual game had not even started to run yet. I again ran the game in DOSBOX and generated an even larger log, this time it was over 120 megabytes in size until I stopped it running. After some more searching in this file, I noticed that at some point it runs interrupt 0x2F with AX=0xFB42 and BX=0x1002. Looking at Ralf Brown's Interrupt List this interrupt was called "Borland RTM.EXE 1.0 - EXECUTE COMPILED PROGRAM", which looked like it might be the "loader" that the error message talks about. I tried to break the game execution in DS2x86 at this interrupt, but the game did not get that far before printing the error. The code in question looked like the following:
   02E6:0710 push es mov ax,ds mov es,ax mov bx,0146 mov dx,0154 Pointer to "C:\GAMES\JJRABBIT\rtm.000" string mov ax,4B00 int 21 DOS 2+ - EXEC - LOAD AND EXECUTE PROGRAM jnc 0725 jmp 085802E6:0725 mov ax,4D00 int 21 DOS 2+ - GET RETURN CODE (ERRORLEVEL) cmp ax,0300 Is the return code == 0x0300 (terminate and stay recident, no errors)? je 0734 call 0805 jmp 076B02E6:0734 pop es mov ax,FB42 mov bx,1002 mov dx,[01E0] int 2F Borland RTM.EXE 1.0 - EXECUTE COMPILED PROGRAMI noticed that DS2x86 ran fine up to the 02E6:0725 address, but did not get to the 02E6:0734 address. So, it looked like the return code was not correct. And indeed, even though the "rtm.000" program stayed resident in memory, my INT 21 AX=4D00 handling never returned anything in the AX high byte! I fixed the INT 21 AX=4D00 handler to set the AX high byte properly, and then the game again progressed further in DS2x86.
5. The next problem was similar to the "les di,[bp+06]" problem above, but this time the opcode was "mov es,[bp-02]". I added the Page Fault handling to this opcode as well, and then I got the actual game to start and print the welcome text strings!
   
6. The problem was that after printing the welcome strings, nothing more happened. It looked like the whole MIPS side had hung, as I could not even drop into the debugger. So, I began hunting for the location where the hang occurred, by looking for returns from routines, and then tracing over the subroutines they called until I got to higher and higher level, and found the routine that made the system hang. After that I then burrowed deeper and deeper into the hanging routines, until I was at the lowest level, and found out that the system did hang immediately after the game gave a 0x1C (start 8bit auto-init DMA playing) command to the SoundBlaster DSP port. I checked the emulated SoundBlaster variables in DS2x86, and noticed that at that point (which actually begins the digitized audio playing) the length of the memory block to play was still zero! When using auto-init DMA, the length needs to be given using a separate DSP command 0x48, but the game never does that! Instead, it looks like there is a bug in the game code, as it has set the DMA block size to 0x7FF before starting the playing, but the DSP command 0x1C is given like this:
   0417:25DE cmp word [22ED],0200 jbe 25F8 Jump if SoundBlaster DSP version <= 2.00 mov al,1C call 22F5 Send DSP command 0x1C (start 8-bit autoinit DMA) mov ax,[22F1] ax = 0x0800 dec ax ax = 0x07FF call 22F5 Send DSP command 0xFF (undocumented!) mov al,ah call 22F5 Send DSP command 0x07 (undocumented!)0417:25F7 retLooks like the game coders have mistakenly used the same system as when using the DSP command 0x14 for normal 8bit DMA playing, which needs the length of the playing buffer as parameters, first the low byte and then the high byte. DOSBox simply ignores the zero length, same as those 0xFF and 0x07 DSP commands, but all of these caused problems in DS2x86. I hacked the DSP command 0x1C handling in DS2x86 so that if the SB transfer length is zero, the DMA transfer length is used instead. This seemed to help, as the game dropped into debugger for those unsupported 0xFF and 0x07 SB DSP commands, but then started up properly, began to play music and then started up the demo game!

There are 0 comments - Join In and Discuss Here