DS2x86 progress

April 9th, 2012, 22:51 Posted By: wraggster

Sorry, no new version released today. I have been working on Windows 3.11 support so extensively, that I have not had time to do any other improvements. I did check Alien Legacy, and found out that it uses CON device (stdin) input when selecting the sound support (in the beginning of the game). It reads the user selection with a C-language equivalent of fscanf(stdin, "%d", &value). I do not have proper support for stdin buffered input via file handle yet in DS2x86 (nor in DSx86). I tried to quickly code a hack for that, but it did not help with the problem, so I moved back to working on Windows 3.11.
The Windows 3.11 work has progressed slowly, as I keep running into various difficult problems that take several days to solve. The first weird problem was that after going to VM86 mode and executing a real-mode interrupt call in VM mode, the code returned to inside a string in emulated BIOS area! That is, the IRET opcode that pops the return address from stack, popped a real-mode address F4E4:1637, which was inside the BIOS ROM area.
I began looking for a problem in my code, tracing backwards for code that sets those invalid values to variables later used as the return address, until I found code that begins scanning from the end of the BIOS area downwards, until it finds a lower case 'c' letter, and then it stores the address of this letter as the return address for all DOS interrupt calls! I didn't think this made any sense whatsoever, so I began debugging the Windows 3.11 behaviour in DOSBox. After a while I realized that it behaves exactly the same way also in DOSBox! It still took me a while to believe that this was indeed the correct behaviour. Windows 3.11 returns from VM mode back to proper protected mode using a General Protection Fault exception. This fault is generated by trying to execute a protected mode ARPL opcode in VM mode. This opcode has a hex value of 0x63, the same as the ASCII letter 'c'. So, Windows just saves some code space by executing the faulting opcode directly from BIOS. I added a GPF generation code to that opcode, and then finally managed to progress a bit further.
The next problem was that I got a BSOD exception in DS2x86, and the exception adderss pointed to within my emulated file routines. After some thinking it occurred to me that with paging on, the page tables I use in my file routines might not have been properly set up. I haven't had to check this before adding VM mode, as the DOS file routines could only be used in real mode with paging off. Now Windows wants to call them with paging on in VM mode, so I needed to add a much more complex page table address calculation to every DOS routine that needs a memory address (which is pretty much all of them). This will cause a slight slowdown to every DOS operation, but since those are very slow compared to the actual CPU emulation in any case, this is probably pretty much unnoticeable. It is a major architectural change, though, so it is possible that some game will get broken because of this change.
After fixing the BSOD problem, I again felt I managed to get good progress done, as I ran into several opcodes that needed VM mode enhancements, like GPF generation for PUSHF and STI opcodes when the current I/O protection level did not allow the VM-mode program to access the CPU flags without the protected mode monitor intervening. This work progressed nicely, until Windows simply dropped back to DOS with a message "Video initialization failed".
This problem was again somewhat more difficult to track down. Windows 3.11 reads all the VGA register values into it's memory during the logo screen display (which is in 640x480 16-color mode). It then goes to the EGA-compatible 640x350 16-color mode, and then to the normal text mode. What was weird, was that after going to the text mode it checked that the register values it had saved when in 640x480 graphics mode were for a text mode! This again seemed to make no sense, so again I needed to debug the behaviour in DOSBox. In DOSBox the values saved to memory were indeed for text mode. I then added a watch for the memory address that seemed to change somewhere between storing and later reading the value. I noticed that the memory values changed when DOSBox was executing the INT 10 mode change routine (the C-language routine within the DOSBox sources, not in the emulated machine code)! This was pretty strange, but I finally found out that DOSBox detects if the INT 10 code is running in VM mode, and actually launches a separate emulation core (called IOFaultCore) for every port access (of which there are several dozen when changing the video mode)! This separate core then causes a GPF and calls the Windows 3.11 trap routine, which then checks that the VGA registers are getting changed and stores the new values into memory areas.
In DS2x86 and DSx86 I have not paid much attention to the VGA registers when changing the modes. My mode change C routine simply copied the new mode-specific register values to the memory addresses that the emulated I/O port routines used, in case a program wants to read the VGA ports after the mode change. I had used simple memcpy() calls, which obviously will not work at all for the Windows needs.
So, I spent a day changing the DS2x86 mode change routine and INT 10 emulation so that after calling the mode change C routine, I execute within the emulated BIOS area a x86 routine that outputs the proper mode-specific VGA CRTC and Graphics register values, so that Windows can trap each of these access and handle them as it wishes. I actually first thought that I could simply read the port value and then write it back, so that I don't need to actually access the VideoParameterTable, but that did not work. Windows traps also the VGA port read routines, and replaces the actually read value with the value from it's memory table! I believe this is how it virtualizes the screen access so that it can prohibit a game from changing the graphics mode for real, but still make the game think that the graphics mode has been changed. When running such code within DS2x86, there is sort of two layers of emulation on top of each other!
After I finally got Windows to think that it is running on 100% VGA compatible display adapter, I had a simple "Unsupported INT 15 call!" problem (Windows tried to detect the BIOS mouse device type), but after implementing that, Windows startup simply drops back to DOS with no messages. This is again a bit more difficult to find and fix, and this is what I am currently doing. I still have various missing features in the VM mode support, which may be the cause, but I still need to spend some time debugging this problem and comparing the behaviour of DS2x86 with that of DOSBox.
Have a Happy Easter, and I hope you can wait for a while longer for the next version. It has been interesting learning about Windows internals while attempting to make it run in DS2x86, so I plan to continue with that for a while longer, at least.

http://dsx86.patrickaalto.com/DSblog.html

There are 0 comments - Join In and Discuss Here