Programming War Stories: Connectix(aarongiles.com)
aarongiles.com
Programming War Stories: Connectix
https://aarongiles.com/programming/war-connectix/
9 comments
...oh man i just had a déjà vu xD
http://www.crynwr.com/qcpc/
http://www.crynwr.com/qcpc/
Virtual PC was a great piece of software back then in 2003-2006.
You just installed a Virtual PC app. After that, you could install and run nearly any x86 OS right in that app. No hypervisor or SLAT tweaks were required; it all worked literally on any host x86 CPU you can imagine.
Fast forward a few years and Virtual PC was killed off in favor of a new Hyper-V technology from Microsoft. To be precise, Virtual PC was just rebadged as Hyper-V, plus it took some amount of changes.
You immediately felt a substantial difference: instead of an app that always works everywhere, you've got an OS component that should be installed with a lot of ceremony and tricks. Hyper-V was also very picky on CPUs it could run on: it wanted some obscure features (like SLAT) that only the newest and more pricey CPUs could provide at that time. It also had miserable performance when it came to graphics and mouse interactions. A big and disappointing departure from a Virtual PC tiny app that could do it all on every CPU you had at hand.
You just installed a Virtual PC app. After that, you could install and run nearly any x86 OS right in that app. No hypervisor or SLAT tweaks were required; it all worked literally on any host x86 CPU you can imagine.
Fast forward a few years and Virtual PC was killed off in favor of a new Hyper-V technology from Microsoft. To be precise, Virtual PC was just rebadged as Hyper-V, plus it took some amount of changes.
You immediately felt a substantial difference: instead of an app that always works everywhere, you've got an OS component that should be installed with a lot of ceremony and tricks. Hyper-V was also very picky on CPUs it could run on: it wanted some obscure features (like SLAT) that only the newest and more pricey CPUs could provide at that time. It also had miserable performance when it came to graphics and mouse interactions. A big and disappointing departure from a Virtual PC tiny app that could do it all on every CPU you had at hand.
The Hyper-V transition for Virtual PC is loosely covered in the next chapter: https://aarongiles.com/programming/war-microsoft/
How the standalone app originally worked: "In order to begin taking over the machine, we first had to gain access to kernel mode. To do this, we just identified an unused interrupt vector in the OS’s interupt vector table, and patched it to point to our own code—which lived in user mode. (Security experts are rightly freaking out at reading this.)"
How the standalone app originally worked: "In order to begin taking over the machine, we first had to gain access to kernel mode. To do this, we just identified an unused interrupt vector in the OS’s interupt vector table, and patched it to point to our own code—which lived in user mode. (Security experts are rightly freaking out at reading this.)"
Thanks for the link. I worked on a processor simulator a while ago; it brought back memories of getting things running... You can do some really funky things when you implement enough of an MMU. When we showed a demo during one of our executive updates, we showed them that we could run one of our hypervisors inside the simulator and boot to an OS, which was impressive enough at the time. However - through the entire demo, what we didn't mention was that the simulator was actually _itself_ running inside an instance of the simulator, and that the entire demo was really:
A Host laptop (Linux, little endian), running the simulator (of a different processor ISA, big endian), where we'd launched yet another Linux, running the simulator (of yet a different processor ISA), where we'd launched the hypervisor, which was hosting a Linux guest OS which we could SSH into. :-)
At the end of the demo, we exited out of one of the shells, and then showed them that the entire demo was running inside the sim, and quietly proved how far we'd come.
Yes, it was slow, but - it was mathematically correct, and simulating a processor for which the silicon hadn't yet been taped out.
I'm really proud of that team. :-)
A Host laptop (Linux, little endian), running the simulator (of a different processor ISA, big endian), where we'd launched yet another Linux, running the simulator (of yet a different processor ISA), where we'd launched the hypervisor, which was hosting a Linux guest OS which we could SSH into. :-)
At the end of the demo, we exited out of one of the shells, and then showed them that the entire demo was running inside the sim, and quietly proved how far we'd come.
Yes, it was slow, but - it was mathematically correct, and simulating a processor for which the silicon hadn't yet been taped out.
I'm really proud of that team. :-)
I think you're doing Hyper-V a misfavor here. Hyper-V is essentially a server-only product; VirtualBox has had long since taken over as the desktop virtualization of choice. So it's not surprising to see no fuzz about graphics or mouse, the current expierence is just enough to install an OS from which point on you'd want to do anything remotely anyways.
And, for better or for worse, even VirtualBox needs those "shiny advanced features" (that are in just about every non-Atom intel processor since, like, 2006ish) to virtualize x86-64 code, because that's just how the instruction set works; there is nothing that can be done to avoid that. Nobody really wants to run a VM cluster on x86 hardware, not even back then.
Also, these special hardware features are what made virtualization finally fast enough for everything and are one part of what made it possible to virtualize just about any workload you'd imagine today.
And, for better or for worse, even VirtualBox needs those "shiny advanced features" (that are in just about every non-Atom intel processor since, like, 2006ish) to virtualize x86-64 code, because that's just how the instruction set works; there is nothing that can be done to avoid that. Nobody really wants to run a VM cluster on x86 hardware, not even back then.
Also, these special hardware features are what made virtualization finally fast enough for everything and are one part of what made it possible to virtualize just about any workload you'd imagine today.
Hyper-V was indeed designed to work presumably on servers. But the story does not end there.
Let me continue the story.
Windows Phone 8 came out in 2012. It was the version that signified the shift from Silverlight-based runtime to so called WinRT.
How on Earth Virtual PC could be related to Windows Phone? Here is the thing.
Windows Phone Emulator required a working Hyper-V setup. Say, you had no real Windows Phone device, but wanted to take a look at this new shiny thing. Run an emulator!
But wait. Emulator was based on Hyper-V, and Hyper-V required a CPU with SLAT feature. To give you a context, if you had an Intel Core i3 CPU, you were out of luck. You could not run the Windows Phone emulator! The same applied to a lot of other widespread CPU models. I guess only 8% of developers world-wide were lucky enough to have a CPU with a SLAT at that time.
So Microsoft ended in an intricate situation when Hyper-V was originally designed for server use only, but ended up as a critical dependency for an important desktop component (Windows Phone Emulator).
I guess you know where that headed. Just a few developers could take a grasp on Windows Phone Emulator, while the majority of them were out of luck. This factor undermined Windows Phone development. It was essentially a time bomb that led to Windows Phone app starvation in upcoming years, among other factors.
Combined with a Moore law slowdown, SLAT requirement turned out to be a weapon of mass destruction for Windows Phone. People who bought Core i3 in 2012 usually didn't bother to upgrade. So Windows Phone Emulator was much of a dead horse from the start.
--------------------------------------------------
Regarding those fancy CPU features for virtualization assistance. One may think that SLAT was an absolute requirement for Hyper-V, but here is a funny point of anecdata.
In order to workaround that nasty SLAT requirement, some developers resorted to the following scheme on CPUs that din't support SLAT: they installed VMware, installed a guest Windows OS. Then, they enabled nested virtualization in VMware machine settings. Then, they were able to set up Hyper-V in guest OS.
Imagine that, VMware actually emulated SLAT guest CPU on SLAT-less host CPU.
And only then, after all those hoops and wiggles those poor souls were able to run Windows Phone Emulator. What an experience.
Interestingly enough, Windows Phone Emulator was pretty snappy in that double virtualization setup. But the whole experience was ruined.
In a hindsight, that was a writing on the wall for Windows Phone.
Let me continue the story.
Windows Phone 8 came out in 2012. It was the version that signified the shift from Silverlight-based runtime to so called WinRT.
How on Earth Virtual PC could be related to Windows Phone? Here is the thing.
Windows Phone Emulator required a working Hyper-V setup. Say, you had no real Windows Phone device, but wanted to take a look at this new shiny thing. Run an emulator!
But wait. Emulator was based on Hyper-V, and Hyper-V required a CPU with SLAT feature. To give you a context, if you had an Intel Core i3 CPU, you were out of luck. You could not run the Windows Phone emulator! The same applied to a lot of other widespread CPU models. I guess only 8% of developers world-wide were lucky enough to have a CPU with a SLAT at that time.
So Microsoft ended in an intricate situation when Hyper-V was originally designed for server use only, but ended up as a critical dependency for an important desktop component (Windows Phone Emulator).
I guess you know where that headed. Just a few developers could take a grasp on Windows Phone Emulator, while the majority of them were out of luck. This factor undermined Windows Phone development. It was essentially a time bomb that led to Windows Phone app starvation in upcoming years, among other factors.
Combined with a Moore law slowdown, SLAT requirement turned out to be a weapon of mass destruction for Windows Phone. People who bought Core i3 in 2012 usually didn't bother to upgrade. So Windows Phone Emulator was much of a dead horse from the start.
--------------------------------------------------
Regarding those fancy CPU features for virtualization assistance. One may think that SLAT was an absolute requirement for Hyper-V, but here is a funny point of anecdata.
In order to workaround that nasty SLAT requirement, some developers resorted to the following scheme on CPUs that din't support SLAT: they installed VMware, installed a guest Windows OS. Then, they enabled nested virtualization in VMware machine settings. Then, they were able to set up Hyper-V in guest OS.
Imagine that, VMware actually emulated SLAT guest CPU on SLAT-less host CPU.
And only then, after all those hoops and wiggles those poor souls were able to run Windows Phone Emulator. What an experience.
Interestingly enough, Windows Phone Emulator was pretty snappy in that double virtualization setup. But the whole experience was ruined.
In a hindsight, that was a writing on the wall for Windows Phone.
Note that Android's emulator suffers from the same issues due to HAXM requirement changes.
WP died due to multiple reboots on the whole toolchain.
Silverligh/XAML => WP 8 model => WP 8.1 UAP model => WP 10 UWP model
After a while we got fed up of rewriting, hence .NET Standard and the UWP Windows compatibility package, UWP Windows Toolkit.
The emulator had very little to do with it, as many of us would deploy to real devices anyway.
WP died due to multiple reboots on the whole toolchain.
Silverligh/XAML => WP 8 model => WP 8.1 UAP model => WP 10 UWP model
After a while we got fed up of rewriting, hence .NET Standard and the UWP Windows compatibility package, UWP Windows Toolkit.
The emulator had very little to do with it, as many of us would deploy to real devices anyway.
Android emulator originally came as qemu emulating arm CPU. The Intel build with HAXM, was added later, as the accelerated option.
HAXM has another downside: it doesn't run on AMD CPUs. If you wanted accelerated emulator on AMD CPU, you could use the Linux version with KVM or Mac version with Hypervisor.framework up until few months ago, when the emulator started to support the Microsoft Hypervisor Framework.
HAXM has another downside: it doesn't run on AMD CPUs. If you wanted accelerated emulator on AMD CPU, you could use the Linux version with KVM or Mac version with Hypervisor.framework up until few months ago, when the emulator started to support the Microsoft Hypervisor Framework.
Yep and it sucked much more than WP ever did, only after everyone started ignoring it while making use of Genymotion or Microsoft's own implementation, did Google care to actually improve it.
So bad emulator support wasn't an issue regarding Android's adoption.
So bad emulator support wasn't an issue regarding Android's adoption.
But it may have some affect on app development. Even today most apps are iOS first.
Using the iOS simulator compared to the Android emulator is like night and day.
When you run your app on the iOS simulator, your app is compiled to x86 code and linked against an x86 build of the iOS frameworks - no emulation involved. The code —> build -> test cycle is so much faster with iOS.
With classic Windows Mobile with the .Net compact framework, you could do a lot of your testing just running your app on your desktop without ever using the emulator.
Using the iOS simulator compared to the Android emulator is like night and day.
When you run your app on the iOS simulator, your app is compiled to x86 code and linked against an x86 build of the iOS frameworks - no emulation involved. The code —> build -> test cycle is so much faster with iOS.
With classic Windows Mobile with the .Net compact framework, you could do a lot of your testing just running your app on your desktop without ever using the emulator.
Android emulator runs native Intel code, just like iOS emulator, since 2013. That's what the HAXM thing is about. Anyone who didn't notice that during the last 5 years or so, probably doesn't do much Android development anyway.
Also, I will here agree with pjmlp, having real device for development and testing with that real device will beat any emulator, native or not.
Also, I will here agree with pjmlp, having real device for development and testing with that real device will beat any emulator, native or not.
Yes it does but an emulator is good for negative testing - if it doesn’t work on the emulator it probably won’t work on the device and if it does. You should still be worried. It’s not great for positive testing.
iOS still shines fit on device testing - you can conceivable have a representative sample of all iOS devices that your app should run on. I would probably want to have an SE, a 7s, and 8 plus and a X on the iPhone side and be okay with just one 9.7 inch iPad.
iOS still shines fit on device testing - you can conceivable have a representative sample of all iOS devices that your app should run on. I would probably want to have an SE, a 7s, and 8 plus and a X on the iPhone side and be okay with just one 9.7 inch iPad.
Both working device and emulator are equally important. People usually divide in two camps depending on their role in the project: device lovers and emulator lovers.
Solo developers, product leads and testers tend to choose the real device as their first destination. Linear coders and logic writers usually choose emulator.
Add a device variety testing. It's very hard to do right without emulator. Having all those device variations in physical form may be expensive and consume a lot of space.
Solo developers, product leads and testers tend to choose the real device as their first destination. Linear coders and logic writers usually choose emulator.
Add a device variety testing. It's very hard to do right without emulator. Having all those device variations in physical form may be expensive and consume a lot of space.
Anecdote:
My first job as a mobile developer was writing Windows Mobile apps for ruggedized devices. We had to test our apps outside with gloves on to make sure that the interface was usable by the field techs in the middle of the winter even if they lost their stylus.
My first job as a mobile developer was writing Windows Mobile apps for ruggedized devices. We had to test our apps outside with gloves on to make sure that the interface was usable by the field techs in the middle of the winter even if they lost their stylus.
In US and other iOS centric countries yes, but good luck getting users across the globe on regions where Android rules like Europe, Africa, South America and Asia.
Also getting Mac hardware in some of those regions is quite expensive endeavor versus the average salary, if there are any stores at all selling them.
Also getting Mac hardware in some of those regions is quite expensive endeavor versus the average salary, if there are any stores at all selling them.
No doubt a legacy from when Virtual PC had to be that way - it worked on PPC Macs:
https://macvm.com/virtual-pc/
https://macvm.com/virtual-pc/
My favorite is how they managed to get Connectix Virtual from 1989 to support the Macintosh IIci. Even https://tidbits.com/1994/07/25/transition-to-powerpc-ram-dou... mentions that "I want to thank the PowerPC software team in particular for spending two hours with us in March thumbing through the source code answering specific questions were stuck on at that crucial stage of the project. "
Connectix Virtual Game Station was a GREAT piece of software.
I played tons of PSX games with it, it was simple and did a great job.
I played tons of PSX games with it, it was simple and did a great job.
The Gaming Historian did a video on PS1 emulators, including Virtual Game Station: https://youtu.be/UGHul1PrXCE
I never tried Connectix's PS emulator, but I do remember that era and purchasing one of the weird Bleem CD keys for a product that took forever to come out. Eventually ePSXe became the dominant emulator. I remember playing and beating FF9 on it.
I'm really glad he worked for a company that said, "Go for it," and in doing so, set a pretty important legal prescient.
It's not the first time a company has reversed engineered a BIOS. That's how we got some of the first IBM PC Compatibles. Compaq did the same thing, except they had two teams. One that reverse engineered a BIOS and wrote a spec, and another that wrote the new BIOS, giving them a clean-room design they could defend legally.
I'm really glad he worked for a company that said, "Go for it," and in doing so, set a pretty important legal prescient.
It's not the first time a company has reversed engineered a BIOS. That's how we got some of the first IBM PC Compatibles. Compaq did the same thing, except they had two teams. One that reverse engineered a BIOS and wrote a spec, and another that wrote the new BIOS, giving them a clean-room design they could defend legally.
I bought two products from Connectix that made up for the sorry state of MacOS at the time. RAM Doubler that helped work around how Mac allocated memory and Speed Doubler, a better 68K emulator for PPC Macs.
Speed Doubler was fantastic on 68K Macs as well. It had a smarter disk cache and added multithreading to the Finder for file copies and trash emptying long before MacOS 8
I was an early adopter of Virtual PC on OSX. I have to say that it never worked to a performance level of being useful even for business needs like running basic business apps like Excel, Outlook, Visio, Ecco Pro and the like.
I abandoned the idea of VirtualPC once Microsoft bought connectix as they were still the EvilCorp of the time.
I did not see any useful PC virtualization on OSX until Apple switched to Intel and VMware jumped into the game.
I abandoned the idea of VirtualPC once Microsoft bought connectix as they were still the EvilCorp of the time.
I did not see any useful PC virtualization on OSX until Apple switched to Intel and VMware jumped into the game.
And that’s why I had the DOS Compatibility Card for the PowerMac in the pre OS X days.
Virtual PC for PPC was a work of art. Some early version had Voodoo1/2 passthru which managed to eke out several frames per second in a game like Counterstrike. Although the emulator was good enough to let me play the network install of Starcraft at a LAN party on my souped-up PowerMac.
https://www.google.com/search?client=ubuntu&channel=fs&q=con...