Linked by Thom Holwerda on Mon 17th Sep 2012 16:56 UTC, submitted by Andy McLaughlin
OSNews, Generic OSes "Visopsys (VISual OPerating SYStem) is an alternative operating system for PC-compatible computers, developed almost exclusively by one person, Andy McLaughlin, since its inception in 1997. Andy is a 30-something programmer from Canada, who, via Boston and San Jose ended up in London, UK, where he spends much of his spare time developing Visopsys. We had the great fortune to catch up with Andy via email and ask him questions about Visopsys, why he started the project in the first place, and where is it going in the future."
Thread beginning with comment 535487
To read all comments associated with this story, please click here.
The hardest part
by Alfman on Mon 17th Sep 2012 17:51 UTC
Alfman
Member since:
2011-01-28

Pingdom: "What’s been the hardest part of working on Visopsys?"

Andy: "Hardware support, definitely. The world of PC hardware has always been a bit daunting, since there are so many different manufacturers, all interpreting the standards in their own ‘unique’ ways. You might write something like an IDE driver that works on every system you try, release it, and then find out it isn’t working properly on hundreds of other peoples’ systems."


I have to agree completely with his answer. Designing and implementing the OS foundation is the "easy" part. It's the part we're all most interested in as programmers. Going through and actually implementing the drivers (often without specifications and even without all hardware configurations) is the part that ultimately bogs down the hobby OS developer.


What we need is some kind of universal driver standard that can be shared across all operating systems. Ideally this would be in source form and the layer could be optimised away by the compiler. This way a driver wouldn't be written for "Windows X" but instead for the "2012 PC driver standard". The OS would implement the standard and immediately support numerous compatible hardware devices. It's a pipe dream though. For it's part, MS would never participate, and their cooperation would be pretty much mandatory.

Reply Score: 3

RE: The hardest part
by ferrels on Mon 17th Sep 2012 18:01 in reply to "The hardest part"
ferrels Member since:
2006-08-15

That all sounds good in theory, but in practice it reduces your OS to running on hardware of the least common denominator (in terms of performance)....your driver may work on 98%+ of the hardware out there but the performance suffers immensely. VESA drivers for video are a good example of this. Almost all video hardware will work in VESA mode, but you're essentially left with only the most basic functions and features....i.e. no hardware GPU acceleration, limited color depths and resolutions, etc.

Edited 2012-09-17 18:02 UTC

Reply Parent Score: 2

RE[2]: The hardest part
by Alfman on Mon 17th Sep 2012 18:13 in reply to "RE: The hardest part"
Alfman Member since:
2011-01-28

ferrels,

Well, VESA was written ages ago and at that time they performed pretty well because graphics were not accelerated.

The reason I stated the "2012 PC driver standard" was because I envisioned the standard itself to be updated every few years to adopt to new hardware interfaces. Non-standard extensions could be implemented too, but the idea would be for new functionality to ultimately be incorporated into the standard at some point.


The great thing about this is that the standard could be both forward and backward compatible.

As an example. an OS might support the 2012 standard for webcams. Come 2020, when the standard is depreciated, a generic 2012->2020 wrapper layer could never the less assure that all 2020 operating systems could continue to run the 2012 web cam drivers. (I think I uncovered another disincentive for manufacturers to support this ;) )

Conversely, it'd be possible to have a generic 2020->2012 conversion driver to allow an older OS to run the newer hardware drivers.


Edit: Extending EOL is only a side benefit, but the intention would be to eliminate the duplication of work in OS drivers and make it much easier for all operating systems to support all hardware at least in their basic modes. More advanced features should still be possible even if they're not supported in all operating systems.

Edited 2012-09-17 18:19 UTC

Reply Parent Score: 2

RE: The hardest part
by tidux on Mon 17th Sep 2012 18:12 in reply to "The hardest part"
tidux Member since:
2011-08-13

That's what the BIOS does. Welcome to 16-bit mode.

Reply Parent Score: 4

RE[2]: The hardest part
by Alfman on Mon 17th Sep 2012 19:07 in reply to "RE: The hardest part"
Alfman Member since:
2011-01-28

Of course, BIOS is strictly a legacy interface today, essentially undeveloped since the 1980's. Never the less, in it's prime, consider what a huge phenomenal success BIOS was in achieving a level of hardware independence that would have been impossible without it.

The small OS I wrote did run on PCs other than mine. I didn't have to do anything extra to make it run on a laptop, it just did because those standards existed. I am absolutely positive that Andy and all other indy-OS devs you'll find will agree that they crave a standard interface that would just make their OS work everywhere without having to reinvent the wheel and write new drivers for the Nth time.

Let's all have a good laugh comparing the idea to a 16 bit BIOS, but on a serious level I'd rather not dismiss the notion of a modernised standard as a joke. It would be tremendously useful in promoting innovation in the operating system space by making it much easier for alternative operating systems to be taken seriously as competitive platforms.

Reply Parent Score: 3

RE: The hardest part
by Brendan on Tue 18th Sep 2012 04:12 in reply to "The hardest part"
Brendan Member since:
2005-11-16

What we need is some kind of universal driver standard that can be shared across all operating systems. Ideally this would be in source form and the layer could be optimised away by the compiler. This way a driver wouldn't be written for "Windows X" but instead for the "2012 PC driver standard". The OS would implement the standard and immediately support numerous compatible hardware devices. It's a pipe dream though. For it's part, MS would never participate, and their cooperation would be pretty much mandatory.


This has been attempted several times before. The most popular attempt was UDI (Uniform Driver Interface), which failed despite being backed by several large companies (including Sun/Solaris).

Some OSs (e.g. Linux) had religious objections ("OMG what if we wrote drivers and Microsoft could use them!"), some OSs had security problems ("OMG binary blobs created by unknown third-party developers running at the highest privilege level because our kernel is monolithic!"), and some OSs had technical reasons for not using it (e.g. very different driver interfaces, capabilities and feature sets). The end result was that very few people wrote drivers for it because most OSs didn't support it (and then OSs that didn't have religious, security or technical reasons for avoiding it didn't bother supporting it anyway because there weren't many drivers).

The ancient BIOS services are not acceptable because they're single-tasking synchronous interfaces (e.g. your OS freezes while the firmware waits until a DMA transfer completes). For exactly the same reason, UEFI is not an acceptable answer either. For a decent OS, you want to be able to be transferring data to/from disk while transferring data to/from network while transferring data to/from sound card while transferring data to/from USB; where the CPUs are all still free to do unrelated/useful work (and aren't stuck in a "while(waiting) {}" loop).

- Brendan

Reply Parent Score: 2

RE[2]: The hardest part
by Alfman on Tue 18th Sep 2012 05:30 in reply to "RE: The hardest part"
Alfman Member since:
2011-01-28

Brendan,

Those are great points. I'm not under any delusions that it would be easy to get everyone on board the shared driver model. Even if we managed to solve all the technical issues, I think many corporations would reject it on account of them benefiting from high barriers to entry in the field of alternative operating systems. So I'm really very sceptical myself that we could pull it off in this day and age when computers are only getting more closed rather than more open.


Never the less, I still like dissecting the theory and appreciate your critique.


1. "Some OSs (e.g. Linux) had religious objections ('OMG what if we wrote drivers and Microsoft could use them!')"

Well, presumably the manufacturers would be on board (if they weren't the model would be bound to fail anyways). This means the manufacturers would be providing the drivers and that they would run on linux, visopsys, hurd, etc. as well as windows.


2. "some OSs had security problems ('OMG binary blobs created by unknown third-party developers running at the highest privilege level because our kernel is monolithic!')"

This is true. In principal I don't object to requiring source code with drivers, but that's not a very realistic sale. Many linux distros already have proprietary blobs, and nobody would be forced to install those. On the one hand, the high availability of shared manufacturer drivers could decrease the incentive of volunteers to write open source drivers for the same hardware. On the other hand, these volunteers could put their time to much better uses instead of reinventing the wheel.

Ultimately though if you don't trust the manufacturer of your hardware to write stable/trustworthly software, then arguably you've got no business installing their hardware on your machine either.


3. "and some OSs had technical reasons for not using it (e.g. very different driver interfaces, capabilities and feature sets)."

This is indeed probably one of the more controversial aspects, but the way I see it the drivers should be as modular as possible to make it easy to hook them into however the OS needs them. At the extreme, all operating systems today have no choice but to work with fixed interfaces anyways (ones provided by the hardware). Moving this into software shouldn't be that much of a burden to a system's design. Like I said earlier, *ideally* this wrapping layer would be in a kind of source form and it's overhead could be optimised away when it's compiled&linked with callees within the OS.


"The ancient BIOS services are not acceptable because they're single-tasking synchronous interfaces (e.g. your OS freezes while the firmware waits until a DMA transfer completes)."

Great observation. Clearly these would have to be asynchronous. Also, all device drivers should be able to run in parallel on SMP. Which is a reasonable requirement assuming they don't share resources.

Just a note: many linux drivers themselves were subject to the big kernel lock until recently, which caused similar driver serialisation bottlenecks.
http://lwn.net/Articles/380174/


I wish there would be a viable market for this, it is a project I think I would enjoy working on.

Edited 2012-09-18 05:38 UTC

Reply Parent Score: 4

RE: The hardest part
by aargh on Tue 18th Sep 2012 07:28 in reply to "The hardest part"
aargh Member since:
2009-10-12

What we need is some kind of universal driver standard that can be shared across all operating systems.


You mean like Genode? Now, why hasn't someone already thought of that...

Reply Parent Score: 1

RE[2]: The hardest part
by Laurence on Tue 18th Sep 2012 08:16 in reply to "RE: The hardest part"
Laurence Member since:
2007-03-26

"What we need is some kind of universal driver standard that can be shared across all operating systems.


You mean like Genode? Now, why hasn't someone already thought of that...
"


Genode isn't really a universal driver standard though.

In fact, I'm still not 100% certain what Genode is setting out to achieve even after (loosely) following the project for a few years. but from what I can gather, it's a complete micro kernel architecture design to allow running guest kernels (such as Linux) and thus their drivers.

Reply Parent Score: 2

RE: The hardest part
by Laurence on Tue 18th Sep 2012 08:03 in reply to "The hardest part"
Laurence Member since:
2007-03-26

That's a great idea in theory, but wouldn't it push kernels into a more hybrid or even -dare I say it- micro kernel design?

Without wanting to get into a debate about micro vs monolithic kernels; if a unified driver format meant constraints on the design then I could see a lot of potential advocates turning their back on said driver format. Particularly those who have a monolithic kernel (Linux being the biggest loss).

I guess, even if the above supposition was correct (and there's a very good chance I'm talking complete BS here lol), then at least a unified format could offer "fall back" drivers for occasions when optimised / native kernel drivers are not available (much like the aforementioned VESA mode - which is invaluable for setting up servers)

Reply Parent Score: 2

RE[2]: The hardest part
by Alfman on Tue 18th Sep 2012 16:03 in reply to "RE: The hardest part"
Alfman Member since:
2011-01-28

Laurence,

"That's a great idea in theory, but wouldn't it push kernels into a more hybrid or even -dare I say it- micro kernel design?"

I don't think so, but it's worth investigating. Linux can call VESA or UEFI without becoming more hybrid (note that's not the exact model I'm proposing per-say, but I never the less think it's a valid counter-example). Actually my gut instinct is to say the opposite may be more of a concern, how would a microkernel incorporate these drivers?

Obviously the microkernel's goal is to isolate the drivers from one another, would it be able to jail the drivers and still have them work? That depends how they're written. The standard would have to be very clear about how drivers could interact with the system, no direct manipulation of GDT or interrupt tables, drivers would need to request permission to access ports instead of assuming they're running in ring-0. They'd need standard ways to coordinate memory mapping. These murky details all need to be ironed out for sure, but with a well defined standard, a good reference implementation, a robust test suite, and a certification process, then we should have quality drivers that work everywhere without worrying about OS-specific quirks. I don't think an existing operating systems would need too many changes (assuming it's drivers were already modular and self-contained). It wouldn't be too different from writing a new OS-specific driver for a new piece of hardware, only this particular OS-specific driver will be capable of driving all hardware supported by the shared driver standard.

Edited 2012-09-18 16:07 UTC

Reply Parent Score: 2

RE: The hardest part
by Vanders on Tue 18th Sep 2012 10:44 in reply to "The hardest part"
Vanders Member since:
2005-07-06

...all interpreting the standards in their own ‘unique’ ways. You might write something like an IDE driver that works on every system you try, release it, and then find out it isn’t working properly on hundreds of other peoples’ systems


IDE? All. My. Hate.

I often said that it appears every single manufacturer of an ATA controller took the specification, had it translated into Japanese by a 13 year old Spanish teenager and then handed it to a Brazilian programmer to read out loud to a Polish developer as he tried to implement it. I have no other explanation for how hardware designers could all come to such odd interpretations of something that was supposedly a written standard.

Reply Parent Score: 2

Beautiful.
by gus3 on Tue 18th Sep 2012 20:17 in reply to "RE: The hardest part"
gus3 Member since:
2010-09-02

My friend, this belongs in the Fortunes database.

Reply Parent Score: 1

RE: The hardest part
by demetrioussharpe on Wed 19th Sep 2012 00:10 in reply to "The hardest part"
demetrioussharpe Member since:
2009-01-09

What we need is some kind of universal driver standard that can be shared across all operating systems. Ideally this would be in source form and the layer could be optimised away by the compiler. This way a driver wouldn't be written for "Windows X" but instead for the "2012 PC driver standard". The OS would implement the standard and immediately support numerous compatible hardware devices. It's a pipe dream though. For it's part, MS would never participate, and their cooperation would be pretty much mandatory.


This has already been tried; it's called UDI (Uniform Driver Interface). Unfortunately, it's exceptionally hard to get everyone on board with such an effort. The open source community largely ignored it, because it allowed the proliferation of binary drivers to continue. None of the other OS vendors had any incentive to participate in it because they don't have a problem getting manufacturers to create drivers for them. It's a shame, because there's also a reference implementation available, but no one seems to care. Perhaps all of the hobby OS's should work to create drivers for this API to allow driver sharing. If you'd like to check UDI out, here're a few links:

http://en.wikipedia.org/wiki/Uniform_Driver_Interface
http://www.projectudi.org/
http://projectudi.sourceforge.net/

Reply Parent Score: 1

RE[2]: The hardest part
by Alfman on Wed 19th Sep 2012 00:45 in reply to "RE: The hardest part"
Alfman Member since:
2011-01-28

demetrioussharpe,

Thank you for the links. UDI had been mentioned already, and as far as I can tell it fell into obscurity a decade ago for the political reasons that have already been mentioned. It was a good idea but I don't think it wasn't a complete solution either, only targeting system devices like network and block devices. I couldn't find anything in UDI for webcams, scanners, or even mice.

To those of you who may be questioning why bother talking about this when the chance of adoption is next to none, well...I'm an os-dever at heart, I fantasize about how things should be. I suspect it's the same thing that drives people like Visopsys's creator.

Reply Parent Score: 2