Submitted by BlackBerry QNX is an embedded operating system targeting applications in the automotive, general embedded, and medical markets. However, it is not your garden-variety embedded OS: QNX is a full-blown, UNIX-like, POSIX-compliant operating system with all of the features you would expect of a desktop or server-class OS. Compatibility with other systems means that, at least in theory, porting various open source projects to SDP 7 should be a relatively easy task. And so, while there is no official support in this release for a desktop environment, there is nothing precluding someone from building such a system. With that in mind, I set myself the task of building a BlackBerry QNX 7 desktop.
Written by QNX kernel developer Elad Lahav, so you know the information in this article is solid.
If I remember correctly QNX used to ship with a window manager ages ago – it was called Photon. Quite fast and powerful even without any hardware acceleration. What happened to it?
I miss playing around in Photon, it was actually quite advanced for its time and for being mostly a dogfooding experiment. I think the last time I played with it was in the QNX Momentics 6 release some time in the early/mid 2000s. I had it booting with full hardware support on a PIII based Acer machine, that also coincidentally ran BeOS R5 with all hardware working.
Yes, I won a copy of it on CD many years ago (Version 4 I think it was) It was really fast and great looking, There was also a demo on a single floppy that booted to the desktop and had a browser and some other software all on the one disk.
I seem to recall it was modular so you only installed the parts you needed making it small and fast with no bloat.
If they positioned an updated version in the market to compete with current stuff I would certainly be interested and I am sure many other folk would be too.
QNX completely gave up on desktops several years ago when they replaced Photon with Screen, but even before that, the non-commercial versions had been increasingly crippled. The only QNX versions that had full non-commercial distributions were 6.0 and 6.1 (which both came out in 2001). I think QNX’s proprietary license has held it back considerably in many ways.
I wonder how things would have turned out if QNX had been a research OS under a free (or at least semi-free) license. It might have stopped Mach from single-handedly destroying the reputation of microkernels. We might all be using microkernel OSes had QNX been free like Mach was.
I’m going to try to fix the lack of a free QNX-like OS (no, Minix 3 and GNU/Hurd don’t count as far as I’m concerned) by writing one myself. I have much of the design already planned out and have been thinking about the best ways to do things for quite a few years now. My OS will be called UX/RT and while it won’t be a clone of QNX at all, it will have a broadly similar architecture. I’m not cloning QNX because I want to improve on it in some ways (among other differences from QNX, UX/RT will have a fully unified filesystem-based IPC API, and will have support for process-specific namespaces like those of Plan 9). Linux binary compatibility will be a top priority, and that should mostly solve the problem of application support. Driver support will also mostly be taken care of by Linux compatibility (there are already a couple of “Linux kernel as a library” projects that should be possible to use to provide device, network, and disk filesystem drivers). Hopefully the Linux application and driver support should make it at least somewhat successful as an alternative OS.
Edited 2017-04-29 12:42 UTC
I would like to hear more about this, Do you have a website or anything to follow ?
I did create a Sourceforge site several years ago but I haven’t uploaded much to it and haven’t made a lot of progress since then (and changed my plans some since then as well).
I really should create a GitLab/GitHub repository. Currently I just have an in-memory bootloader although I’ve almost gotten it into a useful state (I did have it working a while ago but I decided to change the boot protocol since it wasn’t extensible enough). Next thing to do after that is to patch seL4 to support QNX-like booting from a memory filesystem image as well as an early video console.
Plan9 type local namespaces is a straight-forward extension of the current namespace management of QNX, just add more namespace roots.
I don’t know what “fully unified filesystem-based IPC API” means however the IPC design of QNX Neutrino is strongly connected to the filesystem management. Essentially an IPC channel can be viewed as a handle to a file. Don’t know if extending that IPC-filesystem connection is wise as the IPC system have to be as fast as possible.
Retrofitting Linux drivers into a microkernel isn’t likely to lead to something good IMHO – the driver models differ too much.
BTW there once was a project to combine ideas from both QNX and Plan9, it could perhaps be of interest?
https://en.wikipedia.org/wiki/VSTa
Yeah, I’ve experimented with VSTa a little and it’s the most QNX-like free OS but it’s still not quite what I want. For one, it implements processes in the kernel, whereas UX/RT will be based on seL4 and (similar to QNX) will have a process server that also implements filesystem namespace management (although unlike QNX the process server will be a separate binary rather than being built into the kernel). Also, it’s written in C, whereas UX/RT will use Rust for new code (although there will still be a lot of C code borrowed from Linux and BSD of course).
Which entity checks permissions has always been an interesting topic in the design of microkernels. As you mention, in QNX this is mostly done by the server, though QNX 7 also has channel abilities to prevent you from connecting to particular servers.
The alternative, which you suggest, has the major disadvantage that another entity (either the kernel or an external path manager) needs to mirror all nodes provided by all servers (and, in particular, all files and directories in all file systems) in order to enforce permission checks (which also makes it extremely hard to dynamically mount and unmount remote file systems). In QNX, the path manager is oblivious of any nodes other than the mount points of the various servers.
Neither solution is perfect…
Writing a new OS with a slick, clean, design that can output “Hello World” over a serial connection is not hard. Once you get there, however, you are faced with two problems:
1. Lack of drivers
2. Lack of applications
(And that is assuming you are writing your own file system, network stack and display manager, which are not trivial).
Unless you are Microsoft, Google or Apple, you cannot get hardware vendors to write drivers for your system, or application vendors to port their software. Even BlackBerry at its heyday had trouble with that, and QNX today, even though quite popular in certain segments with millions of deployments, is facing this challenge.
At that point, as a small OS vendor, your only recourse is to make your system compatible with other systems, at which point your slick, clean design breaks apart (or becomes horribly inefficient, which is what happened to most microkernel systems).
elahav,
As someone’s who’s been there, I completely agree with everything you said. It’s only gotten worse since the times when I was doing my own kernel since these days you’ve got more proprietary chipsets than ever. It’s not for lack of talent or effort, just the lack of mindshare. I was quite passionate about OS development back in the day, but I had to move on. I thought (and still think) there are some things I could do better than something like linux, but indy projects really don’t have any widespread viability due to the issues you bring up.
Consolidation has annihilated independent operating systems, the world doesn’t need or want more operating systems even if developers can build them.
Edited 2017-04-30 12:21 UTC
Lets see what Google brings to the table with Fuchsia
Applications are hard to write, sure. However that can be (mostly) solved by being compatible enough to some Linux setup. There are a number of popular UI and system abstraction layers that makes porting software easier but even those require some emulation of other systems.
Drivers can be “solved” by limiting hardware support. The base system is “easy”, USB, PS/2, PCIe, SATA etc. can be written as if systems are actually compatible with documented (and semi-documented) interfaces and then be patched when a system is detected that have some quirk. Graphics drivers and network drivers are the two big problem areas IMHO.
That’s true in theory, but fails in practice. The reason is that many applications are developed to observed behaviour rather than standards. A few examples from my own experience:
1. Linux over-commits memory. Consequently, many applications start by requesting huge amounts of memory. There is no problem on Linux so long as that memory is not used (at which point the application is killed with a SIGBUS). QNX has a strict no-over-commit policy for safety reasons, which means that these apps fail to initialize.
2. Skype bypasses the C library (or is statically linked, can’t tell from the binary blob they ship), which means that it uses hard-coded kernel-call numbers. Implementing Linux compatibility in the C library doesn’t help.
3. Many applications are buggy, but these bugs may be hidden on Linux for various reasons (e.g., different memory layout, Linux mutexes not checking for errors). I remember contacting the developers of a particular application that was writing outside of buffer bounds, but they refused to fix their code as it works on Android.
As a Linux user since 1999 (Red Hat 5.0) I really appreciate the irony of vendors content with their code running on Linux only.
Yes, I didn’t mean to suggest that porting “foreign” applications would be easy, just that is possible to get a reasonable subset of programs even into a smaller OS. As an example a web browser, a text editor and a compiler covers most of my use cases. Add RDP and/or VNC and the things I couldn’t do would be reduced to watching movies, play games or watching demoscene demos.
UX/RT will implement both push and pull APIs for exporting directories to the process server. There will also be a hook to allow external processes to be notified when a directory is accessed to allow for automounting. UX/RT will use in-memory per-process ACLs (with the possibility of specifying fallback to filesystem permissions for groups of files) rather than the traditional Unix all-or-nothing model, so centralizing security in the kernel and process server will probably be the best way to do things.
Edited 2017-05-01 00:06 UTC
I guess you are already aware of the genode-projekt. Just in case:
http://genode.org
They have done some pretty work to encapsulate linux drivers to be used as user-level citizens on microkernels.
But in my opinion the rump-kernel/any-kernel (striped down user-level-bsd-kernels) project offers a better solution. It is also used by genode for filesystems … but you could provide almost anything from audio over network to display by utilizing rump-kernels.
Advantage: since rump-kernel is a integrated part of NetBSD any new hardware supported by NetBSD will be automatically available as rump-kernel. And NetBSD has a much more stable driver-api than Linux does.
http://rumpkernel.org
hope to hear more from your project soon!
As stated in the article, QNX 7 is not backwards-compatible with earlier versions, so I guess that explains why he can’t use Photon out-of-the-box. As for why Photon hasn’t been further developed to run on QNX 7, I guess the answer is that the Photon widget library has been supplanted by Qt, and as the author mentions most embedded applications are fullscreen so there is no need for Photon’s windowing capability.
This gets me thinking: It would be pretty cool to see a lightweight, fully cross-platform desktop environment based on “pure” Qt without the massive amount of dependencies KDE brings. In addition to serving platforms like QNX it could also be a hit on platforms like the Raspberry Pi or any other platforms that are “essentially embedded but desktop-capable”.
Edited 2017-04-29 11:06 UTC
Something along the lines of LXQt http://lxqt.org/about/ ? (which is not fully cross platform but is becoming available in more Linux repos for the Pi e.g. https://packages.debian.org/search?suite=all&arch=armhf&searchon=nam… )
Hey thanks, that really does look cool!! I’d of course heard of LXDE but wasn’t really actively following its development and totally missed that they made a Qt port. I wonder if this might be usable on QNX. Going to write a comment on the article right now.
Porting LXQt was my first thought as well. Unfortunately, the code is heavily dependent on the X protocol, so that was a dead end.
Wayland looks more promising, as the protocol seems to be better separated from the display manager, and can hopefully be ported to work with QNX Screen.
Besides LXQt there’s Lumina.
https://lumina-desktop.org/
Please make this happen!!!!!
I wondered how they did provide backwards compatibility while extending the system to 64 bit, there were a number of quirks that would be hard to extend. Not doing it at all explains that.
If QNX hadn’t decided to protect even basic information of their operating system from this release (forwards?) it’d be trivial to learn that. They used to have excellent documentation of their operating system and how to make use of it available on their website – now they require a login to even watch what documents are available…
Does anybody happen to have a copy of the QNX source code from when it was publically available?
(under Apache license I think, and at least the kernel & basic libraries have definitely been there, they promised to add the Photon code and more drivers later but don’t know if they actually did it. Just remember that I signed up for an account, but I was specifically looking for the GUI part at that time which wasn’t available, so I forgot about it)
Would love to get a copy of it, being fascinated by low-level hacking (:
… and yeah, that floppy demo MarkHughes mentioned was pretty awesome. A fully functional web capable OS fitting in 1,44MB … somehow I’m truly wondering what makes current Windows and OS X fill a whopping dual layer DVD (and even more, how on earth can applying updates to Win take so . much . time …)
-x-x-x-
andreww591, do you have any material about your project online yet? This is not that unlike of what I am thinking about from time to time, but while I’ve actually toyed around with an own kernel 12 years or so ago, it is of course much too much for me and as a single person to really achieve something working. On the other side, I might be interested in participating in an open source collaboration, for fun and refresh my coding skills..
The genode project might have some good parts to borrow when it comes to drivers (they even have a HW accelerated Intel mesa GL) … but overall it’s truly frustrating that tech documentation about hardware is so scarce (if at all).
I am idealizing about a “hybrid” microkernel (VM, VFS, scheduling, messaging in the kernel, everything else outside..) renewing some of the very basic POSIX concepts like the pipes -attaching meta data to them, moving away from simple char[]s that get parsed etc. to something more object orientated.. same for /proc & /sys …
Fine-grained permissions, no more sudo – maybe? But have to read up about these per-process namespaces you’ve mentioned.
Together with a decent debugger a microkernel should be able to make driver development much more painless if I’m not completely wrong – imagine writing a driver just like every other application, compile & run w/ breakpoints etc.. as a crash wouldn’t hurt the system, just try to fix the code, run it again …
Virtualization with the microkernel as a hypervisor could also open interesting abilities to easily(er) reverse engineer device drivers from Windows by capturing the communication between that certain hardware and the OS/driver?
Edited 2017-04-29 19:51 UTC
Possibly, although I don’t think it’s all that much easier to implement a VMM-level debugger on a microkernel.
Release a damn ready-to-go image for the RPi – maybe even sell a preconfigured RPi in a nice QNX/BB case.
This shouldn’t be hard.
Photon was one of the *awesome* things about QNX… too bad they let it rot. I have a 6.5 Hobby license, however with the sources this closed… I have enough projects that are open source to fill my time 🙂