Building The Next Generation, Part 3: Security and Files

This series explores the sort of technologies we could use if we were to build a new platform today. The first 2 parts covered the Hardware and core OS. In this third part we look at security, the file system, file management and throw in a couple of other random ideas for good measure.


<rant mode=”on”> I intensely dislike the industry’s tendency to blame users for security problems, this is a cop-out, a dereliction of responsibility. The security problems are created by the computer industry and it is up to it to fix them. Viruses, Trojans and other nasties have been around for many years, how can the users be at fault when the industry has done seemingly nothing to defeat them? A car user is not expected to know the inner workings of a car in order to drive, why should he know the inner working of a computer? Security is not simple, [Security] but that is no excuse for blaming the user. </rant>

If there are patches to be downloaded the system should – by default – check for them daily. Of course if the system was properly designed in the first place you wouldn’t need many patches. Microsoft was warned about potential security problems many years ago, did they do anything about it? Who’s fault is that? That said there are more secure OSs than Windows especially the Unix based OSs such as Linux / *BSD and OS X (despite thinly disguised marketing efforts that say otherwise).

In any new system security should be built in from the start. It should assume everything wants to cause damage so restrict a program’s potential to do so. I don’t believe in building a wall as it can be broken through, I believe lots of walls are a lot harder to get through and present many opportunities for repelling an attack, thus security should be considered in all parts of the system. No one part should be relied upon. All parts of the system should by default, be secure.

Virus Scanning

Scanning everything with a virus scanner should be a standard part of the system, not an add on. Even if there are no viruses (or Virii) the system should scan an incoming file to see if it or part of it is executable.

Sandbox all new files

All files, not just e-mail / web downloads, that way they can’t do any damage. If an executable is not run immediately mark it so when it is run it can be sand-boxed first time. FreeBSD can sandbox applications using Jails [Jail].

You could go further and sandbox everything at all times and proactively look for things to sandbox – i.e. if an application attempts to download an executable file and execute it in memory, the system should either prevent or sandbox this behaviour.

Don’t allow programs to delete all files

Also, don’t allow programs access to the backed up files (see File System section). This will prevent a virus or errant program from deleting all the files in your home directory. If it tries the files should be moved to backup, the system should monitor the file system for this type of behaviour, warn the user when detected and give them an option of restoring files and either disabling the application or containing it’s actions to specific files. Deleting backups should be a privilege only the user has – no application should have this ability.

Such a scenario is possible on almost all Operating Systems and it’s happened to me in Linux – back in 2000 an alpha release of Opera browser crashed and removed the contents of my home directory, needless to say I wasn’t exactly happy about it. I expect an alpha to be unstable and missing features, I did not (and do not) expect it to take out my home directory. If an application was so bad that it could do such a thing think of what a malicious programmer could do. Currently applications by default can cause as much damage as they wish in your home directory, the system should prevent this.

Automatically identify all files

Warn the user when the identification is incorrect, if there is a text based identifier, change it (but tell the user). Text based file identifiers can be and are abused and should no longer be relied upon.

Don’t run services as Root

This is impossible to achieve on most Unix based systems due to the design of the kernel. This is one of the advantages of a microkernel as you don’t need to run services as root [Microkernel] reducing their ability to do damage.

Limit external access

Despite being aimed as a desktop system we may want some server like abilities such as the ability to control the system from outside. This should be possible but only via an encrypted method and should preferably only be possible with a dedicated application. Some non-encrypted connections would of course be possible i.e. Web or FTP servers.

Indirect program launching

A program that launches other programs should not be able to launch programs directly, the launch would go through the interaction engine (described in a later part of this series) which launches the program. This sounds restrictive but making program launches indirect means they wont return to a terminal once done, an attacker thus cannot get access to a terminal by externally crashing a program.

Exported Services

If I run a web server it automatically becomes visible to everyone on the outside unless it has been fire-walled off. I propose a “tunnel” be added to our system and only services which have been explicitly been “exported” along this tunnel can be accessed from the outside world. This would mean the web server would have to be explicitly exported before it could be accessed from the outside world. The Tunnel could also be monitored so what’s going in and out could be tracked and displayed, if there is something undesirable running it can have it’s export disabled (automatic exporting by programs should not be possible).

Quite how a tunnel would be implemented is open to question. One possibility is to have two network stacks: The Internal stack is in contact with the programs in the system. The External stack is in contact with the external network interfaces. The tunnel sits in between connecting the two stacks. You could run a Router, Firewall, NAT (Network Address Translation) and other services on the external stack yet have the inner stack completely isolated from the internet. When you do connect you could go via NAT which itself adds another layer of security.

Security in the file system

“Invisible” file backups could be used which only the system would be able to access, they would never be presented to applications. Only the user via the system would be able to manipulate them (i.e. to delete confidential data). A full OS backup could exist like this so if an attacker manages to circumvent the controls preventing them from changing system files, the system would detect the difference and replace the changed file with the correct one. The invisible OS backup files should not be presented to anyone, not even the user.

Code level security

Another method of improving security is to improve the code in the system, audit [Audit] the code for errors and bad code. Better code means less bugs and less bugs means the system is more stable and more secure.

Another method of improving security is to enforce correctness in either the language itself or at runtime. Some programming languages make it easy to write insecure code [BadCode] but one method of improving it is to add additional modules at compile or run time which check the program as it runs. Perl already does this in it’s “Taint” [Taint] mode but it may also be possible in languages like C [Code].

These techniques could be taken further – if libraries include potentially bad functions why not remove them and replace them so they are not in the system API. Scan executables for dubious function calls, if they are present refuse to run that software. It’ll make porting software a royal pain in the arse but software will be better for it.

Trying to modify and existing system to accommodate such a level of security will not be easy and liable to break applications and irritate users. This is one of the benefits of starting anew since there is nobody to annoy. It will be interesting to watch the reaction of the market to Service Pack 2 for Windows XP.

Security is a difficult subject but by restricting the system defaults like this and making a paranoid system which is constantly checking itself and everything coming in will make life more difficult for attackers. Locking the system down from unnecessary changes and keeping the bugs to a minimum means security is built right into the system, it’s not added after the fact where it can never be as effective. It also works, OpenBSD [OpenBSD] have been using this philosophy and at least some of these techniques for years.

Of course power users will not like a system like this but there’s no reason why controls could not be added to allow developers and power users more access to and control of the system.

The File System

Of course for our new platform we should have the best file system we can build. It should ideally have some of the following features / functionality:

  • 64 bit
  • Journaled
  • Non fragmentory
  • Automatic analysis and identification of files
  • Arbitrary metadata for files with indexing to facilitate high speed searching
  • Support for “live” searches – that is it should monitor the file system for changes and tell interested applications about the changes

This may sound like a radically advanced file system, but I have just described the file system “BFS” which was introduced to the world in BeOS. There’s a great hullaballoo about OS X “Tiger” adding search technology and Windows Longhorn using a database as a file system (an approach Be tried and later abandoned due to low performance). This is another example of existing technology spreading through the industry [Indexing] even if it’s old news to some of us!

I don’t however think that Be or BeOS users ever fully utilised the capabilities of BFS and in that respect support for it’s capabilities could be better integrated into the system. The apparently very similar technology in OS X Tiger looks like it’ll be a lot more integrated and thus useful to the end user.

Gnome Storage [Storage] wants to go further and translate files when storing them so you can read them with different programs without needing to support different formats. Similar functionality was available with Datatypes on the Amiga and Translators in BeOS though these were used by applications directly. I’m not sure if I like the idea of doing this as part of the file system as it’ll add a performance impact every time something is read / written and the use of XML means conversions to / from non-linear structures, computers are inherently bad at dealing with these in an efficient manner (throwing a faster CPU at it will not help, it’s a memory latency problem).

Search: Finding it better

If I’ve just downloaded pictures from a USB stick and want to edit them I have to either find then and activate the editor or, run the editor and manually find them with the file requester. This could be made easier by adding a few buttons to the file requester which use search functionality, press “latest files” and they’ll just appear. You would never need to go looking for files again.

Of course this wont work for everything, in order for metadata to properly work it needs to be added to the files and there are limits to how much of this can be done automatically. Adding metadata will need to be done at least partially manually and that becomes a user interface issue. Nobody is going to add metadata to every file so a window could appear with a set of options to help this process, the context will be important as well as the type of file in deciding which options. Past actions could also be analysed so the system can guess and present options based on those.

This need not be as complex as it sounds, if I download an image it can give me options of Astronomy, Humour, Technical, Cool, Geeky, Personal etc. If it’s really clever it can look at the originating web site and give a guess – i.e. anything from NASA is likely to be in the Astronomy category (this would require the web browser to store the URL as metadata with the downloaded image).

Metadata can also be potentially be used for all sorts of other not so obvious things. If you are running a second OS as this system allows, it will expect files in certain places. Your file system layout may not be in agreement with this alternative i.e. you may not want to add a new partition to your Hard Disc. You could use metadata to create a virtual file system, write the desired path as metadata but put the file in a directory for the second OS’s files. When the second OS wants a file the file system it gets the file from this directory by looking for a file with that piece of metadata.

This has interesting security possibilities as the second OS will not be reading directories directly so even if it tries asking for something else it will never get anywhere as there will be no other files with the correct metadata.

You could also create a file system layout from metadata by doing the above in reverse. We could do the same with normal metadata, to go back to my downloads example lets say I’ve saved a few humour and astronomy images in my downloads directory. A single command could move them to the right place – i.e. home/nick/pictures/humour or home/nick/pictures/astronomy.

I’d like to see other functions added to the file system such as automatic backups, a technique which has been used on big iron for years. Again metadata could help out here – store the backups in another location with the original path in metadata. This would save having your directories have multiple copies of each file and rapidly becoming unreadable.

Searching could also be enhanced by storing parts of files in metadata. The system could search for files containing specific strings but look in the metadata index first for speed (Apple is doing this in Tiger).

File Management

Of course all these files will still need to be managed. One of the most useful applications I used back in my Amiga days was a “dual panel” file manager. There were many of these with the most famous being SID or Directory Opus v4 [DOpus4] (note: not the later versions which changed the layout). I’ve never found an equivalent on any other platform though you can get some for Unix/Linux [LinOpus]. You can also simulate a dual panel manager by placing 2 Finder windows side by side in OS X 10.3, you use drag and drop in place of the copy button (I have two Finder windows permanently set up like this).

The dual panel approach is very easy to use and you can navigate around the system at good speed if it is implemented well. I’ve never found a better way to manage files, you can of course use the desktop (spatial or browser) or even a command line but I don’t find any other method works as well or as quickly.

Of course we can now improve on it by adding support for drag and drop and metadata. You could have a music button which lists all your music files irrespective of where they are. The same could be done for pictures and other data formats. Actions could be enhanced, double clicking the file will launch it with a viewer / player, double clicking on the directory display will bring you to the file’s location, dragging the directory to the other panel opens that directory there.

History and hierarchy lists would allow you to go upwards or backwards to previous/other locations. Adding the ability to navigate via menus could allow faster jumps to other parts of the system.

It’s difficult to describe how powerful yet easy a program like this could be, DOpus and it’s like were very good at their task, to the point that I never used the desktop or the shell. Combined with metadata searching these tools could be even better.

Loading and saving files can also use some of the same technology (but not the dual panels). A standard file requester could allow fast navigation and allow searching of metadata to suggest the best location to retrieve or store a file.

Other Random Ideas:

Building a new OS means new concepts can be tried out, here are couple of other concepts:

Large Page Sizes

Existing Operating systems still have hangovers from the hardware they were originally designed on, i.e. memory management typically deals in page sizes of 4 KBytes. This seems a little small given today’s system requirements which are running into hundreds of megabytes of memory for user desktops and gigabytes for workstations. I’d suggest the use of a flexible page size starting in the order of 32 KBytes, It does have the disadvantage of using larger memory blocks but this can be reduced by allocating smaller memory requests inside already allocated blocks, this could be done by embedding a small memory manager into applications at runtime, when an application wants memory the memory manager could activate and check if there is sufficient memory in it’s allocated blocks, if not it will ask the OS for more memory.

This will not only make efficient use of memory but also reduce OS calls / context switches. Using large pages also shrinks the entire page directory so more of it can then be held on the CPU, page directory look-ups for data can then be done without needing to go to main memory as often. In fact if paging to disc is disabled (quite possible with today’s memory sizes) it’s possible the entire page directory will fit on the CPU giving a performance boost as it’ll never need to be flushed.

The Grid

The BeOS media kit allows you to arbitrarily redirect the output of media software, this for instance allows you to add an audio filter or effect to an MP3 player, even if the MP3 player has no support for audio effects.

The Grid is the same thing but extended to the entire system and even across multiple systems. If I want to save a file on a different computer I’d usually need to have to explicitly support this in the file system. The Grid would allow it to be transparent, an extra drive would appear on your file save requester and you select the drive and press save as normal, the file system just saves as normal and The Grid pushes the data across to the other computer.

The Grid could also enhance the existing media kit by moving some processing to the other system. You could conceivably even spread programs across multiple systems. Providing an application or OS service knows how to communicate via The Grid the systems will take care of the communication’s transport. The Grid could do away with the need to transfer data across a full network stack as The Grid will itself act as a simple network.

Again this comes back to the idea of simplicity, instead of having multiple applications supporting the functionality it is placed into the OS so anything can use it. Such a principle is also found in the Unix successor “Plan9” [Plan9], indeed The Grid could make good use of Plan9’s Plumbing [Plumbing] concept.


There are various trade-offs to be considered when contemplating a new system, we all want good performance but would we trade it for better security and stability? In today’s systems that question was asked years or even decades ago and the answer was no. Today CPUs provide 99.9% of the population with more power we can use and there’ll be heaps more power when we get multi-core CPUs on the desktop. Today with increasing numbers of viruses and attacks I’d accept the trade, it’s not as if anyone outside benchmark fanatics will notice the difference. I for one would like to use a system which is damn near impossible to hack and damn near impossible to crash.

The system I am describing is not so much new ideas but about taking the best ideas, old and new and putting them together to make something better than has gone before. Having a new system gives us the ability to do this but as the system allows other OSs to run along side we have a combination giving us the best of both worlds. However we can experiment beyond the bounds of OS design, we can also explore other areas and improve on those as well. It’s quite telling that even today very few if any have yet to get usability right. It’s not an easy subject, more a question of opinions than absolutes. The Mac is famous for it’s usability and on that very topic…

A correction:

In Part 1 my description of the Macintosh did not attribute it’s ease of use to Jef Raskin’s original vision of the machine of which it was a major part. This has since been corrected, my apologies for the error.

37 years ago, Jef Raskin was one of the people who introduced the concept of usability to the world, it’s taking the world a long time to catch up. In Part 4 I cover how we can make our new OS more useable.

References / Further Information

[Security] Security

[Jail] FreeBSD Jails

[Microkernel] Microkernels have security & stability advantages.

[Audit] Auditing code is a method the OpenBSD group use to make their system more secure.

[BadCode] Embedding security in programming languages.

[Taint] Perl has a Taint mode.

[Code] Suggestions for improving security by embedding runtime modules.

[OpenBSD] OpenBSD has a reputation for security.

[Indexing] Description of various document indexing systems.

[Storage] Features in Gnome Storage

[DOpus4] Screenshot of Directory Opus 4.

[LinOpus] Worker dual plane file manager for *nix Screenshot

Another called Gentoo (not the Linux Distro) also for *nix and Screenshot

[Plan9] Plan9 the successor to Unix. An overview.

[Plumbing] The Plumbing concept from Plan9.

© Nicholas Blachford July 2004

About the Author:
Nicholas Blachford is a 33 year old British ex-pat, who lives in Paris but doesn’t speak French (yet). He is interested in various geeky subjects (Hardware, Software, Photography) and all sorts of other things especially involving advanced technologies. He is not currently working.

If you would like to see your thoughts or experiences with technology published, please consider writing an article for OSNews.


  1. 2004-07-26 7:17 pm
  2. 2004-07-26 7:21 pm
  3. 2004-07-26 7:35 pm
  4. 2004-07-26 7:48 pm
  5. 2004-07-26 8:07 pm
  6. 2004-07-26 8:40 pm
  7. 2004-07-26 8:50 pm
  8. 2004-07-26 9:32 pm
  9. 2004-07-26 9:35 pm
  10. 2004-07-26 9:45 pm
  11. 2004-07-26 10:22 pm
  12. 2004-07-26 10:48 pm
  13. 2004-07-26 11:02 pm
  14. 2004-07-26 11:05 pm
  15. 2004-07-27 12:36 am
  16. 2004-07-27 12:45 am
  17. 2004-07-27 12:53 am
  18. 2004-07-27 12:58 am
  19. 2004-07-27 1:02 am
  20. 2004-07-27 1:14 am
  21. 2004-07-27 1:29 am
  22. 2004-07-27 2:05 am
  23. 2004-07-27 2:32 am
  24. 2004-07-27 3:28 am
  25. 2004-07-27 4:00 am
  26. 2004-07-27 4:30 am
  27. 2004-07-27 5:36 am
  28. 2004-07-27 8:01 am
  29. 2004-07-27 8:40 am
  30. 2004-07-27 8:51 am
  31. 2004-07-27 9:19 am
  32. 2004-07-27 9:32 am
  33. 2004-07-27 10:48 am
  34. 2004-07-27 10:49 am
  35. 2004-07-27 11:11 am
  36. 2004-07-27 11:31 am
  37. 2004-07-27 11:45 am
  38. 2004-07-27 12:01 pm
  39. 2004-07-27 12:12 pm
  40. 2004-07-27 12:45 pm
  41. 2004-07-27 7:46 pm
  42. 2004-07-27 10:11 pm
  43. 2004-07-27 10:23 pm
  44. 2004-07-28 6:53 am
  45. 2004-07-28 8:24 pm
  46. 2004-07-28 11:14 pm
  47. 2004-07-29 1:21 am
  48. 2004-07-29 1:33 am
  49. 2004-07-29 1:42 am
  50. 2004-07-29 4:09 am
  51. 2004-07-29 4:30 am
  52. 2004-07-29 4:39 am
  53. 2004-07-29 12:27 pm
  54. 2004-07-30 4:08 am
  55. 2004-07-30 11:12 am
  56. 2004-07-30 11:16 am
  57. 2004-07-30 11:52 am
  58. 2004-07-30 3:34 pm
  59. 2004-07-31 8:01 am
  60. 2004-07-31 10:50 am
  61. 2004-07-31 10:56 am
  62. 2004-07-31 11:14 am
  63. 2004-07-31 12:41 pm
  64. 2004-07-31 1:24 pm