Archive

Building a new programming language for games

Earlier this week, coder and game designer Jonathan Blow gave a presentation on his Twitch channel outlining his thoughts on why and how programmers might go about building a new programming language specifically for game development.

"We are literally killing ourselves every project, deathmarching to get games done," said Blow, during a Q&A segment. "It just really doesn't have to be anywhere near that bad -- at least for programmers."

New VIA CPU/chipset might be a good alternative for Free OS fans

VIA is working on a new x86 compatible CPU codenamed Isaiah II, the first in years from the company. Its low power, highly efficient design compares favorably to offerings from AMD and Intel in the same market. It was tested on a VIA branded motherboard with a VIA chipset, giving hope to Free Software users who currently struggle with locked down or unsupported boards from the major manufacturers.

‘How we’re addressing misleading apps in Windows Store’

Microsoft has explained that they have removed more than 1500 apps from the store.

Every app store finds its own balance between app quality and choice, which in turn opens the door to people trying to game the system with misleading titles or descriptions. Our approach has long been to create and enforce strong but transparent policies to govern our certification and store experience. Earlier this year we heard loud and clear that people were finding it more difficult to find the apps they were searching for; often having to sort through lists of apps with confusing or misleading titles.

This process is continuing as we work to be as thorough and transparent as possible in our review. Most of the developers behind apps that are found to violate our policies have good intentions and agree to make the necessary changes when notified. Others have been less receptive, causing us to remove more than 1,500 apps as part of this review so far (as always we will gladly refund the cost of an app that is downloaded as a result of an erroneous title or description).

The upside is that the store becomes a better, less cluttered and misleading place; the downside is that the walled garden is stronger. Is a top down approach really what we want, or is there a a better, community driven, approach that could be taken?

Genode 14.08 introduces new GUI architecture

The new Genode version 14.08 extends the graphical abilities of the framework to the level of flexibility expected from a general-purpose OS. In contrast to contemporary GUI stacks, Genode approaches the problem from the angle of maximizing security. This premise led to a fairly unique design. Further highlights of the new version are a new port of OpenVPN, an upgraded DDE Linux, vast performance improvements of the base-hw kernel, and networking for VirtualBox on top of the NOVA microhypervisor.

It goes without saying that a flexible and dynamic GUI stack is needed for a general-purpose operating system. Since Genode strives to become such a system, this problem had to be covered at some point. The starting point was the existing nitpicker GUI server, which is a secure multiplexer for the physical display and input devices. Regarding widget sets, the framework already featured a few custom graphical applications talking directly to nitpicker, and came with support for Qt and libSDL. However, there was a missing link between the low-level nitpicker GUI server and the applications, namely a window manager and desktop environment. The open question was how to maintain the rigid security provided by nitpicker while also supporting sophisticated window management, visually appealing window decorations, and customizability.

The solution took the Genode team more than a year to fall into place. At its core, it is a clever combination of small components that use existing Genode interfaces and facilitate two features unique to Genode: the virtualization of arbitrary OS services and the sandboxing of each individual process. The solution that comes with the new release adds merely 3000 lines of code to the trusted computing base of graphical applications while enabling advanced dynamic GUIs. The complex parts of the GUI such as the rendering and behavior of window decorations and window-layout management are stuffed away in sandboxes so that those complex (and potentially bug-prone) parts cannot compromise the privacy of the user. In fact, the security of the GUI stack does not even depend on a correctly working C runtime. So its attack surface is orders of magnitude smaller compared with commodity OSes. Of course, the current version is just a step on the road towards an integrated desktop environment but now, in contrast to one year ago, the path to walk on is clear.

Besides addressing the GUI stack, the new release comes with an updated execution environment for device drivers of the Linux 3.14.5 kernel. Thanks to DDE Linux, Linux subsystems such as the TCP/IP stack and the USB stack can be executed directly on the microkernels supported by Genode. The primary motivation behind the update was ongoing work on bringing the Intel wireless stack to Genode.

Functionality-wise, the highlights of the new release are a new port of the OpenVPN client that can now be used as Genode component, added networking support for guest OSes running in VirtualBox on top of NOVA, the use of multiple processors by the Seoul virtual-machine monitor, and the addition of pluggable file systems. Those and many more topics are covered in the detailed release documentation.

RetroBSD: Run old BSD Unix on a microcontroller

Modern microcontrollers are becoming quite beefy. The Microchip PIC32 line is actually an implementation of the MIPS32 4K architecture - and with 512K of flash and 128K of RAM you can even run Unix! RetroBSD is a port of BSD 2.11 for the PIC32. You might not be able to run X11, but it is still very useful and a great reminder of how small Unix used to be - and how far it has come.

Linux 3.15 released

Linux kernel 3.15 has been released. This release resumes much faster in systems with hard disks, it adds support for cross-renaming two files atomically, it adds new fallocate(2) modes that allow to remove the range of a file or set it to zero, it adds a new file locking API, the memory management adapts better to working set size changes, it improves FUSE write performance, it adds support for the LZ4 algorithm in the zram memory compressor, it allows to load 64-bit kernels from 32-bit EFI firmware, it adds support for x86 AVX-512 vector instructions; it also adds new drivers; and many other small improvements. Here's the full list of changes.

DragonFly BSD 3.8.0 released

The new release includes new USB stack (USB4BSD), which supports USB3; updated video drivers for Intel and AMD cards (although latter are still disabled by default); binaries in /bin and /sbin are now dynamic, allowing for PAM and NSS. The HAMMER2 filesystem is also included, but not ready for general use just yet.

Genode 14.05 with new ports management, block-level encryption

The just released Genode version 14.05 comes with new tools that greatly improve the interoperability of the framework with existing software. Together with a new process-local virtual file system embedded in the C runtime, this change should clear the way to scale Genode well beyond the current state. Besides these infrastructural changes, the release comes with a new block-level encryption facility, enables USB 3.0 on x86, introduces SMP support to the base-hw kernel, enables real-time scheduling on the NOVA hypervisor, and adds guest-addition support for VirtualBox on NOVA.

After the feature-rich release 14.02, the Genode developers took the chance to thoroughly revisit the tooling and overall structure of the framework. The goal was to improve its scalability with steadily growing amount of third-party software combined with the framework. Genode-based system scenarios combine the work of up to 70 open-source projects. However, until now, the framework lacked proper tools to manage such third-party code in a uniform way. In particular, upgrades of third-party software were poorly supported. To overcome those problems, the project took the wonderful Nix package manager as inspiration, created a set of new tools, and reworked the build system to make the porting and use of third-party software much more enjoyable and robust.

Most ported 3rd-party software relies on a C runtime. Genode offers a fairly complete libc based on FreeBSD's libc. However, translating the POSIX API to the Genode API is not straight forward. For example, Genode does not even have a central virtual file system service. Hence, different applications call for different ways of how POSIX calls are translated to the Genode world. Until now, the different use cases were accommodated by specially crafted libc plugins that tailored the behavior of the C runtime per application. However, as the number of applications grew, the number of libc plugins has grown too. In the new version, the framework consolidates the existing libc plugins to a generic virtual file system (VFS) implementation. In contrast to a traditional VFS that resides in the OS kernel, Genode's VFS is a plain library embedded in the C runtime. To the C program, it offers the view on a regular file system. But under the hood, it assembles the virtual file system out of Genode resources such as file-system sessions, terminal sessions, or block sessions. Since each process has its own VFS configured by its parent process, the access to resources can be tailored individually per process.

In addition to the infrastructural changes, the new version comes with plenty of platform-related improvements. Genode's custom kernel platform for ARM devices named base-hw has received multi-processor support and a new memory management that alleviates the need to maintain identity mappings in the kernel. The NOVA microhypervisor has been adapted to make static priority scheduling usable for Genode. Thereby the kernel becomes more attractive for general-purpose OS workloads on the x86 architecture. Also related to NOVA, the project has continued its line of work to run VirtualBox on this kernel by enabling support for guest-additions, namely shared folders, mouse-pointer synchronization, and real-time clock synchronization.

In line with the project's road map, the new version features a first solution for using encrypted block devices. The developers decided to use NetBSD's cryptographic device driver (CDG) as a Genode component. One motivation behind the use of CDG was to intensify the work with the rump kernel project, which allows the execution of NetBSD kernel subsystems at user level. After the project successfully used rump kernels as file-system providers with the previous release, extending the use of rump kernels for other purposes was simply intriguing.

These and more topics are covered in the comprehensive release documentation for Genode 14.05.

Clive: new operating system based on Plan 9 design

On May 23 Francisco J. Ballesteros announced a new operating system on the 9fans mailing list. Clive is influenced by the Nix (not to be confused with NixOS) and Plan 9 operating systems, featuring "zx" - a universal resource access protocol conceptually similar to Plan 9. The system is written in a modified version of the Go language.

More details are available in the design paper and in the manual.

Think OS: a brief introduction to operating systems

Think OS is an introduction to Operating Systems for programmers.

In many computer science programs, Operating Systems is an advanced topic. By the time students take it, they usually know how to program in C, and they have probably taken a class in Computer Architecture. Usually the goal of the class is to expose students to the design and implementation of operating systems, with the implied assumption that some of them will do research in this area, or write part of an OS.

This book is intended for a different audience, and it has different goals. I developed it for a class at Olin College called Software Systems.

Microsoft open sources big chunk of .NET

At its Build developer conference today, Microsoft announced that it was open sourcing a wide array of its .NET libraries and related technologies and creating a group, the .NET Foundation, to oversee the development and stewardship of the open source components.

Perhaps the highlight of the announcement today was that the company will be releasing its Roslyn compiler stack as open source under the Apache 2.0 license. Roslyn includes a C# and Visual Basic.NET compiler, offering what Microsoft calls a "compiler as a service".

This is more than just a code dump - Microsoft is launching the .NET Foundation, with representatives from Microsoft, GitHub, and Xamarin, among others, to act as stewards for the various related open source projects.

Genode 14.02 runs VirtualBox, supports NetBSD’s file systems

With the release of version 14.02, the Genode project has added two major features to the OS-construction framework: Using VirtualBox on top of Genode/NOVA, a wide range of unmodified guest OSes can now be integrated as components into Genode-based systems. The second feature is the addition of the file systems of the NetBSD kernel as rump kernels. The release documentation covers plenty of further improvements.

With storage and virtualization, the new release addresses two topics that are fundamental for using Genode as general-purpose OS, and both topics have been approached in a pretty holistic manner.

When it comes to storage, the project has significantly advanced over the past year but a few key pieces were still missing, namely mature file systems and a block cache. After having investigated FUSE-based file systems in the previous release, the project started exploring so-called rump kernels, which enable the execution of subsystems of the NetBSD kernel at user level. Originally, the rump kernel project was created to ease the development of drivers on NetBSD. The basic idea behind them is to link a driver to a stripped-down version of the NetBSD kernel that does not contain any privileged instruction. Hence, it can be executed in user mode. To interact with the outside world, a rump kernel uses a small so-called "hypercall" interface. By implementing this interface on top of the Genode API, rump kernels have become usable on Genode now. The immediate benefit is the availability of the time-tested file systems of the NetBSD kernel. But in the future, other NetBSD subsystem such as the TCP/IP stack or device drivers could be considered just as well.

At block level, the project took the chance to redesign the internal interfaces of the existing block-level components to support fully asynchronous operation. This step enables the effective use of modern disk-controller features such as native command queuing, and even the out-of-order processing of block requests. As the cherry on top of this line of work, there is a new block-cache component.

Over the past 5 years, virtualization has always played a role for the project. It started with running the paravirtualized OKLinux on top of the OKL4 microkernel. Later L4Linux was made available to Genode running on the Fiasco.OC kernel. Once the NOVA hypervisor found its way to the framework, the project embraced the use of the Vancouver virtual machine monitor, which enabled the use of unmodified Linux kernels. However, none of these solutions appealed well for a large user base, mainly because they were difficult to use or lacked features. By adding support for VirtualBox on top of NOVA, the project has finally found an answer to the question for product-grade virtualization on top of the framework. The integration of VirtualBox with Genode was no ordinary porting work but quite an engineering feat, which turns the architecture of VirtualBox pretty much upside down. In contrast to the host operating systems already supported by VirtualBox, Genode's version does not extend the host kernel in any way. VirtualBox leverages hardware-based virtualization (VT-x) but lives as a plain user-level program with no special privileges.

As with each new version, there are numerous smaller improvements and new features. For example, a new pseudo file system called trace-fs makes it possible to interactively use Genode's event tracing mechanism via Unix tools such as cat, grep, and echo.

All the changes are covered in detail in the release documentation of version 14.02.

Enyo introduces Moonstone, Spotlight, Enyo 2.4

The webOS Enyo team has released Moonstone (UI) and Spotlight (user interaction) libraries as part of a new webOS TV SDK>, powering the next generation of LG Smart TV. There is also a new version of Enyo coming (2.4).

While Moonstone and Spotlight are certainly highlights of Enyo 2.4, you may be just as interested in the robust new data-layer support you’ll find in this release. Enyo 2.4 has support for observers, one- and two-way bindings, computed properties, models and collections, and a set of new data-aware UI controls.

Why games should enter the public domain

A few days ago I inadvertently caused a bit of a fuss. In writing about GOG's Time Machine sale, I expressed my two minds about the joy of older games being rescued from obscurity, and my desire that they be in the public domain. This led to some really superb discussion about the subject in the comments below, and indeed to a major developer on Twitter to call for me to be fired.

I wanted to expand on my thoughts.

Fascinating article on Rock Paper Shotgun from John Walker on why he thinks software copyright (and possibly other kinds too) should come with a much shorter shelf life. Although ostensibly about videogames, much of it could be said to apply to recent events in mobile OS development too.

Linux 3.13 released

Linux kernel 3.13 has been released. This release includes are nftables, the successor of iptables, a revamp of the block layer designed for high-performance SSDs, a power capping framework to cap power consumption in Intel RAPL devices, improved squashfs performance, AMD Radeon power management enabled by default and automatic AMD Radeon GPU switching, improved NUMA and hugepage performance , TCP Fast Open enabled by default, support for NFC payments, support for the High-availability Seamless Redundancy protocol, new drivers and many other small improvements. Here's the full list of changes.

Valve listens to feedback, tweaks Steam Controller further

Valve has been listening to the feedback they've been getting, including the complaints people have had about the lack of a proper digital, 4-way pad, and now they are set to replace the touchscreen on the controller with a set of 8 individual buttons. (source: Gamespot).

In other news, Valve's Anna Sweet tried to alleviate the concerns various parties have expressed and confirmed that their plan is to bring music, TV and movies on the platform in a streaming form, making it suitable for more than just gaming.

MirageOS: rise of the virtual library operating system

ACM Queue posted an article titled "Unikernels: Rise of the Virtual Library Operating System", written by the main developers of MirageOS. The article is a good overview of the motivation and design of Mirage, which is a specialized lightweight hypervisor guest operating system for Xen. It's developed in OCaml and benefits from its features like memory safety, module system and compiler optimisations. It's comparable to OSv on many fronts.

More here.

OSv: a new open source cloud operating system

OSv is a cloud-based operating system with build images for Xen, KVM, Amazon web services and VMware coming up. It is written from scratch, "designed from the ground up to execute a single application on top of a hypervisor, resulting in superior performance and effortless management". Linux.com ran an interview with some of the developers behind OSv.

The operating system will be optimized for Java applications "by exposing OS internals and direct access for page tables, scheduling decisions and the raw IO layer". It will not be restricted to only Java applications though and will run JavaScript, Scala, Clojure, JRuby, Jython and more on JVM. Surprisingly, C is also supported.

OSv promises "Zero OS management" with "no need for administration, template management, configuration and tuning". Common Java framework integration consists of "frameworks such as Tomcat, JBoss, SpringSource . Common open source technologies such as Hadoop and NoSQL are being optimized and integrated to run on top of OSv."