Interview with Matthew Dillon of DragonFly BSD

Today we feature a very interesting interview with well known *BSD hacker Matthew Dillon over his latest project, DragonFly BSD (also known for his Linux kernel contributions, Amiga C compiler hacking back in the day and the Backplane distributed database). Matthew discusses DragonFly’s status, goals, the overall BSD platform, innovation, and more. Update: Added one more question at the end of the Q&A.

1. Please tell us about the general status of DragonFly BSD.


Matthew Dillon: The project has been going very well. We’ve primarily been focused on the ‘guts’ of the system during this initial period, and you can get a fair idea of the work that has been accomplished so far by looking at the Diary page on our site.


Most of the work so far has been to operating systems internals. The
work has been a combination of new work, like our light weight kernel
threading core, plus selective backports from FreeBSD-5 to keep the
system’s device drivers up to date (e.g. such as the USB subsystem).


From a userland perspective we have maintained a FreeBSD style
environment, so DragonFly basically runs everything that FreeBSD-4.x
can run. The packaging system probably won’t be done until the second
release so we are at the moment leveraging off of FreeBSD’s ports system
for user apps. Everything you’d expect of a BSD system (X, mozilla,
etc) is available to DragonFly users.


The first release is slated for some time in mid-June, hopefully
before the USENIX Technical Conference. That will be the 1.0 release.
We’ve been fairly careful to maintain as high a level of reliability
as possible during development and I think we’ve done a pretty good job
meeting that goal. The first release is intended to be more of a
technology showpiece then an integrated end-user platform.


2. Are you using any bits and pieces from FreeBSD-5, or you only strictly importing/exporting to FreeBSD-4 codebase?


Matthew Dillon: DragonFly began as a fork off of FreeBSD-4, because that was the most
reliable starting point and because we wanted to do major core pieces
of the system quite differently from the direction FreeBSD-5 took. For
example, we are focused on more of a compartmentalized threading model
to scale to SMP rather then the mutex model that FreeBSD-5 has chosen
to use. But the FreeBSD-4 codebase is of strictly limited utility as
a source of new code and maintainance updates. FreeBSD developers
are doing nearly all new coding in the FreeBSD-5 branch.


So, basically, we are doing the major core pieces of the OS differently,
such as our significantly evolved threading and messaging subsystems,
but we are also maintaining a FreeBSD-5 compatible (or mostly compatible)
device driver API in order to be able to bring in all the excellent
device driver work that has gone into FreeBSD-5. It’s simple logic,
really… we don’t have the manpower to be able to accomplish both
our infrastructure goals *AND* be able to maintain pace with new PC
hardware at the same time. This methodology allows us to proceed on
both fronts by focusing our own new work on the infrastructure and
bringing in FreeBSD-5’s device driver work. This isn’t to say that we
don’t do some of our own DD work, but the vast majority of it is
take from FreeBSD-5 by design.


3. What is the primary goal of dragonfly, servers or desktops?


Matthew Dillon: Both. When it comes right down to it the idea of targeting a system
to the ‘server’ is simply another word for ‘reliability and scaleability’,
and the idea of targeting a system to the ‘desktop’ is simply another
word for ‘out of the box GUI’. It’s not really an either-or deal.
All UNIX systems, including Linux, the BSD’s, DragonFly… basically
use the same desktop components so supporting a desktop environment
comes down to being able to provide integrated solutions that work out
of the box.


It is extraordinarily difficult to make GUIs work out of the box on
PCs due to the wide variability in hardware and peripherals, but at the
same time technology has continued to progress over the years towards
standards that actually make this easier to accomplish. At some point
the standards going in one direction will meet the software going in the
other and systems such as Linux and the BSDs (including DragonFly) will
be able to approach the out-of-the-box compatibility that took Microsoft
billions of dollars of development to accomplish. It isn’t a matter of
if, it’s a matter of when.


4. Do you eye the embedded systems market at all?


Matthew Dillon: It is not a focus. We fully intend to make DragonFly operate in
64 bit environments such as the AMD64 (which is also compatible with
the Intel’s latest 64 bit announcements even though Intel doesn’t like to
admit it), but embedded systems are already very well served by
Linux and NetBSD and we would prefer to focus on technology rather
then platform ports. FreeBSD-5 has had a difficult time supporting
the half dozen or so platforms they are trying to work with and I doubt
we would fare any better, so I am not even going to try. Eventually, if
DragonFly becomes popular enough, embedded systems ports will probably
happen, but it’s probably several years down the line.


5. How big is the DragonFly team currently? Are you happy with the development pace?


Matthew Dillon: Very happy. The main kernel@ mailing list has 241 people subscribed
to it as of now and I have handed out 9 commit bits, with another dozen
or so people submitting patch sets outside of that. We have a good
spread of developers focused on different subsystems. For example,
Jeffrey Hsu is focused on threading the network stack while Joerg
Sonnenberg has been focused on the PCI bus and ATA disk driver subsystems,
which frees me up to be able to focus on technological infrastructure such
as the threading subsystem. It’s about at the limit which I can sustain
and still have time to do significant programming myself, in fact!

6. Are you planning to import some of OpenBSD’s “tricks” for extra security?


Matthew Dillon: We’ve been looking at both OpenBSD’s security features, such as their
system call filters, and FreeBSD’s extended attribute and MAC security
infrastructure. Security is important but it’s no where near the top
of the list. There is a huge amount of other infrastructure that really
needs to be done first before we have the resources to address security
beyond the standard UNIX security model. In fact, a good chunk of the
infrastructure we intend to put in, such as syscall messaging and VFS
environments will be needed to serve as a basis for future security work
so it would be a bit premature to start on the security work at this
early date.


7. What do you think of the bsd terrain today? Are the BSDs doing/achieving enough as a platform to keep up with other Unix/Linux/Windows competition? Do you think that BSDs should re-innovate themselves at all levels in order to “go with the times”?


Matthew Dillon: I believe that the BSD’s are doing quite well as a platform, even though
Linux obviously has the eye of the masses. All the BSD’s have steadily
increased their development resources over time and the Linux phenomenon
hasn’t really dampened it. Keep in mind that developers work on the BSDs
for very different reasons then developers who work on Linux. The BSDs
are all about advancing software technologies into new arenas, while
Linux is all about leverage (which is why you see a very well integrated
security subsystem in FreeBSD-5 and OpenBSD and ten different types of
filesystems in Linux). Even better, nearly 100% of the user application
base developed under Linux compiles and runs natively on the BSD’s (and
people often forget that major pieces of software such as the X windowing
system existed long before Linux came on the scene, running on BSD and
commercial UNIX systems). People often forget that Open-Source means
precisely that… open source code, which means portability across
platforms and operating systems.


In anycase, you have to keep in mind that programmers always have their
own reasons for working on a project, and since programmers are not
consumers the reasoning is always very different from the reasoning
a typical consumer might have in choosing a system. This pretty much
guarentees that the behind-the-scenes interest, economics, and even
politics driving the people who actually work on a project like
DragonFly or *BSD, or Linux, has nothing whatsoever to do with what
you might read in the popular press (which is consumer-centric).


8. Currently the installation of DragonFly is very manual. What plans do you have for the installation procedure?


Matthew Dillon: At the moment the Live CD ISO is targeted towards developers rather then
end-users. We have been discussing installer ideas for months. We
probably will not have anything ‘good’ in the first release, but we
will certainly have something reasonable in the second release.


This is both good and not so good. It’s good in that it means we don’t
have the pressure of having to deal with bug reports from non-technical
end users yet. It’s not so good because, obviously, a good installer
will greatly improve our user base and interest in the project.


9. Do you have any plans to port to PPC and maybe merge some code with Darwin?


Matthew Dillon: The PPC is a good target to port to and will probably wind up being
second on my list. The first port is going to be to AMD64 (which also
means Intel’s latest 64 bit announcement, since it’s compatible with
the AMD64). There is no time frame for that but it will likely be
started after the first release. A PPC port has a likely time frame of
a year or two.


10. What feature on your OS has you very excited? How is Dragonfly going to differentiate from other similar solutions? Is innovation among your goals? If yes, what kind of innovations are you looking into doing?


Matthew Dillon: Well I strongly believe that any project needs to have an unattainable
goal, and our unattainable goal (which I hope actually winds up being
attainable) is to develop DragonFly into a transparently cluster-capable
system implementing native SSI (Single System Image). It is something
that no non-commercial system today can do (the type of clustering Linux
supports isn’t even close to the type of clustering that we have as our
goal, and clustering has never been one of the other BSD’s goals as far
as I can tell).


In the short term, I have become very exciting about our light weight
kernel threading technology, in particular the methodlogy we are
adopting to serialize data access by partitioning major subsystems
into threads instead of serializing data access with mutexes (FreeBSD-5
and Linux use a mutex-centric model, DragonFly uses a thread-centric
model). The reason for this excitement is that it is becoming clear
to us that we can develop very clean-looking, elegant, debuggable,
SMP scaleable software using this model whereas using the mutex model
generally results in much less elegant (even ugly),
difficult-to-debug code. Code complexity and code quality is a very
important issue in any large piece of software and we believe we have
hit on a model that directly addresses the issue in an SMP environment
without compromising performance.


11. What is the status of the Backplane DB’s Apache PHP module? Do you still work on the db full time?


Matthew Dillon: The Backplane DB is virtually a whole project unto itself. Due to a number of business issues with the software we were unable to give it a completely open-source license, and because of that encumberance I felt uncomfortable making an OSS project out of it. Backplane is on the backburner for now, I have been working on DragonFly full time for most of the last year.


Backplane Inc is shopping the database around. It’s a very technically competent multi-master database and replication technology but it really needs the resources of a larger company to advance further.

26 Comments

  1. Kaiwainz 2004-03-13 9:30 am EST
  2. Anonymous 2004-03-13 10:17 am EST
  3. Anonymous 2004-03-13 10:26 am EST
  4. Alex M. 2004-03-13 10:27 am EST
  5. Kaiwainz 2004-03-13 10:30 am EST
  6. Emil Oppeln Bronikowski 2004-03-13 10:57 am EST
  7. xispes 2004-03-13 2:43 pm EST
  8. CaptainPinko 2004-03-13 4:24 pm EST
  9. none 2004-03-13 6:15 pm EST
  10. Shawn 2004-03-13 7:58 pm EST
  11. blah 2004-03-13 8:29 pm EST
  12. Anonymous 2004-03-13 9:39 pm EST
  13. bsdrocks 2004-03-13 9:56 pm EST
  14. Kingston 2004-03-13 10:07 pm EST
  15. James 2004-03-13 10:28 pm EST
  16. Kingston 2004-03-13 10:36 pm EST
  17. Aeonsfx 2004-03-14 1:38 am EST
  18. anon 2004-03-14 2:45 am EST
  19. Kingston 2004-03-14 5:38 am EST
  20. Roberto J. Dohnert 2004-03-14 7:33 am EST
  21. Anonymous 2004-03-14 3:25 pm EST
  22. mips 2004-03-14 4:21 pm EST
  23. Anonymous 2004-03-14 4:39 pm EST
  24. Kingston 2004-03-14 5:21 pm EST
  25. Anonymous 2004-03-17 7:54 pm EST
  26. Kingston 2004-03-21 2:44 am EST