As the end of the year approaches, it appears that virtualization’s time in the open source spotlight has all but come. Whether it is because of the machinations of companies like XenSource or analyst endorsement, emulating an enterprise-class infrastructure environment using open source has been on the rise in 2005. And, now, it looks ready to burst next year.
Xen Virtualization Quickly Becoming Open Source ‘Killer App’
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
just like linux, Xen will never be taken seriously when compared to commercial products
I dont think there is anything like xen commercially, it’s very different from vmware or virtual pc and the likes.
I also wonder if it’s not being taken seriously why are intel and amd planning hardware support for this kind of technology?
Not at all true. Everyone is taken seriously these days. Unfortunately often for the wrong reasons.
The commercial products are ripping the best features and ideas out of the open source projects and moving forward.
Open source developers are only beginning to understand the fruits of their labor are not only being used for the advancement of open source, they are being used against them.
Some the commercial applications look at/to open source as little more than an idea mill, with free labor. Small wonder some of these same players are so eager to support and endorse open source.
ALL open source efforts cut both ways……
> The commercial products are ripping the best features and ideas out of the open source projects and moving forward.
In general, wasn’t it the other way?
Its important to recognise that running multiple operating systems like Windows along with Linux is not the only feature needed by many users. If you want that you might have to wait for Xen enabled hardware but let me point out various other features where virtualisation is useful besides that.
* Server consolidation
* Colocation of services
* Security, isolation, maintainance
* Regression tests
Xen is extremely performance efficient when compared to VMware since the operating systems are modified to be Xen aware.
Xen is open source and supports more operating systems and architecture.
Morever it has additional security features and seperation. Traditional virtualisation programs run under a ring 0 vs ring 3 model where all of the virtualisation code is at ring 0 and everything is at ring 3. Xen has three levels ring 0 for itself Ring 1 for the operating systems and ring 3 for all the applications which makes it more capable.
There are probably a number of presentations available that explain all this in detail for those interested.
As an example, Daniel Veillard from Red Hat presented this
http://veillard.com/Talks/2005LinuxTag/Overview.html
Edited 2005-12-14 23:26
I work for Xen.
So I am really getting a kick out of most of these replies.
Some of you guys are very good at making it sound like you know what you are talking about.
But trust me…. You don’t.
I think you just want to make yourself sound smart, when in reality you dont know what you are talking about.
This is how bad info gets passed around.
If you dont know about the topic….Dont make yourself sound like you do.
Cuz some osnewsers belive anything they hear.
I don’t hear anything while reading osnews.
>I dont think there is anything like xen commercially,
>it’s very different from vmware or virtual pc and the
>likes.
Ok..thats an intellgent post.
Could you please share more of this thought with those of us that are in the curious crowd.
What exactly is the bullet points / elevator pitch etc that makes you say that its so different than VMware, especially maybe based on what little I know about VMware ESX.
Thanks!
> OP wrote: …it appears that virtualization’s time in the
> open source spotlight has all but come…
I couldn’t agree more, and just thought I’d mention that Sun has a major Xen project in the cooker too… You can follow the project as it developments here:
http://www.opensolaris.org/jive/forum.jspa?forumID=53
For example, here’s some project status info from one of the project leads:
http://www.opensolaris.org/jive/message.jspa?messageID=10532#10532
Edited 2005-12-14 21:16
“Xen Virtualization Quickly Becoming Open Source ‘Killer App'”
The day Xen will be able to run Windows on my current laptop, then yes, it could be a killer app. Until then, it’s just an open source project limited to the open source world. That’s no killer app…..
>The day Xen will be able to run Windows on my current
>laptop, then yes, it could be a killer app. Until then,
>it’s just an open source project limited to the open
>source world. That’s no killer app…..
You do know that day is almost here. All we are waiting for is the next set of intel and amd chips.
In fact its rumored that the chips that Apple will be using in their intel-based boxes will have the new support that will allow just that.
If this thing would allow me to run Windows and Linux at the same time and have them actually talk to each other, that would be sweet
Well, they’ll talk over the virtual network as if they were real machines. There are also plans for applications in virtual machines to be able to share memory directly directly across VM boundaries (with appropriate security restrictions), which should also enable communication.
Disclaimer: I work on Xen, am employed by XenSource.
Anonymous: The day Xen will be able to run Windows on my current laptop, then yes, it could be a killer app.
Milo_Hoffman: You do know that day is almost here. All we are waiting for is the next set of intel and amd chips.
My current laptop isn’t running on one of the not-yet-released Intel or AMD chips with hardware virtualization support. Neither is my server. Both are running on the existing generation of chips. To use Xen, I would need to buy new hardware. With VMware, I can do what I need with what I have now.
No No No No No No No … the true strength of Xen will never be running Windows — that will only be a nice additional feature. If you want to virtualize Windows, use VMWare or MS’s Virtual Server (or whatever the right name is) or similar.
Xen’s strengths apply to the Enterprise Linux shops, which need to run many Linux images, but may not need dedicated hardware for each image.
Xen’s strengths, IMO, are:
*The ability for Linux to natively to provide zone-style security through separately running kernels, and
*A HAL: dom0 will handle hardware, drivers, etc, and domUs can use an abstracted common interface
If you need to run Windows and Linux on the same hardware, you most likely are not an Enterprise shop. Developers and dabblers may find this useful .. but the money is in the Enterprise.
Just my $0.02,
-dotMatt
Best thing about Xen (apart from all the other niceness):
It’ll be a great tool for people interested in os development. Gimme drivers, gimme gimme
“If you need to run Windows and Linux on the same hardware, you most likely are not an Enterprise shop. Developers and dabblers may find this useful .. but the money is in the Enterprise.”
Okay, I’ll bite………………..
The money might be in the Enterprise, but it’s an open source application.
Where is the money?
The notion of soaking the piss out of the customer to support said application, is paved with huge stones of doubt. (hardware is getting cheaper by the second)
Where is the money?
> The notion of soaking the piss out of the customer
> to support said application, is paved with huge
> stones of doubt. (hardware is getting cheaper by
> the second)
> Where is the money?
The money is a) in support contracts, and b) in proprietary toolset that can be used for cluster management. The stuff you need to set up Xen systems is free. This’ll probably be adequate for small users but for anyone with significant deployments the VMware-style cluster managers will cost $$$.
There’s nothing to stop other companies also writing cluster managers for Xen, but the hope is that XenSource (the support company founded by Xen people) will have a decent slice of the financial pie 😉
Disclaimer: I work on Xen, am employed by XenSource.
Well if it is an “Enterprise” then they care about support. There is this thing called a SLA (Service Level Agreement) which “Enterprises” like the one I work for pay millions of dollars per year for someone to call when things break or don’t work properly.
Windows and Linux on the same box in the enterpise? The cool thing about virtualization is that your vms should be where they have resources. If that means Linux and Windows on the same box so be it. Xen can move machines around, why not use that ability for windows machines as well. Many enterprise shops have to use Windows servers that are heavily utilized. The real power of virtualization is resource management across server farms and the application being able to move them when they hit a threshold to improve performance. Xen currently can only do this with linux (with scripting). I think VMWare will take Xen under its wing and make both managable for that reason.
Many enterprises (fortune 500 companies) are already running many servers with windows and linux on the same box, except they aren’t doing it in Xen- they are doing it in vmware. Soon it wont matter where the vms reside, because VirtualCenter or Dunes will manage where the vms are VMotioned based on load to in order to get the appropriate resources above the threshold.
Pretty cool stuff.
That is not always true. Effectiveness vs efficiency used to be a dilemma for many enterprises. Do we want to get seperate server for each application (effectiveness: more reliable, more scalable, but also a waste of resources)? Or do we consolidate them into one server(efficiency: shared resources, so nothing’s wasted, but higher risks and less reliable).
With this virtualization software it will be possible to create a resource pool, consisting of many nodes. So you get shared resources for application services and database servies, for example. And you get hardware redundancy. Best of both worlds! This is also what ASP’s are aiming for.
And yes, these days we already have clusters without virtualization. But the problem here is that, in order to have total control, each eterprise has to has its own cluster, physically. So you end up with some clusters doing nothing, while other clusters are using 95% busy.
Maybe Linux alongside Windows is not the best example one can come up with, but what about several instances of Linux running together with his Solaris fellows, for example. How would that not make sense?
It would totally make sense. 😀 As long as my vm/application has the resources it needs to do its job, i dont care what os the other vms on the box are running. 🙂
well, as far as i know, all new intel pentium 4 & xeon processors are _shipping_ with “intel virtualization technology” built-in as of _now_, for unchanged prices.
intel press release [14 november ’05]:
http://www.intel.com/pressroom/archive/releases/20051114comp.htm
also, “parallels workstation” is using this technology as of _now_.
parallels features:
http://parallels.com/en/products/workstation/
-kamil
The OS has to be ported to work with Xen. Would it be in Microsoft best interests to make it work with Xen is the question.
MS are planning to have their own paravirtualisation API for guests – it’d be technically possible to support this under Xen but the question is whether it’s a) desirable and b) legal. In any case, it’ll be a while (years, by last estimate) before the MS hypervisor is released.
Unmodified OSes (such as Windows) will (eventually) run under Xen if you have hardware support for virtualisation – interestingly, MS are planning to require this hardware support for running *any* guest, even ones that have been ported to their API.
Windows has been demo-ed running under Xen but I believe the support for unmodified guests is some way from being as solid as the rest of Xen.
Disclaimer: I work on Xen, am employed by XenSource.
What are the legal (b) issues your are thinking about when wondering if ‘paravirtualisation API for guests’ is possible under Xen? Is it a patents issue or more to it than that. To what extent do this legal issues cover and are they stopping anything from being done?
I don’t claim to have the in-depth understanding of how virtualization works in general or Xen in particular, that some here may have. But needing to tweak the OS to work with Xen seems pretty undesirable, from my POV. Doesn’t that kinda mess up the OS a little? Tie it too much to the virtualization SW? Opportunity for introducing instability? Other questions: how will all virtualization SW be able to play on equal footing? Some OS vendors can then just play favorites, adding hooks needed for particular virtualization SW but not others? This seems like especially an issue for closed-source OSs.
With all of this virtualization tech coming out and OSX going intel next month, you can just put a simple window manager and a bunch of xterms to monitor your servers and run a real desktop OS like Windows or OSX
VMware as a company provides/has the managment and provisioning tools that Xen does not. Sure there will be third party companies that create management tools for Xen. Guess who one of them will be? VMWARE! Xen is so powerful. Its fast. I think VMWare would be crazy not to leverage that. We all know VMware loves open source.
I could see virtual center (vmware’s real product) being able to manage Xen farms transparently. As a matter of fact, ESX will probably be free if it wants to survive. Virtual Center and DRS DAS other resource management sw will be the cash cow of VMWare going forward. Xen = Free – MS Hypervisor will be part of Longhorn Server in 200x – ESX server = $3500? Look for that to change.
VMWare is so far ahead right now, and they will soon (well 2 years from now) go head to head with Microsoft’s hypervisor. By this time pieces of Xen will either be incorporated into ESX, or will simply be able to be managed by Virtual Center.
I think virtualization is one of the most exciting things to happen to IT in a while. Its truly a great thing.
fyi, Virtualization is quite an old approach and has mainly been developed by IBM in the middle of the 60s. The IBM370 shipped in the 70s was all about virtual machines. Actually it was full-virtualization, the VMs were supposed to behave as exact replica of the hardware. As the process of virtualizing all the resources was causing a big overhead, IBM developped specific hardware with an enhanced instructions set to speed up virtualization mecanisms.
In a way history is just repeating itself: para-virtualization was introduced to make virtualization easier and faster on commodity hardware and now intel announces that is is developping virtualization-enhanced hardware.
“f this thing would allow me to run Windows and Linux at the same time and have them actually talk to each other, that would be sweet
“
Currently Xen only supports Linux/Unix distributions that have been compiled for it. The OS needs to be specially adjusted to work with it, so the chance of Windows ever working with it is slim to none, just a guess since MS has thier own Hypervisor. If that is what you are trying to do, then VMWare or Parallels offer the best solution for that specific problem.
Just to point this out. Apart from being a ridiculously funny hack the Inel Virtualization Support does not make running multiple operating systems trivial. It virtualizes the ISA, not device drivers or anything of that sort. What it will do is make VMware’s job easier and let them concentrate on virtualizing things like USB devices correctly. It will also make virtualization faster and therefore an easier sell. It will be nice for the XEN people cause it will mean that they have to modify less of the kernel though the part that it will make obsolete is pretty simple anyway, most of the complicated Xen modifications are elsewhere. It will, however, help the VMware people cause it will radically simplify the VMMonitor.
Well… possibly the most complicated modifications to an OS to run on Xen are the memory management routines. The virtual device stuff is quite a lot of code but it’s less painful to get one’s head around 😉
Virtualising the memory management stuff still requires quite a bit of cunningness but it’s possible to move that stuff into Xen itself. Unmodified OSes do run on Xen if you have VT hardware, although the support needs quite a bit more work to get where we want it to be.
The device virtualisation stuff is being serviced by a device emulator based on the IO devices from QEmu. There is some planned rearchitecting to improve its performance under Xen. There are also plans for special “xen-aware” drivers for Windows (and friends) when running under Xen/VT, to maximise the IO performance.
Note that “Xen-native” OSes can run alongside unmodified OSes on Xen/VT.
Disclaimer: I work on Xen, am employed by XenSource.
So, you are saying that the current commercial MS Windows OSs won’t run under Xen hypervisor, even if you use the new CPUs from Intel and AMD with the Virtualisation Technology?
I’m curious how VMware gets Windows to run… Is it possible to do some VMware like stuff to get Xen running a commercial version of Windows. I realize that this isn’t the aim of Xen, but was just wondering if it could be done and what are the difference of Xen running MS Windows at a lower level compared to VMware running MS Windows.
Also, what exaactly are the benefits of these new Intel and AMD chips with the Virtualisation Technology? You seem to make out that its not so significant. BTW. Which of their chips are they in?
> So, you are saying that the current commercial MS
> Windows OSs won’t run under Xen hypervisor, even if
> you use the new CPUs from Intel and AMD with the
> Virtualisation Technology?
It’ll run. Windows XP was demoed at IDF running under Xen on Intel’s VT (although I understand that Windows isn’t very happy on Xen at the moment – there’s some sort of niggling bug that hasn’t been chased down yet). On either company’s hardware (only Intel’s is supported right now) Xen will support running unmodified Windows (and other OSes) side-by-side with ported “xen native” OSes
> I’m curious how VMware gets Windows to run… Is it
> possible to do some VMware like stuff to get Xen
> running a commercial version of Windows.
Technically possible, yes. Worth the effort? Debateable now that virtualisation-aware x86 hardware is coming out imminently. The tricks required to virtualise x86 *without* hardware support are fairly complex and do cost performance. The VMware guys have done an amazing engineering job at getting this work well, but it’s not something that’s critical for Xen.
> Also, what exaactly are the benefits of these new
> Intel and AMD chips with the Virtualisation
> Technology? You seem to make out that its not so
> significant. BTW. Which of their chips are they in?
Relative to unassisted virtualisation they should be faster (for full virtualisation, that is – paravirtualisation was pretty near native performance anyhow). This will benefit VMware users too, when support comes out in their products.
It also makes it tractable to support unmodified OSes with good performance under Xen – without the complexity that would have otherwise been necessary. Intel have kindly contributed VT support for Xen (AMD are following suit with support for their extensions).
Thanks for your time to explain, you’ve now made it much clearer.
But data centers and electricity are not
There was a VERY interesting xen talk soem hours ago – see the log: http://umeet.uninet.edu/umeet2005/talks/?lang=en&s=evaldoa
I had not thought of Xen as a killer app but as the killer way of running multiples of my favorite OS.
…great for running isolated unix systems such as a mix of some of the BSDs, Linux, and/or Solaris.
As for running Windows and OSX, yes, I guess some people will do that. I’ve had VMware for years, and while it currently isn’t the same thing as Xen, you could in theory run the ‘same’ configuration even if not as smoothly. I didn’t do it. Most of EMC’s customers did not either…they were running consolidated servers consisting of (usually) Linux as the host and either Windows or Linux as the guests with very little mixing.
I’d guess the same thing will happen with Xen. If you use Windows or ‘unix’ (aka Linux or BSD or Solaris…) you’ll continue to use mainly either Windows or ‘unix’ in consolidated and isolated CPU partitions. Integrating the OS into Xen to further stabilize the parts and to isolate them from crack attacks will happen.
Using Xen to create a machine that runs 10 operating systems will not be done often except for the “Wow, look at what it can do!” factor. A few will run one or two operating systems. Many more will run many copies of the same OS or a close cousin.
Finally we’ll get the equivalent of LPARs without expensive IBM hardware.
Dear Mark,
You make entirely too much sense to be chasing rabbits or rainbows. Bill Gates is always taking applications. Do yourself a favor and fill one out…….
Regards, Bill
that was fun……
Hi all,
This could possibly be a top-level story / article but I’ll post it here as the thread exists already. Just posted on xen-devel: http://lists.xensource.com/archives/html/xen-devel/2005-12/msg00543…
AMD (with help from IBM) have just made their first public release support for Xen to run unmodified OSes (i.e. Windows, legacy OSes, etc) on their Pacifica-enabled hardware. As for VT, these OSes will co-exist on the same hypervisor with Xen-native ports of OSes like Linux, *BSD, etc.
This means that when AMD’s Pacifica hardware becomes available to the public Xen should already support for it. Lots of the higher-level code is shared between the VT and Pacifica subsystems in Xen, so improvements to one will often benefit both.
Disclaimer: Yes, it’s me again. I work on Xen, I’m employed by XenSource.
Several weeks ago Sun and Xen made an announcement that Xen and Solaris would support each other. Does anyone have details and dates on this???
All speculation involving the MS hypervisor is highly non-concrete, since there are only a few presentations regarding what they intend to produce (I’d hazard a guess that they’re really only just getting started on this project).
The MS hypervisor architecture plans are very similar to the architecture of Xen 2.0 / 3.0 (and, for that matter, IBM’s POWER hypervisor) – which is sensible as it’s proven to give good performance and hardware support. Although MS will use (and require) the Intel VMX / AMD SVM extensions, they will also support paravirtualisation – both technologies have a special “call the hypervisor” instruction for this reason.
The following is pure speculation: I’d not be entirely surprised if MS tried to legally protect their hypervisor API from reimplementation. This ensures that their OS products will run best on their hypervisor, so it would make good business sense to do so if possible.
Interestingly, VMware are also working on a paravirtualisation API which they will be launching next year. I think they’re hoping for support for this to be included in Linux. As you’d expect, there’s something of a debate amongst the kernel developers as to whether Linux should explicitly support running on proprietary software platforms. I think the VMware API may legally be open for reimplementation by other hypervisors but don’t quote me on that 😉
In both cases, even if the API can be supported in Xen there’s the question of whether it *should*. For the next couple of years, my guess is that it won’t be worth it. However, when / if closed-source OSes start supporting a particular API it *may* be worth evaluating the performance gains possible and perhaps reconsidering this issue.
Solaris was running on Xen 2.0 ages ago… I haven’t heard anything from them which implies they’ve been busy on a Xen 3.0 port. Now that Xen 3.0 is released, the hypervisor interface is stable and will be supported for a long time – hopefully we’ll see something out of Sun soon.
My impression was that they’d probably want dom0 (“host”) and domU (“guest”) support – otherwise it’d be necessary to run some other OS like Linux in order to run Solaris 😉 (nb. that would be an easy way to leverage Linux’s hardware support whilst still running your apps under Solaris).
You should probably either assume that work is in progress and / or that they’ll want to find some strategically significant time to announce what they’ve done: they seem to be rather tight-lipped on their own forums at the moment. Just speculation on my part, though 😉
Apparently the primary bug that was crashing Windows guests is now solved (as of today) and they have stably run test suites for several hours. Like I said, the full-virtualisation stuff is under continuous improvement 😉
There’s still some interesting rearchitecting to be done to maximise performance, as well as the usual bug fixing and feature expansion.
I was going to write about how Xen was the killer app and would rock this planet, but I think it might be suboptimal. I’ve seen a couple of microkernel pages which basically state Xen is the enemy, that its a hack to get old generation technology running in triplicate. And they’re right…
L4 & microkernel systems have the capacity to skip paravirtualization entirely. Why are we running paravirtualization in the first place? Isolation. That isolation doesnt /require/ entirely seperate OS’s. Think of the RAM costs! Every single OS copy needs to have everything in its own ram. This is the limitation most current Xen hosts are hitting; ram size. With a microkernel, you could potentially share libraries while still maintaining iron clad seperation, giving each user what looks and runs like their own host. You could get effectively the same thing with soo much less price.
Admitantly, most OS’s lack the ability to grok having many many versions of a library installed. The dependency issue in getting shared libraries working would be, err, complex, if you were attempting to make a truly compliant base platform that you could make entirely seperate userspace hosts off of.
I also just heart the uber-fast IPC.
> needing to tweak the OS to work with Xen seems pretty
> undesirable, from my POV.
Needing to do so is unfortunate… but x86 makes it hard to do virtualisation efficiently (in terms of runtime performance and engineering effort) without modifying the OS. In general, on any platform, you’d still like the OS to be aware of virtualisation so that you can avoid operations that are usually expensive to handle in the VMM.
> Doesn’t that kinda mess up the OS a little?
Not if it’s done properly 😉 It typically only involves modifications to the architecture-dependent layer of the OS – most of the code is untouched. Also, the architecture presented is pretty close to that of the underlying hardware, so lots of code remains shared.
For x86, NetBSD and FreeBSD are adding a “xen” architecture directory. For Linux, other members of the Xen team are currently restructuring the Xen support code so that it’ll be *part* of Linux’s arch/i386. Supporting Xen becomes an optional part of the i386 Linux build.
On IA64, the changes required to run paravirtualised on Xen are really much smaller. On PPC there is already a well-specified hypervisor interface (used by IBM’s hypervisor products) and Linux already uses this interface: Xen/PPC will just provide that and Linux/PPC should Just Work (when compiled with the existing hypervisor support).
> Tie it too much to the virtualization SW?
Not really. The OS isn’t tied to anything, any more than its tied to any of the architectures / machine types it runs on. However, a Xen/i386 guest kernel won’t run on native hardware (“transparent paravirtualisation” where a kernel can run natively or on a paravirt hypervisor is possible – the IA64 port will do this, but it’s easier on that platform).
> Opportunity for introducing instability?
Yes, in principle, but in practice the same testing and “many eyes” review occurs for hypervisor support code as for any other architecture. Also, since “normal” x86 is so hard to virtualise if you don’t modify the guest, you’ve got to have a large complex virtualisation layer in the hypervisor which could still cause instability. Things are easier now hardware support for virtualisation is becoming available, though…
> how will all virtualization SW be able to play on equal
> footing? Some OS vendors can then just play favorites, adding
> hooks needed for particular virtualization SW but not others?
> This seems like especially an issue for closed-source OSs.
It gets thorny here; there are some in the OSS community who don’t want to see open source OSes including explicit support for running on proprietary hypervisors. OTOH, they already do include some support for this (e.g. Linux has support for IBM’s hypervisor).
As for closed OSes such as Windows, MS may decide not to support hypervisors from other vendors but there’s nothing to stop those vendors adding hypervisor-aware device drivers to Windows. It won’t be as good as more extensive modifications would allow but it’ll still be reasonable. The tradeoff if they decide to do this is that they’ll be slower than other OSes when running on any popular hypervisor except their own…
It seems likely the outcome will be the result of a combination of politics, technology and market forces – it’ll be interesting to see where it all goes.
Various people (myself included) are working on memory sharing techniques that should address the lack of sharing issue you raised. Personally I think of microkernels and VMMs as being part of the same spectrum, attacking related problems from different angles. In a VMM we’d tackle the memory sharing issue slightly differently but the end result should be similar.