• strict warning: Only variables should be passed by reference in /var/www/sites/www.netomata.com/sites/all/themes/clean/template.php on line 126.
  • strict warning: Only variables should be passed by reference in /var/www/sites/www.netomata.com/sites/all/themes/clean/template.php on line 126.
  • strict warning: Only variables should be passed by reference in /var/www/sites/www.netomata.com/sites/all/themes/clean/template.php on line 126.
  • strict warning: Only variables should be passed by reference in /var/www/sites/www.netomata.com/sites/all/themes/clean/template.php on line 126.
  • strict warning: Only variables should be passed by reference in /var/www/sites/www.netomata.com/sites/all/themes/clean/template.php on line 126.
  • strict warning: Only variables should be passed by reference in /var/www/sites/www.netomata.com/sites/all/themes/clean/template.php on line 126.
  • strict warning: Only variables should be passed by reference in /var/www/sites/www.netomata.com/sites/all/themes/clean/template.php on line 126.
  • strict warning: Only variables should be passed by reference in /var/www/sites/www.netomata.com/sites/all/themes/clean/template.php on line 126.

Blogs

NCG now listed on freshmeat.net

Netomata Config Generator (NCG) is now listed as a project on freshmeat.net, which says it "maintains the Web's largest index of Unix and cross-platform software" and "is the first stop for Linux users hunting for the software they need for work or play ... continuously updated with the latest developments from the 'release early, release often' community".

Confirmed NCG requires at least Ruby 1.8.6

We've confirmed that NCG requires at least Ruby 1.8.6, and won't work under Ruby 1.8.5 or earlier. When we released NCG, last week, we weren't sure of this; we had done all of our development under Ruby 1.8.6, but hadn't yet tested it under any other version of Ruby, and said as much in the "REQUIREMENTS" file in the release. This shouldn't be a big obstacle, since Ruby 1.8.6 has been out for about two years, is generally considered stable, and is the "standard" Ruby version on many platforms.

Yesterday, Bob Sutterfield (who I highly recommend, by the way, if you need a senior IT manager who really understands infrastructure; we've worked closely together in the past, and he's great at building and running high-performance IT infrastructure teams) tried to install NCG on a "Rails all-in-one-trial" server image [login required] from RightScale, running on Amazon EC2. For some reason, this RightScale server image still includes Ruby 1.8.5, despite the fact that Ruby 1.8.6 has been out for two years now. He discovered that NCG depends upon Ruby methods that were added in Ruby 1.8.6, and don't exist in Ruby 1.8.5.

We fully intend for NCG to work under versions of Ruby after 1.8.6, though we haven't tested under any of them yet. In particular, we intend for NCG to work under Ruby 1.9, and will take any necessary steps to make that happen. (For what it's worth, Ruby 1.9 just had its "first stable release" (Ruby 1.9.1) about 3 months ago.)

We'll be updating the "REQUIREMENTS" file in our next release, to reflect this new information.

Modeling point-to-point links for NCG

A colleague is using NCG to generate configs for a network consisting of a set of point-to-point links, and asked what I thought would be a good way to get the data about the links into his Netomata network model. I wrote up my thoughts on a Point-to-Point Links Example page in the Netomata Wiki.

Follow Netomata on Twitter

Netomata now has a Twitter stream: http://twitter.com/netomata

Netomata Config Generator (NCG) released to public beta

I'm pleased to announce that we have just made the first public beta release of our Netomata Config Generator (NCG) product!

Netomata Config Generator (NCG) creates complete, ready-to-install config files for network devices and services from a common light-weight model of your network. Because these config files are generated programmatically (rather than by hand), and generated from a shared model (rather than being managed separately for each device or service), they are more likely to be consistent and complete, which makes your network more reliable, easier to troubleshoot, and easier to expand in both size and functionality.

The inputs to NCG are a model describing your network (neto and neto_table files), and templates (ncg files) for the config files of the various devices (routers, swiches, load balancers, firewalls, etc.) and services (SNMP, DNS, DHCP, etc.) that you want to generate config files for. From these inputs, NCG produces complete, consistent, ready-to-install config files for those devices and services.

For more information about Netomata and the philosophy behind NCG, see

Download

NCG can be downloaded from the Netomata Config Generator (NCG) page.

NCG requires Ruby 1.8.6 (it may work under other versions, but hasn't been tested under other versions) and the "Facets" Ruby Gem. NCG should work just fine on pretty much any UNIX/Linux-based operating system which has Ruby 1.8.6 available. The REQUIREMENTS file (included in the distribution, or available via the "Files" section of the NCG page) details how to install the necessary prerequisites on a variety of different platforms.

Documentation

Please see the Netomata Config Generator (NCG) program documentation pages.

License

NCG is released as free open source software under the GNU General Public License, version 3. Please see the Netomata Config Generator (NCG) License page for full details. If you would like to discuss alternative licensing terms, please email license@netomata.com.

Status

The current release is a public beta, and should definitely be considered experimental in nature. Commands and file formats are all subject to change, as we work out what paradigms work best and figure out how to get the most out of this tool. We'll try to limit future changes that break backwards compatibility, but we can't promise that at this stage; right now, we believe that it's more important to experiment with how best to build and use this tool, rather than to carve anything in stone.

To make an anology to programming languages, this release is like the interpreter for a language that is so new that the standard libraries for the language haven't been developed yet. The basic language capability is there, but the functionality and leverage normally provided by standard libraries doesn't exist yet.

Join us!

Please download NCG, give it a try, and join the community that's working to make this a useful and valuable tool for improving the reliability and scalability of networks like yours!

VLANs and Bonding and Xen, oh my!

Updated 4 Jun 09 to add missing 'pre-up ip link set vlanXX up' lines in 'vlanXX' entries in /etc/network/interfaces. Oops, sorry; I'm not sure how those lines disappeared between my actual working configs and this blog post! -Brent

Some things seem like they ought to be straightforward:

  • You're using Xen for virtual hosting
  • Your dom0 server has two physical interfaces, connected to two different upstream switches for maximum reliability
  • Your environment includes multiple VLANs
  • You want to set things up so that any given Xen domU guest can be easily attached to any one of those VLANs.

So, what you need to do is:

  1. Bond the dom0's physical ethernet interfaces (eth0 and eth1) into a "bond0" interface
  2. Set up VLANs via the "bond0" interface.
  3. Make those VLANs available to Xen domU guests via virtual bridges, one per VLAN

The hard way (trust me on this): use some variation/modification of Xen's "network-bridge" script to try to set everything up. Good luck. It will take you days, if you can get it working at all.

The easy way: set it all up in the dom0's /etc/network/interfaces:

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

auto eth1
iface eth1 inet manual

auto bond0
iface bond0 inet manual
        slaves eth0 eth1
        # It's important to use active-backup mode when you've got 2 separate
        # upstream switches. The various other bonding modes only work when
        # you're connecting to the SAME upstream switch; they are useful for
        # increasing bandwidth, but not for failover protection against an
        # upstream switch failure (or local interface failure, cabling failure,
        # and the like).
        bond_mode active-backup
        # "bond_miimon 100" checks the CARRIER status of the physical
	# interfaces every 100ms, and switches from the primary to the
	# backup interface if the carrier fails.  So, if the upstream
	# switch hangs but doesn't drop carrier, you're screwed, because
        # bond_miimon won't detect that.
        bond_miimon 100
        # Given that bond_miimon won't detect certain upstream switch
	# failures, it might seem like you want to use bond_arp_ip_target
	# monitoring instead, which purports to make sure you can actually
	# move traffic by sending ARP requests and looking for replies. 
	# Unfortunately, in this configuration, the bond0 interface doesn't
	# actually have an IP address (the vlanNN VLAN bridges that are
	# established below do, but not the bond0 interface itself), so
	# the ARP requests/replies won't work and you can't use
	# bond_arp_ip_target.
        #       bond_arp_ip_target 10.5.1.2 10.5.16.3
        #       bond_arp_interval 100

# Set up the bonding subinterface for VLAN16
auto bond0.16
iface bond0.16 inet manual

# Set up the bonding subinterface for VLAN32
auto bond0.32
iface bond0.32 inet manual

# Set up the bonding subinterface for VLAN48
auto bond0.48
iface bond0.48 inet manual

# Now set up the bridge for VLAN16, attach the bond0.16 interface to it,
# and assign the dom0 host's IP address on this subnet
auto vlan16
iface vlan16 inet static
        address 10.5.16.31
        netmask 255.255.255.0
        broadcast 10.5.16.255
        # Our default route is via this subnet to the router at 10.5.16.1
        gateway 10.5.16.1
        pre-up ip link set bond0.16 down
        pre-up brctl addbr vlan16
        pre-up brctl addif vlan16 bond0.16
        pre-up ip link set bond0.16 up
        pre-up ip link set vlan16 up
        post-down ip link set vlan16 down
        post-down brctl delbr vlan16

# And set up the bridge for VLAN32, attach the bond0.32 interface to it,
# and assign the dom0 host's IP address on this subnet
auto vlan32
iface vlan32 inet static
        address 10.5.32.31
        netmask 255.255.255.0
        broadcast 10.5.32.255
        # no "gateway" line, so no default route via this interface
        pre-up ip link set bond0.32 down
        pre-up brctl addbr vlan32
        pre-up brctl addif vlan32 bond0.32
        pre-up ip link set bond0.32 up
        pre-up ip link set vlan32 up
        post-down ip link set vlan32 down
        post-down brctl delbr vlan32

# Here's how to set up a bridge for a VLAN that the dom0 server does NOT
# have an IP address on.  This enabled you to attach domU guests to this VLAN
# without them being able to contact the dom0 server (and vice versa); the dom0
# server acts strictly as a dumb bridge for this VLAN.
# Note that the last word on the "iface" line is "manual" instead of "static".
auto vlan48
iface vlan48 inet manual
        pre-up ip link set bond0.48 down
        pre-up brctl addbr vlan48
        pre-up brctl addif vlan48 bond0.48
        pre-up ip link set bond0.48 up
        pre-up ip link set vlan48 up
        post-down ip link set vlan48 down
        post-down brctl delbr vlan48

Now, when you're setting up a Xen domU guest, to attach it to a particular VLAN, you simply add a "bridge=vlanXX" parameter to the "vif" line in the guest's .cfg file:

vif         = [ 'ip=10.5.32.99,bridge=vlan32,mac=00:16:3E:69:76:02' ]

The guest will see this as interface "eth0", with the IP address as assigned (make sure it matches what's in the guest's own /etc/network/interfaces file, or that the guest gets via DHCP).

If you want to attach a Xen domU guest to multiple VLANs, simply use multiple "vif" entries, like so:

vif         = [
                'ip=10.5.16.99,bridge=vlan16,mac=00:16:3E:64:1C:A7',
                'ip=10.5.32.99,bridge=vlan32,mac=00:16:3E:64:1C:A8',
              ]

The guest will see these as interfaces "eth0" (on VLAN 16) and "eth1" (on VLAN 32).

Note that you do NOT configure the domU guest itself for VLAN support. As far as the domU guest is concerned, there is no VLAN involved, and it is attached directly to the network in question. It's the same as if the domU guest were a physical machine, and were attached to a Cisco switch with "switchport access vlan XX" and "switchport mode access" set for the port the guest is attached to. The Xen dom0 server takes care of all the VLAN and bonding magic, simplifying the domU configurations.

Book Review: Dave Hitz' "How to Castrate a Bull: Unexpected Lessons on Risk, Growth, and Success in Business"

Dave Hitz is one of the cofounders of NetApp, which is a Silicon Valley success story, and has been with the company since 1992. I just finished reading his new book, How to Castrate a Bull: Unexpected Lessons on Risk, Growth, and Success in Business, and I highly recommend it.

In the book, Dave tells the intertwined stories of both his professional history (which included a stint as a working cowboy on an isolated cattle ranch; thus the title of the book) and the history of NetApp, and he shares many of the insightful lessons that he has learned along the way.

Dave is a natural story teller, and besides being fun to hear, his stories usually manage to memorably convey some point of wisdom. He's also a very nice guy, which is unfortunately rarer than you might wish in Silicon Valley. I've had the privilege of knowing Dave socially and professionally for a number of years, and I'm always pleased when I run into him at some industry or social event, because I know that a fun and interesting conversation will likely ensue.

The book is a great peek into what it takes to grow a startup from scratch to 8000 employees and billions of dollars in annual revenue, while also making it one of the most respected Silicon Valley companies and a fixture on every year's Fortune Magazine Best Companies to Work For list. He talks about the various stages of the company's growth, from the early product development days ("beat Auspex!"), to the hyper-growth phase (double the company's size and revenue every year for several years in a row), to the dark days following the dot-com crash in 2001 or so, to today's renewed growth in "The Age of Data".

Dave especially focuses on how NetApp has built, maintained, and continues to evolve the positive corporate culture that is one of its major strengths; that didn't happen by accident, and Dave discusses how they did it (and why!).

Strongly recommended.

Milestone: eating our own dogfood

We've reached an important milestone today: we're eating our own dogfood!

As of today, we're using our Netomata Config Generator (NCG) tool to generate a complete, correct, and consistent config for the core switch/router in our office and lab setup.

While our office switch/router is only an older 24-port Cisco 3550 that we picked up cheap on eBay, being able to generate its config is an excellent proof of concept; from here, it's easy to see how to extend our templates to generate a config for a much larger, more modern, and more complex Cisco router.

My next task is to extend the templates and definitions for our network to other types of devices and services, such as configs for MRTG and Nagios monitoring. Again, once the basic templates for a given service or type of device are done, its usually easy to see how to extend those templates to cover much larger and more complicated instances of those services and devices.

So, onwards towards a beta release soon, hopefully later this month!

VPN hell

So, a couple of days ago, I started out to do something that I thought was going to be fairly simple: set up a VPN from my primary laptop (a MacBook Pro running Mac OS X 10.5.6) to a VPN server running on a Linux host in our new lab infrastructure.

Warning: irate rant follows. Read at your own risk!

What a nightmare! I'm an expert at network security (I wrote one of the canonical books on the subject!), and I spent two full days beating my head against a wall of flakey software and impenetrably complex configurations before I finally figured out something that (so far, anyway) appears to work. And this was for something about as simple as it gets: connect a laptop to a server. All I can say is, the various VPN products out there (especially the open source ones) are burying their users in options, and are in serious need of simpler configuration paradigms.

I started off intending to use L2TP over IPSEC, since that's reported to be the state of the art at the moment, and supposedly works on both my Mac and our Linux server. OK, it's easy enough to get started: just install Openswan for the IPSEC part, then xl2tpd for the L2TP part. Unfortunately, the documenation for both sucks, and I spent hours figuring out the configuration for both, and then more hours figuring out the right finicky options to openssl to set up a certificate authority, generate server keys, and generate workable client keys for the Mac. And in the end, I couldn't get the Mac to use the client keys anyway... I got the client keys and certificate authority credentials into the appropriate Keychains on the Mac, so that I could choose "Certificate" as the authentication method in the authentication tab of the VPN setup pane in the Networking part of System Preferences, and it would let me choose the relevant certificate from the list, but the setting wouldn't "stick". It was just bound and determined to use "Shared Secret" for the authentication method, which the Openswan documentation had warned of problems with coming from the Mac. OK, so I figured that was enough time wasted on this way; let's try something else.

My second try was to use PPTP. Again, it was easy to get the PPTP package installed on the Linux server. Configuration was somewhat simpler than L2TP over IPSEC, if only because there were fewer options. But, in the end, that didn't work either. I'm not sure if the problem was on the server side, client side, or perhaps even the client firewall (which is also doing NAT). So, give up on that method, too.

Finally, I fell back to an old standby, OpenVPN, which is an SSL VPN. It's easily available on Linux. There's a perfectly good GUI wrapper for it for the Mac, called Tunnelblick. The OpenVPN documentation is excellent; it takes you step-by-step through setting everything up, and includes a number of well-though-out example configurations. They even provide a bunch of scripts to generate the appropriate certificate authorities, server keys, and client keys for you, without you having to dive into the morass of openssl options. In less than an hour, I had a working VPN.

The folks developing the Openswan IPSEC and xl2tpd L2TP servers could do a lot worse than to emulate the documentation, sample configs, and tools (especially the scripts that simplify dealing with openssl for the certificate authority, server keys, and client keys). And Apple needs to do a better job of supporting standards-based VPN software, and apply some of the much-vaunted user interface expertise to make this all easier for the end user to configure; if somebody like me can't figure it out, there's no way an average user is even going to come close.

Getting Xen 3.3 working on Ubuntu 8.10 (Intrepid Ibex)

We've obtained much of the lab equipment that we need to get started, so I've been working on getting it all installed and configured. The first thing I wanted to set up was a Xen server, which could then host virtual servers providing various essential services (DNS, DHCP, .deb/.rpm caching, VPN access, lab wiki, console server, syslog/Splunk, MRTG, Nagios, etc.) for the rest of the lab.

I've used Xen on Debian Linux as my primary server environment for several years, and been reasonably happy with it. The latest release of Xen (version 3.3.0) includes some new features that I think might be useful to us (such as optional full hardware virtualization, so that you can run guest operating systems without needed to modify their kernels, as is required for the older "paravirtualization" model that has historically been Xen's basis), but the latest version that's available in the Debian archives is Xen 3.0.3. I've been hearing a lot about Ubuntu Linux, which is a Debian derivative, and when I noticed that Xen 3.3 was available for Ubuntu, I decided to give Ubuntu a try.

It took a little work and a few bugfixes to get Xen 3.3 fully installed and configured on the latest Ubuntu release (8.10, aka "Intrepid Ibex"), but it seems to be working just fine now. Here are the steps I went through to get it working, in case anybody else might be interested.

1: Install Ubuntu 8.10

First, you need to download an Ubuntu 8.10 installation image, burn it onto a CD-R, and boot your server from that. There are several flavors of installation image to choose from; server versus desktop, GUI versus text-based installer, 32-bit versus 64-bit, and so forth. I chose the appropriate image for a 32-bit server (since that's what I have) with the text-based "alternate installer" (since that's reportedly just like Debian's installer, and I hadn't bothered to hook a mouse up to my server).

When you boot from the installation CD, you're presented with an installation startup screen. There are some option menus across the bottom of the screen; one of them (F4, I think, but I'm not sure) has an option "Install a command-line system", and that's what I chose. You could probably do all this with the standard Ubuntu GUI system as well, but I didn't see the point in devoting the resources (disk space, complexity, my time to figure out and use the GUI, etc.) to a GUI for a system that's going to be running headless in a rack most of the time.

1.1: Disk Partitioning and Layout

The next part of the installation process is disk partitioning and layout. In this particular server, I have two 320GB SATA drives. I decided that I wanted to run them in an RAID1 configuration, with the bulk of the space devoted to LVM for maximum flexibility in allocating disk space to virtual guests. My standard RAID1/LVM setup consists of partitioning each disk identically:

Partition Size Type Notes
1 128 MB Linux RAID auto (0xFD), bootable for md0, which will be /boot
2 2 GB Linux swap (0x82) for swap space
3 8 GB Linux RAID auto (0xFD) for md1, which will be / (root filesystem
4 (rest of disk) Linux RAID auto (0xFD) for md2, which will be the LVM physical volume (PV)

Then, you combine the physical partitions from the two disks into RAID1 ("MD") partitions:

RAID Partition Physical Partitions Format Mount Point
md0 sda1 and sdb1 ext3 filesystem /boot
md1 sda3 and sdb3 ext3 filesystem / (root)
md2 sda4 and sdb4 LVM physical volume (irrelevant)

(Note that sda2 and sdb2 do NOT get merged into a RAID partition; the kernel will use both of them directly for swap space, which does not need to be RAID-protected.)

Finally, you set up an LVM volume group (I call mine "vg0") with RAID1 partition md2 as its sole physical volume (PV). This is the volume group that you'll allocate logical volumes (LVs) out of for disk and swap space for Xen guest hosts.

1.2: Install Kernel and Software

At this point, the Ubuntu installer will walk you through installing the basic operating system kernel and software packages onto the partitions that you just laid out. When it's done, the system will reboot from the newly-installed disks, and you'll have your basic system up and running.

2: Configuration and Additional Installation

2.1: Log in and set root password

You can now log in to the system on the console using the username and password that you specified during the Ubuntu installation process. Once logged in, you can do "sudo -i" to become root, which you'll need to be for the rest of these steps.

It's apparently not standard on Ubuntu to be able to log in as root, but I like to be able to, so I immediately set a root password:

dom0# password root

2.2: Set up network interfaces

The Ubuntu installer prompts you for information that it uses to set up the first network interface (eth0), which it apparently assumes gets its address via DHCP. If you have additional interfaces, or want to use static addresses rather than DHCP, you need to edit /etc/network/interfaces:

dom0# vi /etc/network/interfaces
... (see 'man interfaces' for details, as well as the comments and examples within the file)
dom0# ifdown eth0 ; ifup eth0 ; ifup eth1

(The "ifdown" and "ifup" commands shutdown and restart the eth0 and eth1 interfaces, so that the changes you just made in /etc/network/interfaces take effect.)

2.3: Configure DNS

You probably need to edit /etc/resolv.conf to tweak your DNS client configuration:

dom0# vi /etc/resolv.conf

2.4: Ensure everything is up to date

It's likely (almost certain, in fact) that there are updates available for some of the packages that the installer installed from the CD-R. So, as soon as you've got your network working, you should bring everything up to date:

dom0# aptitude update
...
dom0# aptitude safe-upgrade
...

2.5: Install and configure NTP

I like having an accurate clock on my systems, so the next step for me is to install and configure an NTP server on the system:

dom0# vi /etc/ntp.conf

I usually take advantage of the NTP Pool Project to find a set of 4 or so NTP servers to peer with, so I put the following in my /etc/ntp.conf file:

server 0.us.pool.ntp.org
server 1.us.pool.ntp.org
server 2.us.pool.ntp.org
server 3.us.pool.ntp.org

If your ISP offers an NTP server (see if you get a response to "ping ntp.your-isp-domain.net"), you might want to add that to the list as well, since it's probably topologically "closer" to you (and thus will give you more reliable time) than the random servers you get assigned from the NTP Pool Project.

Once you're done editing your /etc/ntp.conf file, you need to restart the NTP daemon so that your configuration changes take effect:

dom0# /etc/init.d/ntp restart

2.6: Install and configure SSH

I want to be able to access and manage my server remotely via SSH, so I need to install and configure that, then restart it so that my config changes take effect:

dom0# aptitude install openssh-server
...
dom0# vi /etc/ssh/sshd_config
...
dom0# /etc/init.d/ssh restart

About all I usually change in the /etc/ssh/sshd_config file is the "Port" specification, which controls which TCP port the SSH server listens for incoming connections on. I like to move my SSH server from the standard port (22) to some other port (which I memorize). This won't stop a determined attacker, but it keeps the riff-raff from continuously rattling the doorknob.

2.7: Install Xen packages

Now we're ready to install the various Xen-related packages. Ubuntu makes this very easy by defining a special "ubuntu-xen-server" pseudo-package which brings in everything you need:

dom0# aptitude install ubuntu-xen-server

2.8: Install fixed Xen kernel

Well, almost everything you need... It turns out that there is apparently a bug in the standard Ubuntu 8.10 kernels, which keeps networking from working for Xen guest hosts. That's kind of important, but there's a patched kernel available. So, we need to get curl, and then use that to get the patched kernel, then install the patched kernel:

dom0# aptitude install curl
...
dom0# cd /root
dom0# curl -O http://www.il.is.s.u-tokyo.ac.jp/%7Ehiranotaka/linux-image-2.6.24-16-xen...
...
dom0# dpkg -i linux-image-2.6.24-16-xen_2.6.24-16.30zng1_i386.deb

(this will leave a copy of the installation package in root's home directory, /root, in case you need to do it again or something)

2.9: Disable TLS Libraries

Xen has trouble with the TLS libraries, so you need to simply disable them:

dom0# mv /lib/tls /lib/tls.disabled

2.10: Reboot into your new Xen kernel

At this point, you're ready to reboot into your new Xen kernel, and start creating guest virtual hosts

dom0# reboot

2.11: Install xen-tools

This is optional, but I find the xen-tools package very useful for creating Xen guest installations:

dom0# aptitude install xen-tools
...
dom0# vi /etc/xen-tools.conf

Among other changes in the /etc/xen-tools.conf file, you'll want to uncomment the "lvm =" line and set it to "lvm = vg0", so that disk and swap space for your guest hosts are created out of the LVM space that you set up during partitioning.

2.12: Fix xendomains problems

2.12.1: Fixing /etc/init.d/xendomains

There is a silly bug in the /etc/init.d/xendomains script, which is used to start and stop Xen guest hosts when the dom0 server starts and stops, which will keep it from working if you don't fix it. Here's a patch:

--- /etc/init.d/xendomains.orig	2008-10-06 11:08:55.000000000 -0700
+++ /etc/init.d/xendomains	2008-12-29 18:48:23.000000000 -0800
@@ -183,7 +183,7 @@
 {
     name=`echo "$1" | cut -d\  -f1`
     name=${name%% *}
-    rest=`echo "$1" | cut cut -d\  -f2-`
+    rest=`echo "$1" | cut -d\  -f2-`
     read id mem cpu vcpu state tm < <(echo "$rest")
 }

(just edit the file, search for "cut cut", and change it to simply "cut".)

2.12.2: Disable save/resume on dom0 reboot

With Xen, you should be able to "save" a running guest domain's state, and then "resume" that guest at some later time (even if the underlying dom0 host reboots in the mean time). This is handy, as it means that you can do a quick reboot of your dom0 server without having to reboot your guests; they'll simply "freeze" for a minute or so while the dom0 server reboots, then continue as they were. This "save before a dom0 shutdown, and resume after a dom0 boot" behavior has worked great in past versions of Xen, and is the default behavior of the "xendomains" script.

Unfortunately, with this version of Xen (3.3.0 on Ubuntu 8.10), it doesn't quite work... The "resumed" domains have problems, and break in subtle ways (processes start getting hung in "defunct" state rather than being harvested by their parents, for example). So, unfortunately, it's safer for xendomains to shut down the guest domains when the dom0 shuts down, and boot them when the dom0 boots. To do this, you edit the /etc/default/xendomains file and set the XENDOMAINS_SAVE= value to blank (nothing).

dom0# vi /etc/default/xendomains
... set XENDOMAINS_SAVE= to ""

3: Conclusion

This is by no means everything you'll want to do to customize your own installation of Xen 3.3 on Ubuntu 8.10 (Intrepid Ibex), but it should be enough to get you started, and to get you past a few key hurdles (needing the alternate kernel, and fixing the xendomains bugs and problems). Good luck!

Next time, I'll share some notes on how to set up a Xen dom0 that has multiple network interfaces such that guest hosts can access one or more of those networks according to their Xen configuration file, and how your Xen dom0 can do NAT for all its guest hosts.

Syndicate content