Working from Home

Headless Sun xVM VirtualBox guests via SMF

I finally had some extra time this weekend to sit down and finish an SMF manifest to support headless VirtualBox guests. Overall, I am quite pleased with the result:

The source for the service manifest and method script is available here. These files should be copied to /var/svc/manifest/application/virtualbox/ and /lib/svc/method/, respectively.

I have also created an IPS package for OpenSolaris users; if you are interested in installing the package, you will need to add my private IPS repository first:

% pfexec pkg set-authority -O http://arf.ubound.org/pkg/ arf.ubound.org

Once the repository has been added, you can then install the virtualbox-headless package:

% pfexec pkg install virtualbox-headless

Once the files are in place, you may need to import the manifest manually; since there are no default instances, svc:/system/manifest-import:default has a tendency to skip the manifest during a bulk import (this issue seems to affect the IPS package as well):

% pfexec svccfg import /var/svc/manifest/application/virtualbox/virtualbox-headless.xml

Once you have imported the manifest, it is time to add each guest you wish to manage. I have written the manifest such that a guest is identified by the name of the instance itself. For example, if I wanted to add a guest named qnx641, I would issue the following:

% pfexec svccfg -s virtualbox/headless add qnx641

You may then enable the instance via svcadm(1M):

% pfexec svcadm enable virtualbox/headless:qnx641

Two properties are provided to control the start and stop behavior of each instance:

1. The vbox/start_type property corresponds to the --type argument passed to VBoxManage startvm; by default, it is set to headless. Possible values are: gui, sdl, vrdp, and headless, however only the last two really make any sense when used with SMF.


2. The vbox/stop_method property corresponds to the argument passed to VBoxManage controlvm which is responsible for stopping the instance; by default it is set to savestate. Possible values are: pause, resume, reset, poweroff, savestate, acpipowerbutton, and acpisleepbutton

For those unfamiliar with SMF, these properties may be set by using this pattern:

% pfexec svccfg -s virtualbox/headless:qnx641
svc:/application/virtualbox/headless:qnx641> addpg vbox application
svc:/application/virtualbox/headless:qnx641> setprop vbox/stop_method = astring: "poweroff"
^D

Since this is a "wait" model service, it does not properly support a user shutting down the guest outside of svcadm(1M) invocations; the service will continue to report its status as online. In this case, a simple disable or restart will resolve the problem.

Enjoy!

dnet(7D): Integrated in onnv_123

The GLDv3 conversion for dnet(7D) has been putback. This patch provides GLDv3 and VLAN tagging for all currently supported devices, as well as link notification for those which support an MII transceiver.

The final webrev is available here: http://cr.opensolaris.org/~stallion/dnet/

Distributed Computing Humor

Those that know me well know that I have a soft spot for a certain variety of computing problems. A good friend sent the following to me earlier this morning:

Installing OpenSolaris 2009.06 on SPARC without wanboot

With the release of OpenSolaris 2009.06, SPARC is now an officially supported platform.

Unfortunately, older SPARCs with an OBP revision less than 4.17 are not currently supported due to lack of wanboot in the firmware. It is still possible to use wanboot on these machines, however it requires SXCE install media.

To add insult to injury, the AI client miniroot (understandably) makes a number of assumptions about the environment, particularly with respect to network interface configuration (specifically netbootinfo and dhcpagent). For the curious, these issues have been documented in Bugzilla under Bug 9549.

What follows is a workaround to install OpenSolaris on older SPARC hardware using wanboot from the SXCE install media. This process assumes that you have correctly configured the AI server and client as documented in the OpenSolaris Automated Installer Guide.

To get started, you will need to download and burn the SXCE install media (snv_111 or higher is required due to a number of recent fixes to wanboot). Place the install media into your DVD drive, drop into the PROM, and issue:

{0} ok boot cdrom -F wanboot -o dhcp

At this point, the system will boot wanboot from the install media and pick up its configuration from DHCP.

Eventually, the boot process will fail with an error:

Rebooting with command: boot cdrom -F wanboot -o dhcp
Boot device: /pci@8,700000/scsi@6/disk@6,0:f  File and args: -F wanboot -o dhcp
<time unavailable> wanboot info: WAN boot messages->console
<time unavailable> wanboot info: configuring /pci@8,600000/pci@1/network@0

1000 Mbps FDX Link up
<time unavailable> wanboot info: Starting DHCP configuration
<time unavailable> wanboot info: DHCP configuration succeeded

<time unavailable> wanboot info: Default net-config-strategy: dhcp
<time unavailable> wanboot progress: wanbootfs: Read 366 of 366 kB (100%)
<time unavailable> wanboot info: wanbootfs: Download complete
Sat Jul 11 01:40:14 wanboot progress: miniroot: Read 175868 of 175868 kB (100%)
Sat Jul 11 01:40:14 wanboot info: miniroot: Download complete
SunOS Release 5.11 Version snv_111b 64-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
strplumb: open /devices/pseudo/clone@0:sd failed: 19
Hostname: opensolaris
Remounting root read/write
Probing for device nodes ...
Preparing automated install image for use
The AI image will be retrieved from /export/aiserver/osol-0906-ai-sparc/ directory
Downloading solaris.zlib archive
--18:41:16--  http://10.8.0.8:5555/export/aiserver/osol-0906-ai-sparc//solaris.zlib
           => `/tmp/solaris.zlib'
Connecting to 10.8.0.8:5555... failed: Network is unreachable.
FAILED
Requesting System Maintenance Mode
(See /lib/svc/share/README for more information.)
Console login service(s) cannot run

Enter user name for system maintenance (control-d to bypass):

At this point you will need to login as root (the password will be empty) and configure your network interfaces manually:

# ifconfig -a plumb
# ifconfig <interface> dhcp

You will also need to enable DNS; this is required for pkg(1) to locate and install packages. It should be noted that the wanboot miniroot is very minimal; you do not have access to some of the more common commands such as cp and rmdir. Also, the miniroot is using dcfs(7FS) which means some additional steps need to be taken to modify the filesystem contents:

# cat > /etc/resolv.conf
nameserver <address>
^D
# rm /etc/nsswitch.conf
# cat /etc/nsswitch.dns > /etc/nsswitch.conf

At this point, the network has been configured correctly, however we will need to clean up state to allow the svc:/system/filesystem/root:live-media service to continue booting:

# umount /etc/netboot
# rm -rf /etc/netboot
# umount /tmp

Before clearing the svc:/system/filesystem/root:live-media service, I had to disable a couple of services for a clean boot. The svc:/platform/sun4u/dscp:default and svc:/platform/sun4u/sckmd:default services were not necessary for my hardware and if left enabled, caused the install process to fail. These services are safe to disable unless you have the hardware referenced in the services:

# svcadm disable dscp
# svcadm disable sckmd

You are now free to clear the svc:/system/filesystem/root:live-media service:

# svcadm clear root:live-media
Remounting root read/write
Probing for device nodes ...
Preparing automated install image for use

Before logging out, you will also want to clear the svc:/network/dns/multicast:default service to ensure service discovery works correctly:

# svcadm clear dns/multicast

Once you logout from the maintenance shell the AI process will continue:

# logout
The AI image will be retrieved from /export/aiserver/osol-0906-ai-sparc/ directory
Downloading solaris.zlib archive
--18:43:25--  http://10.8.0.8:5555/export/aiserver/osol-0906-ai-sparc//solaris.zlib
           => `/tmp/solaris.zlib'
Connecting to 10.8.0.8:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 83,334,656 (79M) [text/plain]

100%[====================================>] 83,334,656    50.36M/s

18:43:26 (50.28 MB/s) - `/tmp/solaris.zlib' saved [83334656/83334656]

Downloading solarismisc.zlib archive
--18:43:26--  http://10.8.0.8:5555/export/aiserver/osol-0906-ai-sparc//solarismisc.zlib
           => `/tmp/solarismisc.zlib'
Connecting to 10.8.0.8:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3,857,408 (3.7M) [text/plain]

100%[====================================>] 3,857,408     --.--K/s

18:43:26 (47.53 MB/s) - `/tmp/solarismisc.zlib' saved [3857408/3857408]

--18:43:26--  http://10.8.0.8:5555/export/aiserver/osol-0906-ai-sparc//install.conf
           => `/tmp/install.conf'
Connecting to 10.8.0.8:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 61 [text/plain]

100%[====================================>] 61            --.--K/s

18:43:26 (2.44 MB/s) - `/tmp/install.conf' saved [61/61]

Done mounting automated install image
Configuring devices.
Reading ZFS config: done.

opensolaris console login: Service discovery phase initiated
Service name to look up: 0906sparc
Service discovery finished successfully
Process of obtaining configuration manifest initiated
Configuration manifest obtained
Automated Installation started
The progress of the Automated Installation can be followed by viewing the logfile at /tmp/install_log

At this point you should be good to go - Enjoy!

Building ONNV uts on OpenSolaris 2009.06

Overview
Earlier this month, I converted my primary Opteron server running 10+ zones on Solaris 10 over to OpenSolaris 2009.06. With a more modern environment available on my best hardware, I decided to create a zone specifically for hosting builds for Source Juicer packages and OS/Net.

Since the first release of Project Indiana last year, a number of people have been working toward making ON development possible. In April, Liane Praza made the `osnet' package available in the /dev repository along with updated instructions on the Project Indiana project site. With the release of a number of missing packages to the /extras repository earlier this year and a little elbow grease it is now possible to build ON from within Project Indiana with no need to install auxiliary packages from SXCE.

Unfortunately the `osnet' package is only available from the /dev repository which leaves those following /release out in the cold. Also, `osnet' is a one-size-fits-all solution to dealing with ON dependencies; everything is installed unconditionally (even some of the heavier bits, such as both versions of Apache, and PostgreSQL).

Since I spend about 99.9% of my time hacking in uts, I wanted a lighter-weight solution.

There are a number of caveats to using a build environment such as this; some obvious, some not:

• Be aware that you can build any version of ON which supports the dependencies available in /release.


• If you plan to do more than just build kernel modules, built modules should match the installed kernel version.


• An snv_108 (or greater) host is required to avoid Bug 6798733; It's worth mentioning that Edward Pilatowicz has posted a workaround for builds prior to snv_108.

Setup
In my case, I was creating up a build environment within a zone. If you wish to do the same, you must ensure that the SUNWcar and SUNWkvm packages are installed:

% pfexec pkg install SUNWcar SUNWkvm

The following packages are required at a minimum to build uts:

% pfexec pkg install SUNWperl584usr SUNWperl-xml-parser SUNWhea SUNWsprot \
  SUNWbtool SUNWtss SUNWgcc

If you happen to be outside of the Sun firewall, you will also want to install Mercurial and wget. For the remainder of this post, it is assumed that this is the case:

% pfexec pkg install SUNWmercurial SUNWwget

Unfortunately SUNWonbld is only made available as an IPS package in the /dev repository. If you prefer to use a local install for day-to-day development, you will need to install SVR4 packaging support and download the SUNWonbld package from the OpenSolaris Download Center. If you only intend to use nightly(1), then this step may be skipped safely.

% pfexec pkg install SUNWpkgcmds
% cd /var/tmp; wget http://dlc.sun.com/osol/on/downloads/current/SUNWonbld.i386.tar.bz2
% bzcat SUNWonbld.i386.tar.bz2 | tar xf -
% pfexec pkgadd -d onbld SUNWonbld

Update: It appears that SUNWonbld is now available as an IPS package in the /release repository.

You will need to download and install the Sun Studio 12 tarball provided by the Tools Community. Also, you must ensure that /opt/SUNWspro points to your Sun Studio install. Please, please do not attempt to build uts with Sun Studio Express - you will not be pleasantly surprised.

% mkdir -p /opt/SunStudio12; cd /opt/SunStudio12
% gzcat sunstudio12-ii-20081010-sol-x86.tar.gz | tar xf -
% ln -sf /opt/SunStudio12 /opt/SUNWspro

At this point you should update your PATH; this is absolutely critical. If your PATH is not setup correctly, uts may fail to build. If you did not install SUNWonbld locally (see above), then you may drop the first two paths below:

% PATH=/opt/onbld/bin:/opt/onbld/bin/`uname -p`:\
  /opt/SUNWspro/bin:/usr/bin:/usr/sbin:/usr/ccs/bin; export PATH

At this point, you are ready to clone the onnv-gate:

% hg clone ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate

Change your working directory to the freshly cloned gate, then download and unpack the closed binaries from the OpenSolaris Download Center:

% wget http://dlc.sun.com/osol/on/downloads/current/on-closed-bins.i386.tar.bz2
% bzcat on-closed-bins.i386.tar.bz2 | tar xf -

You will also need to copy and edit opensolaris.sh. Pay especial attention to GATE, CODEMGR_WS, and STAFFER. Don't forget to remove 't' from NIGHTLY_OPTIONS if you have SUNWonbld installed locally. I typically alter maxjobs() to a more reasonable value - the default tends to be a little aggressive on a shared system.

% cd onnv-gate; cp usr/src/tools/env/opensolaris.sh .
% vi opensolaris.sh

Finally, issue bldenv(1) to setup your build environment. If you downloaded the debug closed bins, make sure that you pass -d for a debug build:

% bldenv -d opensolaris.sh

Building the Kernel
At this point, building the kernel is pretty straightforward. In order to keep dmake from complaining on each invocation, you will want to set the DMAKE_MODE environment variable if you have not already done so in your .dmakerc:

% DMAKE_MODE=parallel; export DMAKE_MODE

If the SUNWonbld package was not installed earlier (see above), or you prefer to use the tools provided in the gate, you will need to install them first:

% cd $SRC/tools; dmake install

Once the build tools are available, you will need to issue a `dmake setup' from the top level source directory:

% cd $SRC; dmake setup

At this point you are free to build uts:

% cd $SRC/uts; dmake all

As always, YMMV.

dnet(7D): Up for Review

The GLDv3 conversion for dnet(7D) is finally up for review. This patch provides GLDv3 and VLAN tagging for all currently supported devices, as well as link notification for those which support an MII transceiver.

The webrev is available here: http://cr.opensolaris.org/~stallion/dnet/latest/

A followup patch will be provided to support Project Brussels.

Not Dead Yet!

It's been quite some time since I have made a post - things have been pretty hectic around here lately.

Like most hardware geeks, each year I go through a round of hardware and software upgrades - this year was no exception. My long-time girlfriend (now fiancee) and I also moved in with each other around this same time. I am sure most of you can already see where this is going...

I sorely needed to upgrade the main switch to one that supported jumbo frames and VLAN tagging (particularly for gigabit UTP testing). Storage was also in pretty rough shape, and I needed to replace 6 of the 8 disks in the rack. Solaris 10 5/08 had been released (early) in April, further dating the 11/06 install on my primary server (which needed a complete overhaul). Compounding these issues was the absolute need for a new UPS; the Netra 20 was overloading the existing one on a regular basis.

I took the plunge and decided to upgrade everything at once despite already being overwhelmed with the move and the stress that comes with the merging of two homes.

I found out very quickly that my time is finite. Both the move and the upgrade was long and trying. There were (of course) a number of issues with the new hardware and a few kinks to iron out with the new software installs. Work with ON suffered due to a lack of time, and the extended downtime also affected others who use the hardware remotely. Time spent messing with the upgrade also put a strain on relationships I had (naively) not expected. On top of all of that, I managed to get pretty sick, which placed even more stress on my relationships and prolonged downtime even further.

That said, everything is finally finished, relationships patched up, bodies healed, and everything is now approaching some modicum of normalcy. I am finally able to return to work with ON (post forthcoming).

I believe this post (and the last month, for that matter) can be summed up as follows:
moving + marriage + hardware + software = A Really Bad Idea(TM)

Update: For the curious, I have uploaded a few pictures of the upgrades here.

Cassini Emancipation

Last month I purchased a Sun Netra 20/T4 to do some sparcv9 driver development. Since I upgraded all of the equipment last year to gigabit UTP, I also grabbed a Sun GigaSwift (501-5902) to go with it.

Out of curiosity, I dug through onnv-gate to see if the ce driver source had been opened yet; alas it had not. After talking with some Sun folk a bit, it seemed that not only were there no plans to open source Cassini (yet), but much of the code wasn't in the best of shape. Surprisingly, the ce driver still implements the DLPI interface directly rather than making use of the newer frameworks available.

After speaking with John Sonnenschein (who leads the Emancipation Project) and Jason King, we decided to begin an effort to emancipate the Cassini driver. The new driver will be a 'clean room' implementation and will (of course) be open source. It will also support the newer frameworks, namely: GLDv3/Project Nemo, and Project Brussels rather than implementing DLPI directly. As far as naming goes, we decided to go with cge (Cassini Gigabit Ethernet) since it is more in line with the current trend in device naming. Fortunately, with interface vanity naming being integrated in snv_83, users migrating from the original ce driver should have a much easier time of it.

Update: Cyril Plisko has joined the effort. Welcome!

Mercurial and SUNWonbld

For those developers (like myself) whom are external to Sun working with ON, managing change can be something of a pain - particularly for projects which span multiple files. Fortunately, some very intelligent folk at Sun have developed a number of scripts which vastly simplify the review process. Chief among these is webrev (which has traditionally been used on TeamWare workspaces).

Since mercurial was selected as the SCM of choice for external ON development, many of the scripts provided in SUNWonbld are not available (yet). One of the primary goals of the SCM Migration Project is to bridge this gap.

Essentially, the stable SUNWonbld packages offered on the main ON download page is not compatible with mercurial. In order to get mercurial tool support (i.e.: webrev), you must download the SUNWonbld-latest package from the SCM Migration Project page.

A couple things to keep in mind:

• Make sure the most recent SUNWmercurial package is installed (currently, 0.95 is the latest). Check the Source Code Management Tool Downloads page for details.


• Read the webrev man pages thoroughly. A common mistake is to pass a specific file to webrev; this is incorrect. webrev expects a file (denoted as file-list in the man page) which contains a list of files to work against.

Many thanks to James McPherson and Rich Lowe in #onnv-scm for their many suggestions and patience!