Iconoclastic Tendencies

efe(7D) integrated into illumos

2010-10-28T10:18:00.005-05:00

[stallion@titan]:/export/illumos-clone> hg outgoing -v
running ssh anonhg@hg.illumos.org "hg -R illumos-gate serve --stdio"
comparing with ssh://anonhg@hg.illumos.org/illumos-gate
searching for changes

changeset:   13220:0e9358627086
tag:         tip
user:        Steven Stallion 
date:        Sun Oct 24 01:34:25 2010 -0400

description:
        9 Provide an open source implementation of spwr(7D)
        Reviewed by: garrett@nexenta.com, richlowe@richlowe.net
        Approved by: garrett@nexenta.com

modified:
   usr/src/uts/common/Makefile.files
   usr/src/uts/common/Makefile.rules
   usr/src/uts/intel/Makefile.intel.shared
   usr/src/uts/intel/os/master
   usr/src/uts/sparc/Makefile.sparc.shared
added:
   usr/src/pkg/manifests/driver-network-efe.mf
   usr/src/uts/common/io/efe/THIRDPARTYLICENSE
   usr/src/uts/common/io/efe/THIRDPARTYLICENSE.descrip
   usr/src/uts/common/io/efe/efe.c
   usr/src/uts/common/io/efe/efe.h
   usr/src/uts/intel/efe/Makefile
   usr/src/uts/sparc/efe/Makefile

The final webrev can be found here.

newsprint.vim: A (pleasant) monochromatic color scheme for Vim

2010-10-21T15:27:00.008-05:00

I spend over 90% of my time sitting in an editor.

Having switched to a dark-on-light terminal scheme years ago I could never find a color scheme in which I could be productive using a light background, so I stuck with my ancient light-on-dark color scheme for console editing. The effect is actually quite jarring; I use a number of terminal sessions during the day for any number of tasks (including editing). Perhaps it's simply a sign of getting older, but I've found that switching rapidly between sessions can not only cause eyestrain, it also wastes time waiting for my eyes to adjust to a radically different background.

I created a simple color scheme that works well with dark-on-light displays. I drew some inspiration from the google color scheme (notably the use of underlining for language keywords). Rather than saturate the screen with color, I opted for a simpler scheme using a grayscale pallete with bolding and underlining for emphasis. This scheme should work equally well on both color and GUI terminals.

The color scheme can be found here. Simply download and place under $HOME/.vim/colors/. You can select the scheme by issuing: :colorscheme newsprint once in Vim.

Obligatory screenshot:

Overall, I'm pleased with the result. Moving between sessions is a smoother and faster experience. While this scheme probably isn't suitable for long-term editing, it's certainly better for quick edits made from the console.

Enjoy!

"Who forgot to pay the water bill?"

2010-08-20T09:50:00.004-05:00

As mentioned by Garrett D'Amore yesterday, the "tap" has indeed been turned off; code pushes are no longer being made to the ONNV repository. For the first time in four years, I was greeted with the following:

[stallion@titan]:/export/onnv-clone> hg pull
pulling from ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate
searching for changes
no changes found

OpenSolaris is Dead.

2010-08-13T10:07:00.009-05:00

What follows is an email sent internally to Oracle Solaris Engineers which describes Oracle's true intentions toward the OpenSolaris project and the future of Oracle Solaris.

This concludes over four years that I (and many other external contributors) have worked on the OpenSolaris project. This is a terrible sendoff for countless hours of work - for quality software which will now ship as an Oracle product that we (the original authors) can no longer obtain on an unrestricted basis.

I can only maintain that the software we worked on was for the betterment of all, not for any one company's bottom line. This is truly a perversion of the open source spirit.

Solaris Engineering,

Today we are announcing a set of decisions regarding the path to
Solaris 11, and answering key pending questions on open source, open
development, software and binary licenses, and how developers and
early adopters will be able to use Solaris 11 technology before its
release in 2011.

As you all know, the term “OpenSolaris” has been used colloquially to
refer to any or all of a collection of source code, a development
model, a web site, a logo, a binary release, a source license, a
community, and many other related things. So it’s taken a while to go
over each issue from an organizational and business perspective, and
align on the correct next step. Therefore, please take the time to
read all of the detail here carefully. We’ll discuss our strategy
first, and then the decisions and changes to our policies and
processes that implement that strategy.

Solaris Strategy
----------------------

Solaris is the #1 Enterprise Operating System. We have the leading
share of business applications on Solaris today, including both SPARC
and x64. We have more than twice the application base of AIX and HP-
UX combined. We have a brand that stands for innovation, quality,
security, and trust, built on our 20-year investment in Solaris
operating system engineering.

From a business perspective, the purpose of our investment in Solaris
engineering is to drive our overall server business, including both
SPARC and x64, and to drive business advantages resulting from
integration of multiple components in the Oracle portfolio. This
includes combining our servers with our storage, our servers with our
switches, Oracle applications with Solaris, and the effectiveness of
the service experience resulting from these combinations. All
together, Solaris drives aggregate business measured in many billions
of dollars, with significant growth potential.

We are increasing investment in Solaris, including hiring operating
system expertise from throughout the industry, as a sign of our
commitment to these goals. Solaris is not something we outsource to
others, it is not the assembly of someone else’s technology, and it is
not a sustaining-only product. We expect the top operating systems
engineers in the industry, i.e. all of you, to be creating and
delivering innovations that continue to make Solaris unique,
differentiated, and valuable to our customers, and a unique asset of
our business.

Solaris must stand alone as a best-of-breed technology for Oracle’s
enterprise customers. We want all of them to think “If this has to
work, then it runs on Solaris.” That’s the Solaris brand. That is
where our scalability to more than a few sockets of CPU and gigabytes
of DRAM matters. That is why we reliably deliver millions of IOPS of
storage, networking, and Infiniband. That is why we have unique
properties around file and data management, security and namespace
isolation, fault management, and observability. And we also want our
customers to know that Solaris is and continues to be a source of new
ideas and new technologies-- ones that simplify their business and
optimize their applications. That’s what made Solaris 10 the most
innovative operating system release ever. And that is the same focus
that will drive a new set of innovations in Solaris 11.

For Solaris to stand alone as the best-of-breed operating system in
Oracle’s complete and open portfolio, it must run well on other server
hardware and execute everyone’s applications, while delivering unique
optimizations for our hardware and our applications. That is the
central value proposition of Oracle’s complete, open, and integrated
strategy. And these are complementary and not contradictory goals
that we will achieve through proper design and engineering.

The growth opportunity for Solaris has never been greater. As one
example, Solaris is used by about 40% of Oracle’s enterprise
customers, which means we have a 60% growth opportunity in our top
customers alone. In absolute numbers, there are 130,000 Oracle
customers in North America alone who don’t use our servers and storage
yet, and a global customer base of 350,000 (the prior Sun base was
~35,000). That’s a huge opportunity we can go attack as a combined
company that will increase Solaris adoption and the overall Hardware
server revenue. Our success will also increase the amount of effort
ISVs exert optimizing their applications for Solaris.

We will continue to grow a vibrant developer and system administrator
community for Solaris. Delivery of binary releases, delivery of APIs
in source or binary form, delivery of open source code, delivery of
technical documentation, and engineering of upstream contributions to
common industry technologies (such as Apache, Perl, OFED, and many,
many others) will be part of that activity. But we will also make
specific decisions about why and when we do those things, following
two core principles: (1) We can’t do everything. The limiting factor
is our engineering bandwidth measured in people and time. So we have
to ensure our top priority is driving delivery of the #1 Enterprise
Operating System, Solaris 11, to grow our systems business; and (2) We
want the adoption of our technology and intellectual property to
accelerate our overall goals, yet not permit competitors to derive
business advantage (or FUD) from our innovations before we do.

We are using our investment in core Solaris innovation and engineering
to drive multiple businesses, through multiple product lines. This
already includes our Solaris operating system for Enterprise, and our
ZFS Storage product line, and will soon include other Oracle
products. This strategy is all about creating more value from a set
of common software investments: it makes everything you do more
valuable and used by more people worldwide. It also means you as an
individual engineer or manager have an even greater responsibility to
understand the broader business and technical contexts in which your
engineering is deployed.

Solaris Decisions
------------------------

We will continue to use the CDDL license statement in nearly all
Solaris source code files. We will not remove the CDDL from any files
in Solaris to which it already applies, and new source code files that
are created will follow the current policy regarding applying the CDDL
(simply, that usr/src files will have the CDDL, and the very small
minority of files in usr/closed might not have it). Use of other open
licenses in non-ON consolidations (e.g. GPL in the Desktop area) will
also continue. As before, requests to change the license associated
with source code are case-by-case decisions.

We will distribute updates to approved CDDL or other open source-
licensed code following full releases of our enterprise Solaris
operating system. In this manner, new technology innovations will
show up in our releases before anywhere else. We will no longer
distribute source code for the entirety of the Solaris operating
system in real-time while it is developed, on a nightly basis.

Anyone who is consuming Solaris code using the CDDL, whether in pieces
or as a part of the OpenSolaris source distribution or a derivative
thereof, would therefore be able to consume any updates we release at
that time, under the terms of the CDDL, LGPL, or whatever license
applies.

We will have a technology partner program to permit our industry
partners full access to the in-development Solaris source code through
the Oracle Technology Network (OTN). This will include both early
access to code and binaries, as well as contributions to us where that
is appropriate. All such partnerships will be evaluated on a case-by-
case basis, but certainly our core, existing technology partnerships,
such as the one with Intel, are examples of valued participation.

We will encourage and listen to any and all license requests for
Solaris technology, either in part or in whole. All such requests
will be evaluated on a case-by-case basis, but we believe there are
many complementary areas where new partnership opportunities exist to
expand use of our IP.

We will continue active open development, including upstream
contributions, in specific areas that accelerate our overall Solaris
goals. Examples include our activities around Gnome and X11, IPS
packaging, and our work to optimize ecosystems like Apache, OpenSSL,
and Perl on Solaris.

We will deliver technical design information, in the form of
documentation, design documents, and source code descriptions, through
our OTN presence for Solaris. We will no longer post advance
technical descriptions of every single ARC case by default, indicating
what technical innovations might be present in future Solaris
releases. We can at any time make a specific decision to post advance
technical information for any project, when it serves a particular
useful need to do so.

We will have a Solaris 11 binary distribution, called Solaris 11
Express, that will have a free developer RTU license, and an optional
support plan. Solaris 11 Express will debut by the end of this
calendar year, and we will issue updates to it, leading to the full
release of Solaris 11 in 2011.

All of Oracle’s efforts on binary distributions of Solaris technology
will be focused on Solaris 11. We will not release any other binary
distributions, such as nightly or bi-weekly builds of Solaris
binaries, or an OpenSolaris 2010.05 or later distribution. We will
determine a simple, cost-effective means of getting enterprise users
of prior OpenSolaris binary releases to migrate to S11 Express.

We will have a Solaris 11 Platinum Customer Program, including direct
engineering involvement and feedback, for customers using our Solaris
11 technology. We will be asking all of you to participate in this
endeavor, bringing with us the benefit of previous Sun Platinum
programs, while utilizing the much larger megaphone that is available
to us now as a combined company.

We look forward to everyone’s continued work on Solaris 11. Our goal
is simply to make it the best and most important release of Solaris
ever.

-Mike Shapiro, Bill Nesheim, Chris Armes

In Athens!

2010-08-05T11:10:00.001-05:00

As noted previously, I took a job working with Coraid, Inc. back in May. My family and I have finally completed relocating down to beautiful Athens, GA. We are all very excited to be part of the culture here and are looking forward to many happy years.

spwr(7D) is dead. Long live efe(7D)!

2010-05-19T18:41:00.001-05:00

PSARC 2010/168 (EOF spwr) will officially remove support for SMC EtherPower II fast ethernet controllers from ON. Fortunately, the Emancipation project provides an alternative for users whom still require support for these devices.

The efe(7D) driver is a open source rewrite of the previously closed spwr(7D) driver. It supports both GLDv3 and Project Brussels interfaces and supports full 802.1Q VLANs and link aggregation. The driver supports both SPARC and x86.

The current version of the driver requires at least snv_134. Once the 2010.0x release ships and development builds resume, a new driver release will be made to support the committed GLDv3 interfaces putback in snv_136 and IPS.

OSOLefe-20100517-sparc.tar.bz2
OSOLefe-20100517-i386.tar.bz2

The source for the driver may be found in the Emancipation driver-gate:
http://src.opensolaris.org/source/xref/emancipation/driver-gate/efe/

Enjoy!

It's Official...

2010-05-08T14:25:00.002-05:00

About a month ago I accepted a position at Coraid, Inc. as a Solaris kernel developer. My primary focus will be driver development for the company's AoE-based storage HBAs and other related hardware.

After saying my goodbyes in St. Louis last week, I drove down to Athens, GA for two weeks of orientation and team building. I have to say, this last week at Coraid has been a gas - the engineering team here is astounding and leadership goes a long way toward making this a friendly and productive environment in which to work.

Everyone here seems pretty keen on what Solaris/OpenSolaris has to offer (particularly with respect to storage) - I am definitely looking forward to the road ahead.

Who knows? Maybe OpenSolaris will get a little AoE lovin' after all...

Working from Home

2009-11-17T15:42:00.004-06:00

Headless Sun xVM VirtualBox guests via SMF

2009-11-01T18:57:00.017-06:00

I finally had some extra time this weekend to sit down and finish an SMF manifest to support headless VirtualBox guests. Overall, I am quite pleased with the result:

The source for the service manifest and method script is available here. These files should be copied to /var/svc/manifest/application/virtualbox/ and /lib/svc/method/, respectively.

I have also created an IPS package for OpenSolaris users; if you are interested in installing the package, you will need to add my private IPS repository first:

% pfexec pkg set-authority -O http://arf.ubound.org/pkg/ arf.ubound.org

Once the repository has been added, you can then install the virtualbox-headless package:

% pfexec pkg install virtualbox-headless

Once the files are in place, you may need to import the manifest manually; since there are no default instances, svc:/system/manifest-import:default has a tendency to skip the manifest during a bulk import (this issue seems to affect the IPS package as well):

% pfexec svccfg import /var/svc/manifest/application/virtualbox/virtualbox-headless.xml

Once you have imported the manifest, it is time to add each guest you wish to manage. I have written the manifest such that a guest is identified by the name of the instance itself. For example, if I wanted to add a guest named qnx641, I would issue the following:

% pfexec svccfg -s virtualbox/headless add qnx641

You may then enable the instance via svcadm(1M):

% pfexec svcadm enable virtualbox/headless:qnx641

Two properties are provided to control the start and stop behavior of each instance:

The vbox/start_type property corresponds to the --type argument passed to VBoxManage startvm; by default, it is set to headless. Possible values are: gui, sdl, vrdp, and headless, however only the last two really make any sense when used with SMF.

The vbox/stop_method property corresponds to the argument passed to VBoxManage controlvm which is responsible for stopping the instance; by default it is set to savestate. Possible values are: pause, resume, reset, poweroff, savestate, acpipowerbutton, and acpisleepbutton

For those unfamiliar with SMF, these properties may be set by using this pattern:

% pfexec svccfg -s virtualbox/headless:qnx641
svc:/application/virtualbox/headless:qnx641> addpg vbox application
svc:/application/virtualbox/headless:qnx641> setprop vbox/stop_method = astring: "poweroff"
^D

Since this is a "wait" model service, it does not properly support a user shutting down the guest outside of svcadm(1M) invocations; the service will continue to report its status as online. In this case, a simple disable or restart will resolve the problem.

Enjoy!

dnet(7D) integrated into ON

2009-08-19T00:37:00.004-05:00

The GLDv3 conversion for dnet(7D) has been putback. This patch provides GLDv3 and VLAN tagging for all currently supported devices, as well as link notification for those which support an MII transceiver.

The final webrev is available here.

Distributed Computing Humor

2009-07-22T12:48:00.006-05:00

Those that know me well know that I have a soft spot for a certain variety of computing problems. A good friend sent the following to me earlier this morning:

Installing OpenSolaris 2009.06 on SPARC without wanboot

2009-07-11T22:46:00.001-05:00

With the release of OpenSolaris 2009.06, SPARC is now an officially supported platform.

Unfortunately, older SPARCs with an OBP revision less than 4.17 are not currently supported due to lack of wanboot in the firmware. It is still possible to use wanboot on these machines, however it requires SXCE install media.

To add insult to injury, the AI client miniroot (understandably) makes a number of assumptions about the environment, particularly with respect to network interface configuration (specifically netbootinfo and dhcpagent). For the curious, these issues have been documented in Bugzilla under Bug 9549.

What follows is a workaround to install OpenSolaris on older SPARC hardware using wanboot from the SXCE install media. This process assumes that you have correctly configured the AI server and client as documented in the OpenSolaris Automated Installer Guide.

To get started, you will need to download and burn the SXCE install media (snv_111 or higher is required due to a number of recent fixes to wanboot). Place the install media into your DVD drive, drop into the PROM, and issue:

{0} ok boot cdrom -F wanboot -o dhcp

At this point, the system will boot wanboot from the install media and pick up its configuration from DHCP.

Eventually, the boot process will fail with an error:

Rebooting with command: boot cdrom -F wanboot -o dhcp
Boot device: /pci@8,700000/scsi@6/disk@6,0:f  File and args: -F wanboot -o dhcp
<time unavailable> wanboot info: WAN boot messages->console
<time unavailable> wanboot info: configuring /pci@8,600000/pci@1/network@0

1000 Mbps FDX Link up
<time unavailable> wanboot info: Starting DHCP configuration
<time unavailable> wanboot info: DHCP configuration succeeded

<time unavailable> wanboot info: Default net-config-strategy: dhcp
<time unavailable> wanboot progress: wanbootfs: Read 366 of 366 kB (100%)
<time unavailable> wanboot info: wanbootfs: Download complete
Sat Jul 11 01:40:14 wanboot progress: miniroot: Read 175868 of 175868 kB (100%)
Sat Jul 11 01:40:14 wanboot info: miniroot: Download complete
SunOS Release 5.11 Version snv_111b 64-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
strplumb: open /devices/pseudo/clone@0:sd failed: 19
Hostname: opensolaris
Remounting root read/write
Probing for device nodes ...
Preparing automated install image for use
The AI image will be retrieved from /export/aiserver/osol-0906-ai-sparc/ directory
Downloading solaris.zlib archive
--18:41:16--  http://10.8.0.8:5555/export/aiserver/osol-0906-ai-sparc//solaris.zlib
           => `/tmp/solaris.zlib'
Connecting to 10.8.0.8:5555... failed: Network is unreachable.
FAILED
Requesting System Maintenance Mode
(See /lib/svc/share/README for more information.)
Console login service(s) cannot run

Enter user name for system maintenance (control-d to bypass):

At this point you will need to login as root (the password will be empty) and configure your network interfaces manually:

# ifconfig -a plumb
# ifconfig <interface> dhcp

You will also need to enable DNS; this is required for pkg(1) to locate and install packages. It should be noted that the wanboot miniroot is very minimal; you do not have access to some of the more common commands such as cp and rmdir. Also, the miniroot is using dcfs(7FS) which means some additional steps need to be taken to modify the filesystem contents:

# cat > /etc/resolv.conf
nameserver <address>
^D
# rm /etc/nsswitch.conf
# cat /etc/nsswitch.dns > /etc/nsswitch.conf

At this point, the network has been configured correctly, however we will need to clean up state to allow the svc:/system/filesystem/root:live-media service to continue booting:

# umount /etc/netboot
# rm -rf /etc/netboot
# umount /tmp

Before clearing the svc:/system/filesystem/root:live-media service, I had to disable a couple of services for a clean boot. The svc:/platform/sun4u/dscp:default and svc:/platform/sun4u/sckmd:default services were not necessary for my hardware and if left enabled, caused the install process to fail. These services are safe to disable unless you have the hardware referenced in the services:

# svcadm disable dscp
# svcadm disable sckmd

You are now free to clear the svc:/system/filesystem/root:live-media service:

# svcadm clear root:live-media
Remounting root read/write
Probing for device nodes ...
Preparing automated install image for use

Before logging out, you will also want to clear the svc:/network/dns/multicast:default service to ensure service discovery works correctly:

# svcadm clear dns/multicast

Once you logout from the maintenance shell the AI process will continue:

# logout
The AI image will be retrieved from /export/aiserver/osol-0906-ai-sparc/ directory
Downloading solaris.zlib archive
--18:43:25--  http://10.8.0.8:5555/export/aiserver/osol-0906-ai-sparc//solaris.zlib
           => `/tmp/solaris.zlib'
Connecting to 10.8.0.8:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 83,334,656 (79M) [text/plain]

100%[====================================>] 83,334,656    50.36M/s

18:43:26 (50.28 MB/s) - `/tmp/solaris.zlib' saved [83334656/83334656]

Downloading solarismisc.zlib archive
--18:43:26--  http://10.8.0.8:5555/export/aiserver/osol-0906-ai-sparc//solarismisc.zlib
           => `/tmp/solarismisc.zlib'
Connecting to 10.8.0.8:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3,857,408 (3.7M) [text/plain]

100%[====================================>] 3,857,408     --.--K/s

18:43:26 (47.53 MB/s) - `/tmp/solarismisc.zlib' saved [3857408/3857408]

--18:43:26--  http://10.8.0.8:5555/export/aiserver/osol-0906-ai-sparc//install.conf
           => `/tmp/install.conf'
Connecting to 10.8.0.8:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 61 [text/plain]

100%[====================================>] 61            --.--K/s

18:43:26 (2.44 MB/s) - `/tmp/install.conf' saved [61/61]

Done mounting automated install image
Configuring devices.
Reading ZFS config: done.

opensolaris console login: Service discovery phase initiated
Service name to look up: 0906sparc
Service discovery finished successfully
Process of obtaining configuration manifest initiated
Configuration manifest obtained
Automated Installation started
The progress of the Automated Installation can be followed by viewing the logfile at /tmp/install_log

At this point you should be good to go - Enjoy!

Building ONNV uts on OpenSolaris 2009.06

2009-06-15T23:21:00.001-05:00

Overview
Earlier this month, I converted my primary Opteron server running 10+ zones on Solaris 10 over to OpenSolaris 2009.06. With a more modern environment available on my best hardware, I decided to create a zone specifically for hosting builds for Source Juicer packages and OS/Net.

Since the first release of Project Indiana last year, a number of people have been working toward making ON development possible. In April, Liane Praza made the `osnet' package available in the /dev repository along with updated instructions on the Project Indiana project site. With the release of a number of missing packages to the /extras repository earlier this year and a little elbow grease it is now possible to build ON from within Project Indiana with no need to install auxiliary packages from SXCE.

Unfortunately the `osnet' package is only available from the /dev repository which leaves those following /release out in the cold. Also, `osnet' is a one-size-fits-all solution to dealing with ON dependencies; everything is installed unconditionally (even some of the heavier bits, such as both versions of Apache, and PostgreSQL).

Since I spend about 99.9% of my time hacking in uts, I wanted a lighter-weight solution.

There are a number of caveats to using a build environment such as this; some obvious, some not:

Be aware that you can build any version of ON which supports the dependencies available in /release.

If you plan to do more than just build kernel modules, built modules should match the installed kernel version.

An snv_108 (or greater) host is required to avoid Bug 6798733; It's worth mentioning that Edward Pilatowicz has posted a workaround for builds prior to snv_108.

Setup
In my case, I was creating up a build environment within a zone. If you wish to do the same, you must ensure that the SUNWcar and SUNWkvm packages are installed:

% pfexec pkg install SUNWcar SUNWkvm

The following packages are required at a minimum to build uts:

% pfexec pkg install SUNWperl584usr SUNWperl-xml-parser SUNWhea SUNWsprot \
  SUNWbtool SUNWtss SUNWgcc

If you happen to be outside of the Sun firewall, you will also want to install Mercurial and wget. For the remainder of this post, it is assumed that this is the case:

% pfexec pkg install SUNWmercurial SUNWwget

Unfortunately SUNWonbld is only made available as an IPS package in the /dev repository. If you prefer to use a local install for day-to-day development, you will need to install SVR4 packaging support and download the SUNWonbld package from the OpenSolaris Download Center. If you only intend to use nightly(1), then this step may be skipped safely.

% pfexec pkg install SUNWpkgcmds
% cd /var/tmp; wget http://dlc.sun.com/osol/on/downloads/current/SUNWonbld.i386.tar.bz2
% bzcat SUNWonbld.i386.tar.bz2 | tar xf -
% pfexec pkgadd -d onbld SUNWonbld

Update: It appears that SUNWonbld is now available as an IPS package in the /release repository.

You will need to download and install the Sun Studio 12 tarball provided by the Tools Community. Also, you must ensure that /opt/SUNWspro points to your Sun Studio install. Please, please do not attempt to build uts with Sun Studio Express - you will not be pleasantly surprised.

% mkdir -p /opt/SunStudio12; cd /opt/SunStudio12
% gzcat sunstudio12-ii-20081010-sol-x86.tar.gz | tar xf -
% ln -sf /opt/SunStudio12 /opt/SUNWspro

At this point you should update your PATH; this is absolutely critical. If your PATH is not setup correctly, uts may fail to build. If you did not install SUNWonbld locally (see above), then you may drop the first two paths below:

% PATH=/opt/onbld/bin:/opt/onbld/bin/`uname -p`:\
  /opt/SUNWspro/bin:/usr/bin:/usr/sbin:/usr/ccs/bin; export PATH

At this point, you are ready to clone the onnv-gate:

% hg clone ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate

Change your working directory to the freshly cloned gate, then download and unpack the closed binaries from the OpenSolaris Download Center:

% wget http://dlc.sun.com/osol/on/downloads/current/on-closed-bins.i386.tar.bz2
% bzcat on-closed-bins.i386.tar.bz2 | tar xf -

You will also need to copy and edit opensolaris.sh. Pay especial attention to GATE, CODEMGR_WS, and STAFFER. Don't forget to remove 't' from NIGHTLY_OPTIONS if you have SUNWonbld installed locally. I typically alter maxjobs() to a more reasonable value - the default tends to be a little aggressive on a shared system.

% cd onnv-gate; cp usr/src/tools/env/opensolaris.sh .
% vi opensolaris.sh

Finally, issue bldenv(1) to setup your build environment. If you downloaded the debug closed bins, make sure that you pass -d for a debug build:

% bldenv -d opensolaris.sh

Building the Kernel
At this point, building the kernel is pretty straightforward. In order to keep dmake from complaining on each invocation, you will want to set the DMAKE_MODE environment variable if you have not already done so in your .dmakerc:

% DMAKE_MODE=parallel; export DMAKE_MODE

If the SUNWonbld package was not installed earlier (see above), or you prefer to use the tools provided in the gate, you will need to install them first:

% cd $SRC/tools; dmake install

Once the build tools are available, you will need to issue a `dmake setup' from the top level source directory:

% cd $SRC; dmake setup

At this point you are free to build uts:

% cd $SRC/uts; dmake all

As always, YMMV.

dnet(7D) up for review

2008-06-21T14:03:00.002-05:00

The GLDv3 conversion for dnet(7D) is finally up for review. This patch provides GLDv3 and VLAN tagging for all currently supported devices, as well as link notification for those which support an MII transceiver.

The webrev is available here.

A followup patch will be provided to support Project Brussels.

Not Dead Yet!

2008-06-08T12:38:00.006-05:00

It's been quite some time since I have made a post - things have been pretty hectic around here lately.

Like most hardware geeks, each year I go through a round of hardware and software upgrades - this year was no exception. My long-time girlfriend (now fiancee) and I also moved in with each other around this same time. I am sure most of you can already see where this is going...

I sorely needed to upgrade the main switch to one that supported jumbo frames and VLAN tagging (particularly for gigabit UTP testing). Storage was also in pretty rough shape, and I needed to replace 6 of the 8 disks in the rack. Solaris 10 5/08 had been released (early) in April, further dating the 11/06 install on my primary server (which needed a complete overhaul). Compounding these issues was the absolute need for a new UPS; the Netra 20 was overloading the existing one on a regular basis.

I took the plunge and decided to upgrade everything at once despite already being overwhelmed with the move and the stress that comes with the merging of two homes.

I found out very quickly that my time is finite. Both the move and the upgrade was long and trying. There were (of course) a number of issues with the new hardware and a few kinks to iron out with the new software installs. Work with ON suffered due to a lack of time, and the extended downtime also affected others who use the hardware remotely. Time spent messing with the upgrade also put a strain on relationships I had (naively) not expected. On top of all of that, I managed to get pretty sick, which placed even more stress on my relationships and prolonged downtime even further.

That said, everything is finally finished, relationships patched up, bodies healed, and everything is now approaching some modicum of normalcy. I am finally able to return to work with ON (post forthcoming).

I believe this post (and the last month, for that matter) can be summed up as follows:
moving + marriage + hardware + software = A Really Bad Idea(TM)

Update: For the curious, I have uploaded a few pictures of the upgrades here.

Cassini Emancipation

2008-04-15T13:11:00.000-05:00

Last month I purchased a Sun Netra 20/T4 to do some sparcv9 driver development. Since I upgraded all of the equipment last year to gigabit UTP, I also grabbed a Sun GigaSwift (501-5902) to go with it.

Out of curiosity, I dug through onnv-gate to see if the ce driver source had been opened yet; alas it had not. After talking with some Sun folk a bit, it seemed that not only were there no plans to open source Cassini (yet), but much of the code wasn't in the best of shape. Surprisingly, the ce driver still implements the DLPI interface directly rather than making use of the newer frameworks available.

After speaking with John Sonnenschein (who leads the Emancipation Project) and Jason King, we decided to begin an effort to emancipate the Cassini driver. The new driver will be a 'clean room' implementation and will (of course) be open source. It will also support the newer frameworks, namely: GLDv3/Project Nemo, and Project Brussels rather than implementing DLPI directly. As far as naming goes, we decided to go with cge (Cassini Gigabit Ethernet) since it is more in line with the current trend in device naming. Fortunately, with interface vanity naming being integrated in snv_83, users migrating from the original ce driver should have a much easier time of it.

Update: Cyril Plisko has joined the effort. Welcome!

Mercurial and SUNWonbld

2008-02-23T14:34:00.000-06:00

For those developers (like myself) whom are external to Sun working with ON, managing change can be something of a pain - particularly for projects which span multiple files. Fortunately, some very intelligent folk at Sun have developed a number of scripts which vastly simplify the review process. Chief among these is webrev (which has traditionally been used on TeamWare workspaces).

Since mercurial was selected as the SCM of choice for external ON development, many of the scripts provided in SUNWonbld are not available (yet). One of the primary goals of the SCM Migration Project is to bridge this gap.

Essentially, the stable SUNWonbld packages offered on the main ON download page is not compatible with mercurial. In order to get mercurial tool support (i.e.: webrev), you must download the SUNWonbld-latest package from the SCM Migration Project page.

A couple things to keep in mind:

Make sure the most recent SUNWmercurial package is installed (currently, 0.95 is the latest). Check the Source Code Management Tool Downloads page for details.

Read the webrev man pages thoroughly. A common mistake is to pass a specific file to webrev; this is incorrect. webrev expects a file (denoted as file-list in the man page) which contains a list of files to work against.

Many thanks to James McPherson and Rich Lowe in #onnv-scm for their many suggestions and patience!

Server Room Move

2008-01-27T00:14:00.000-06:00

With the help of Erin's brother (and copious amounts of pizza and caffeine), we were able to move the rack this weekend with minimal fuss.

Seeing the rack fit perfectly in the nook I built out earlier this year in the basement was very satisfying. Looks like all of that math finally paid off...

Update: Yes, that is our cat. His name is Hagrid. No, I did not name him.

It Began With A Whimper...

2008-01-02T20:19:00.000-06:00

It is 2008 and another year has come and gone.

Perhaps it is a sign of getting older, but I find myself regretting not taking the time to commit to posterity the events of the past few years.

So, why not make a change?

Fortunately each new year brings with it the perfect vehicle for hapless change: the New Year Resolution. Therefore, I hereby resolve to document the un-interesting, semi-noteworthy, and inane so that future generations may look behind and wonder, "What the hell were they thinking?"

Viva La Resolution!