What is the Blastwave Story ?
1. INTRODUCTION
The "blastwave.org" open source software project has grown in popularity,
complexity, offerings and expectation since its formation. This growth
has been measured through the use of website traffic statistics, software
usage, public response and requests via email traffic as well as requests
for bug fixes. This growth has been consistent from month to month.
The "blastwave.org" community concept and website were created by Dennis
Clarke in response to a perceived need by Solaris users. The first concept
that Dennis tackled, was the need for publicly available open source
software that was maintained by members of the users community. To this
end, the first community project hosted at blastwave.org, was the CSW
packaging effort. This entailed more than just access to software via ftp
or web download. This meant that access to the software maintenance
process would be open to members of the users community. The needs of the
Solaris user would be met by another Solaris user. What was needed was an
open facility that would allow users to create the software packages for
themselves. The "blastwave.org" web site was created to respond to this
need. Those needs are now being met.
Software "maintainers" join the organization via open invitation. Either
we approach them or they approach us with a need. In most cases the need
is mutual and then the access to the site is granted. The maintainer is
then granted access to "build servers" as well as the necessary storage
on the central "blastwave.org" NFS server. The maintainer is expected to
conform to software package standards and the resultant software package
is submitted for open review and testing by the members of blastwave.org
before public release. The software package will be subject to review by
the public and a bug reporting process is in place for feedback. These bug
reports are assigned and then handled based on priority and severity.
The software, upon public release, is delivered to the Solaris user by a
number of "mirror sites" located around the world. The end user is not
expected to manually ftp or http download a file, unpack and then install
the software package as well as all its dependencies. This is seen as an
unreasonable burden on the end user. The popular "pkg-get" software is
used, so that the end user need know nothing other than the name of what
they need. A simple command line entry by the user will result in the
package being downloaded from a mirror site, checked for dependencies,
then all of these software dependencies will be installed for the user.
Most of the software packages will be configured for the user by scripts
written such that the end user will be up and running instantly. The
Apache web server package is very popular for this reason. The software
packages are coordinated to work together, even across different
maintainers. Our collection of apache modules is a fine example of this. A
user can run a single pkg-get command, to install a working php webserver
with openssl support, in one pass. Additional apache modules can easily be
added via the pkg-get process. All the software is created and packaged
for the x86 and Sparc Solaris user on a baseline release of Solaris 8.
All the software packages must be available for both x86 and Sparc before
public release. We review and test intermediate builds internally. The
software is expected to be available as a 64-bit solution whenever it is
possible and reasonable to do so.
-------------------------------------------------------------------------
2. HISTORY
The "blastwave.org" site and build servers were born out of a grassroots
movement which had little in the way of resources. Phil Brown was
looking for a new central home for his CSW packaging project, and
Dennis Clarke offered the resources of the newly created "blastwave.org".
In the beginning, Phil Brown set up his pkg-get software repository and his
initial set of about 50-100 packages. The maintainers list "quickly" grew
to four people. The initial web site was created in less than a day with
an old discarded Sun Ultra 1 server. The build servers were equally
impressive. There was a single Ultra 2 with 200MHz processors and 256Mb
of RAM. The x86 build server was an 400MHz IBM Netfinity pentium based
unit and 512Mb of RAM. We were very thrilled when the web server saw its
first 1000 hits. That was the first month.
As the word of our existence spread through the internet we rapidly saw
maintainers join, create software packages and then respond to public bug
reports with maintenance updates. We had found that there were many
bright and motivated Solaris users that were happy to become the package
maintainers for "their own software". They took personal ownership and
pride in what they had built and were happy to defend its viability via
maintenance and cooperation. Our need for internet bandwidth was not
seen as a great issue as we seldom transfer large amounts of data. Then
after six months we had three mirror sites and some fifty or so software
packages for x86 and Sparc Solaris users. We had become a community open
source site for Solaris users. Then came dramatic growth.
-------------------------------------------------------------------------
3. THE PRIMARY PROJECTS
The initial intent of the "blastwave.org" site was to provide Solaris
users with equal access to open source software regardless of the users
choice of architecture. We now see blastwave.org as an open source
community site for Solaris users.
3.1 CSW - Community SoftWare
Every software package is to be current and to be provided for the
x86 and Sparc editions of Solaris. This is seen as a firm mandate.
The initial members of "blastwave.org" debated on the base release to
support and Solaris 8 was accepted for various reasons. The most
obvious was cost. Solaris 8 was seen as being a very recent and
affordable option for x86 users and Sparc users alike. We also knew
that software that ran well on Solaris 8 would certainly run with
the Solaris 9 release as well as Solaris 10. The Community Software
project was thus begun. Initially the CSW project was the only real
reason for the organization to exist. Recent progress with Solaris
10 has forced the addition of specialized Solaris 10 build servers.
The maintainers have all agreed that we shall use Solaris 8 as our
base operating release. We have also agreed that Solaris 10 is a
special case and that dedicated Solaris 10 build servers are needed
to support the FCS release. It is also clear that dedicated PowerPC
build servers will be required for Blastwave PowerPC port work.
The simple creation of a software package which could then be offered
to the public via FTP or HTTP download was seen as primitive. Debian
Linux provides its users with the ability to simply get software via
a facility called "apt-get". This script would allow the end user to
simply type a command line and then the software and all dependencies
would be downloaded and installed. We felt that this was a far more
intelligent way to proceed. Therefore, Phil Brown wrote "pkg-get",
a package downloading tool which eventually became a package itself:
CSWpkgget. The end user could now select their software from a
list with the command "pkg-get -a". The user may install all that is
needed with "pkg-get -i softwarename". The most wonderful facility
is the ability to update software, all software, with a very simple
command "pkg-get -u". The last, and perhaps MOST salient point, is
that the CSW project provides rapid response to the user community
via software offerings, bug reports and fixes. Users did not need to
wait for months to gain access to latest editions of software or for
a bug report to be handled. The KDE 3.2.1 packages were created in
less than six days from the day that the source was released. This
rapid response ensures that we may offer Solaris users the most up to
date software before any of the Linux distributions. We refer to this
speed as "internet time". Critical bugs will be fixed within days.
The CSW packages for Openssh and Openssl are popular for this reason.
The CSW project has core maintainers and software projects. The admin
and owner Dennis Clarke is responsible for all infrastructure that is
required by the project. Philip Brown is the general director of the
packaging project, at a software level as well as the primary creator
of our standards for packaging and naming. We have a dozen members
that are involved in specific "large software" build projects at
various levels. The process of creating a large package such as KDE,
GCC or GNOME needs input from a number of people to ensure a quality
product. There are another one hundred members that become involved as
required or requested for their own software packages. A number of
software engineers from Sun are now maintainers also and this has
allowed for a gentle merger of the community programmers with internal
Sun developers. The admin labour is now easily sixty hours a week and
night and day to keep everything n blastwave progressing.
There are also key software sets of primary importance. These are the
"software development" tools and "common" foundation packages. These
packages are used internally for some builds. The maintainers have
decided that blastwave.org can offer these packages to Solaris users
in revisions that are more current than that offered by either the
sunfreeware.com site or by the Sun Companion CD Media. This sort of
up to date software is demanded by the end user. Therefore the CSW
project will expand to include primary projects such as GNOME and KDE
by creating servers exclusive to desktop creation and maintenance.
This division within the CSW project will aid in providing the most
up to date software to the Solaris user. These dedicated project servers
are needed because a Mozilla or KDE build can overwhelm the older
build servers for days thus stopping or hindering other work.
The software packages being created for Solaris Express and OpenSolaris
are continually in a state of review. A state of non-stop build and
rebuild of software for each release of Solaris Express or OpenSolaris
code drop places a large demand on the core maintainers. It is clear
that the FCS release of Solaris 10 must be fully supported and the CSW
project aims to provide every major software title to the Solaris 10
user and on every architecture. To this end Blastwave has begun to
test build systems that will allow for a complete nightly build of all
software dependencies and then each strata of software package that
is above those dependencies. This task is non-trivial but required in
order to ensure a high level of quality.
3.2 OpenSolaris & Solaris Express Testing
Each and every package that is created within the CSW project will be
expected to run within Solaris 10. This expectation clearly requires
software testing with the most recent build of Solaris 10. Therefore
separate Solaris 10 servers have been required for testing by the
maintainers. We feel that Solaris 10 build servers for each supported
architecture is absolutely required as the Solaris 10 release will
force the demand upon us.
3.3 Educational Projects
It has been the intent of the author to provide an open facility for
educational institutions. This would entail receiving proposals from
research project leaders within universities and then providing the
facilities for the project. These projects, presumably, will be only
limited by computational and storage requirements. Network bandwidth
to the internet will not be an issue as all experiments would be in a
private network at the blastwave.org main site. The results of the
research and development would be within the public domain. It is the
opinion of the author that experimentation by brilliant minds is the
foundation of invention. Projects that have been discussed are :
3.4.1 OpenSolaris Grid
3.4.2 Open Source Encryption based on chaotic dynamics
3.4.3 Solaris Documentation from a Community Perspective
It should be noted that clear and user friendly docs for
the average end user of Solaris are required. This process
has been begun by the author but halted due to lack of
funding.
See: http://www.blastwave.org/dclarke/stuff/sol10/book.html
3.4 OpenSolaris PowerPC Port - Polaris
Key members of the Blastwave project are also members of the OpenSolaris
project. The opportunity exists to port the Solaris OS to another RISC
processor and to create an independent UNIX distro. While the x86 and
Sparc distro opportunity seems to be far easier to achieve it would
also be very common. The PowerPC port would establish a new market of
UNIX users and be unique. This objective requires a separate project
plan which can be easily integrated into the Blastwave infrastructure.
The end result would be a base of Polaris ( the current distro name )
with the entire Blastwave software suite also. While we do have PowerPC
hardware support from GENESI we do not have infrastructure to support
the project.
3.5 Open Source Subversion Repository
There has been a clear need expressed within the OpenSolaris and Solaris
Community for a common knowledge base which will document the porting
efforts of the software maintainers. With well over 1600 software titles
in the Blastwave software set it makes clear sense to open up a Subversion
repository that documents the changes and knowledge work of the software
maintainers for the world. This would facilitate the open sharing of
knowledge and ease the progress of Solaris developers in other projects.
3.6 Open Build Implementation
After much discussion the Blastwave maintainers have agreed that a build
system is required that will allow for the software base to be rebuilt
without starting from ground zero. A large number of the dependencies
and higher level software infrastructure components need to be rebuilt
in order to verify functionality and interface stability. A key example
is the Zlib library which has recently been rebuilt. The Zlib library is
an underlying dependency to a very large number of other software components
and the only way to ensure stability and quality control is to rebuild all
dependants when a core dependency has been changed. A Sun Engineer has
worked on a brilliant build system that has now been implemented.
