Best Practices for Developers
The objective of this page is to bring together best practices for developers working on pkg, meant to enhance its supportability and usability. This document does not intend to be a comprehensive guide to the processes and tools used to assemble and deliver packages.
Table of Contents
There are two build frameworks currently in use for pkg(5): The Make framework, and the distutils framework. The Makefile provides a wrapper around the distutils setup.py, adding functionality that is specific to Solaris. Anything that is intended to be cross-platform is implemented using the distutils framework. Solaris-specific functionality (e.g. ability to convert a host of SVR4 packages into pkg(5) packages) is implemented in additional Makefiles and is then called by the top-level Makefile. Unless you have a specific reason to use the Make framework, you should use the distutils framework, even when building on Solaris.
- distutils (setup.py)
This framework is based on Python's Distutils framework, and provides a platform-independent interface for building and distributing pkg(5). To use this framework, you must have a Python 2.4 distribution installed on your build machine. The following commands assume the python command is accessible (e.g. in your $PATH).
- python setup.py --help-commands - Command help information, including the pkg(5)-specific command additions. Note that most of the generic distutils commands are useful to the pkg(5) developer.
- python setup.py build - Builds everything, including binary modules, C extensions, and front-end CLIs. The results are placed in ../proto/build_<os>.
- python setup.py install - Does a build, builds dependencies and then creates a prototype area in ../proto/root_<os> containing a hierarchy of files intended to represent what pkg(5) looks like when installed on a particular OS.
- python setup.py bdist - Creates a binary distribution of the resulting build. This is in most cases a simple zip or gzipped tarball of the relocatable pkg(5) binaries.
- python setup.py clean - Un-does what --build did: Removes binary artifacts.
- python setup.py clobber - Un-does what make --install did. This removes the prototype area and the dependencies.
- python setup.py lint - Runs the pkg(5) C Extension source code through the lint checker. In addition, runs the pkg(5) pure Python source code through pylint. It is assumed pylint has been installed into the python installation area site-packages).
- python setup.py test - Runs unit and functional tests. This should be run as root, e.g., with pfexec or sudo.
To use the make framework to build pkg(5), one must build on a machine installed with Solaris. The following main make targets are available:
Multi-Platform Best Practices
pkg(5) was originally designed for use with Solaris. However, the underlying concepts and usefulness apply more broadly. This fact, coupled with the fact that pkg(5) is currently implemented using a mostly cross-platform language (python) has led the team to produce and maintain pkg(5) in a portable manner for use in projects like Update Center, which is developing tooling based on a multi-platform pkg(5). This has a non-trivial impact to pkg(5) developers, who must take care when evolving pkg(5) to do so in a way that allows pkg(5) to continue to be portable to other platforms.
This does not mean platform-specific characteristics and features cannot be used in pkg(5). For example, Solaris' ZFS is used to implement some or all of pkg(5)' rollback capability. However, it does mean the implementation must either make an attempt to implement a degraded version of rollback on when not used on Solaris, or the feature must be gracefully disabled and appropriate warnings issued when the feature would normally be employed.
Writing portable Python
The following notes and best practices aid the pkg(5) developer in keeping pkg(5) portable.
- 'Best Practice: Test C Extensions on Linux'
pkg(5) currently contains some python extensions written in C: the arch extension for getting access to lower-level Solaris platform characteristics, and the elf extension for reading information from within ELF files. The elf C code is portable to other Unix operating systems which use the ELF format (e.g. Linux), using , an open source library for accessing ELF libraries. However, there are still some differences. For example, the EM_AMD64 constant is EM_X86_64 when using libelf. In addition, the routines used to calculate SHA-1 hashes come from a built-in library (libmd) on Solaris, but must use OpenSSL on other platforms. These differences manifest themselves as several #ifdefs in the C extensions. If you are making changes to this code, it is best to test on a Linux platform using this library.
- 'Best Practice: write relocatable code'
As noted earlier, pkg(5) was originally developed for use on Solaris, with some bootstrapping assumptions about its installed location. However, modern pkg(5) now makes no absolute assumptions about its installed locations, and makes references to its needed external files using relative references or paths passed in from the external user. Therefore, you should never make assumptions about the installed location of pkg(5) itself. It may be completely installed and isolated in x:\Program Files\Sun\MessageQueue\pkg, or it may be installed into /usr. pkg(5) code that requires external resources should always make relative references to its files, based on the known location of the package front-end at runtime, and its location relative to the needed files. This would also apply to any future library-level interface, using some kind of $ORIGIN interface for location discovery.
- 'Best Practice: Write unit tests to test known platform differences'
If you develop new functionality that has known differences between platforms, make sure to supply new unit or other tests that test that the expected differences have been properly dealt with.
In addition to python APIs, the use of external resources for testing purposes should be limited to those that exist across all platforms, when required. For example, do not assume the presence of "/bin/ls" for tests that test functionality that is expected to work on Windows.
As noted above, at a bare minimum, testing involves running the supplied test targets (using the appropriate build infrastructure). Ideally, this would be executed on all platforms on which pkg(5) is expected to operate. Generally, the Solaris developers that work on pkg(5) only test on Solaris. Other teams are responsible for porting pkg(5) to other platforms and for ensuring testing is performed on other platforms.