IzzySoft

• Person
• Services
• References
• Software
• Oracle
• Imprint

Software List

Documentation

Demos

Articles

IzzySoft Release Framework: Software Release made easy

What's this about

You manage a couple of software projects? And you are responsible for the file releases? So this probably is for you. A file release usually involves a couple of tasks:

relman in action (click to enlarge)

• compiling the sources
• creating a tarball
• creating *.deb/*.rpm packages
• upload the tarball and/or packages to a/several servers
• update repositories and/or download pages
• announce the availibility of the new version

And always remember what was where: local directories, directories on the server(s). Not forget to update links, version numbers, etc. Wouldn't it be nice if you had a release manager, you could just say relman myprog myversion, and you were done? Once you setup the Release Framework discussed here, this is exactly what is left to you in order to make a release! All archives and packages will not only be created, but also distributed – and even the ChangeLog will be available up-to-date on the website, automagically providing the accurat download links to the new release! Just the announcement is left to you …

Three tools are involved in this framework:

• relman is the "distributing center". In its configuration file, it holds all information for each of your project. Based on this, it calls pkgmake to build the tarball and/or packages, and then distributes everything you configured - optionally running additional scripts.
• pkgmake creates the tarball and/or *.deb/*.rpm packages based on the information passed to it on the command line by relman, plus a RPM SpecFile template
• histview sits on the web server, automatically detecting old and new releases along with its ChangeLog file - thus providing the ChangeLog info along with the download links

Caught fire? Good! Please read on in the corresponding sections:

Contents

• relman
  • Documentation
  • Configuration
  • Sources
• pkgmake
  • Documentation
  • Configuration
  • Sources
• HistView
  • Documentation
  • Configuration
  • Sources

relman

Surely relman is the "distribution center", as it distributes the packages over the net. But it is as well the "control center" for the releases, as all necessary information – especially all details on your software projects – will be stored in relmans configuration file.

Relman itself is just a simple Shell script, reading the data from its configuration file, interpretes them, and calls the necessary tools. Sounds very simple, but can be very powerful. Relman causes not only your packages to be created and distributed, but can do a lot more things: Its flexible configuration allows you to define prerun, prebuild, postbuild and postrun scripts. It also distributes your ChangeLog, which of course should have the right format (see HistView), but it cannot yet provide coffee …

Documentation

Aside from this article, documentation is also shipped with the distribution: typing "man relman" at the command line (of course followed by hitting the "enter" key) will give you access to the command line options, while "man relman.conf" discusses the configuration file. So you have all information available there, and this article will concentrate on the essential, to give you an overview.

Configuration

General

The configuration – which is a simple ASCII text file – is devided in two main sections: The "global settings", and the project specific settings. Specification is as easy as setting values to "0" or "1" (to disable or enable them) in most Extract from config file (click to enlarge) cases. The remaining settings are mostly directory specifications. Since many of the "global settings" act as defaults for the project specific ones, setting up a project can be as simple as adding the minimal two lines (name and source directory). But of course you can override some or even all defaults for "very specific projects", while keeping things simple for the usual ones.

Really global values you specify in the global section are the location of your build environment (usually /usr/src/debian if you have already installed pkgmake – which is usually done if you install relman from a *.deb/*.rpm package, since it depends on that), plus the pkgmake executable to use (in case it is not in your path, or you play with a modified version stored in a different place). All other settings defined here provide the defaults for the projects defined later. This means e.g., that you can define to generally create RPM packages, while disabling this for a special project. Or creating *.deb packages only for selected projects. The more your defaults match your projects, the smaller you can keep the project specific settings. The example config shipped with the application shows this with the "small" project, where only the name and the source directory are explicitely defined for the project.

After having set up the global things, you can go and define your projects. These settings always start with a short name for the given project: If you have, to give a stupid example, a project named "verylongprojectname", where all tarballs and packages shall contain the complete name (i.e. verylongprojectname-0.1.2.tar.gz etc.) – you probably don't want to type the full name always. So all your settings for this project could start with vlpn_ instead:

vlpn_name=verylongprojectname
vlpn_srcdir=/usr/local/source/vlpn

and so on. So for a release, you can simply refer to the project as "vlpn": relman vlpn 0.1.2. For each of your project you can define distribution targets, what packages to make, a.s.o.. But if the settings are the same as your default settings, you don't need to specify them again.

Packages

With simple "boolean" variables, you decide what packages to build. While you usually always will generate the tarball (mktar=1) when starting from the sources, since this is a pre-condition for the other packages, you may decide differently for the packages even per project. So mkdeb decides about Debian packages, and mkrpm about RPMs. The variables group and section will tell pkgmake the RPM group resp. Debian section to use for the package. And if you have to tell pkgmake something special not (yet) covered by relman, there's a variable called pkgmakeparms.

In the previous paragraph, all variables mentioned used the "global form" (see the General section above). Preceding them with the projects prefix (vlpn_ in the above example, i.e. vlpn_mkdeb=1 etc.), you define project specific settings.

Distribution

Distribution can be done using the FTP and SCP protocols. Targets you specify for each package type separately – again, you can do so globally and project specific. A typical global setting would be the distribution to the Sourceforge incoming folder:

tarftp=upload.sourceforge.net
tarftpdir=incoming

This defines the server and directory to distribute the *.tar.gz files to, using the FTP protocol. If you use SCP, there's only one line to specify: The complete server:/path[/filename.ext] – and if you ommit the filename.ext (which you usually will), the name just stays the same as the origin. Again: All this *ftp[dir] and *scp settings are to be done for each filetype to be distributed, i.e. deb, rpm, tar and – a special case – hist, which is for the "histfile" used by Histview.

Scripts

Aside from these, relman offers another feature: using the prerun, prebuild, postbuild and postrun keywords, you can define scripts to run at the different stages. These scripts could e.g. modify your code in the sourcedir (prerun). Or in the build dir (prebuild), without effect on your real sources. Or you use the postrun to rebuild your repository. So after relman completed its job, the packages are not only made and distributed, but everything on the server is also set up so users find the freshly distributed files.

Sources

To find out more about relman, you can visit the following resources:

• Relman Homepage
• Relman Project page at Freshmeat
• Relman ChangeLog and Download directory
• The Software List of this site

pkgmake

Relman being the distribution and control center, pkgmake is simply a backend to relman. Its responsibility is to create all the distribution archives: The *.tar.gz as well as the *.deb and *.rpm packages – depending on what you defined. Other than relman, which you usually invoke with just two parameters, pkgmake usually is invoked by a bunch of parameters – but you don't have to care about when using relman, since all necessary parameters will be set up from relmans configuration. But you are also free to use pkgmake directly, without the help of relman.

Documentation

As it was already discussed for relman, documentation is shipped with the distribution in form of man pages. So for detailed information, at the shell prompt you may type man pkgmake for details on the calling conventions (command line parameters and the like), man pkgmake.conf for configuration details, and man pkgmake.tpl for how to set up your template files.

Configuration

The Config File

Extract from config file (click to enlarge) As it also can be used stand-alone, pkgmake also has its own configuration file. The structure is easy: As it's sourced by the main (Bash) script, it is a Bash script itself – so all settings consist of "variable=value" pairs. All the settings here provide defaults, which optionally can be overridden from the command line. This especially means: The closer these defaults match your projects, the less command line switches will be required for the call. Again, usually you won't have to care too much about this when using relman ;)

But some things are important to set up even when using relman. Primarily this is the information about vendor and packager. At least the latter one usually will be the same for all of your projects, so it absolutely makes sense to add it to the configuration here. Also the vendor normally stays the same – except you work as packager for different teams. Licenses may vary – but the build environment (where the packages are generated) usually also stays the same – as do the executables (DEBBUILD and RPMBUILD). These all are things you should configure defaults for.

Besides: There's a global config file stored in /etc/pkgmake.conf for system wide settings. But you also can create your personal copy in ~/.pkgmake/pkgmake.conf to override the defaults from the global config. This is especially useful if multiple developers share the same machine to store their code and generate their releases: While all will use the same global settings (as the executables at least), they can have their personal settings (e.g. packager info) at the same time.

The template file(s)

Extract from template file (click to enlarge) pkgmake uses RPM .spec files to create RPM and/or Debian packages. And yes, both can be created from the same .spec file – basically. The only sure difference is the software group (RPM) versus software section (Debian), which are completely different. Well: It doesn't really hurt if the Debian package lists a RPM group instead of a Debian section. You can still install this *.deb, and no errors will be reported – since the syntax is the same. The only two reasons against this are: 1) it is not 100% conform to the Debian policy, and 2) the user installing these packages probably will have a lot of additional "sections" listed with only one package in. To prevent this, you can provide both group and section, and pkgmake will use the group for RPM and the section for Debian package creation – as long as you didn't "hardcode" the group specification in your template file.

As you can see in the example template file (the image on the left-hand side), you can use a lot of placeholders in your template file. The more you can do with placeholders, the more flexible the usage will be. Of course, nobody forces you to use any placeholder in your templates – but pkgmake can only replace the placeholders used there. In the best case, you could use one template file for multiple projects – but usually this fails at least due to the %files section. Again: The more placeholders you use, the more flexible you can handle the template. But still, not all things can be done with placeholders, and not all make sense. The decision upon which to use is entirely up to you.

Sources

To find out more about pkgmake, you can visit the following resources:

• pkgmake Homepage
• pkgmake Project page at Freshmeat
• pkgmake ChangeLog and Download directory
• The Software List of this site

HistView

So the release has been done, the project has its "Internet page", and (potential) users should be informed about release and changes – this is where Histview comes into play. The histview class can be used to present the ChangeLog in a user friendly way, listing downloadable files (tarballs and packages) along the notes.

Documentation

Histview in action (click to display) Other than the command line tools relman and pkgmake, the documentation for this web-based PHP tool doesn't supply man pages, but is shipped in HTML format. The packages (and the tarball) contain an Api reference as single HTML files, which you find in the doc/ directory of the tarball, resp. the /usr/share/doc/histview directory when installed from a Debian or RPM package. Examples for the usage are also provided within the main directory (/usr/share/histview after package installation).

In order for Histview to be able to analyze your ChangeLog, the ChangeLog file must apply to a certain structure. Naturally, most ChangeLog are at least close to this or even use this syntax, so for most projects only a few little changes will be needed here. Basically, the rules are: A line starting with v0.0.0 mark a release number (where "0.0.0" represents the release version). This line may be followed by a row of dashes (----) used to underline the version. New additions start with + as first sign of the row, followed by at least one space (consecutive lines are indented), removals with a dash, changes with an asterisk, and bugs with an exclamation mark (!). Empty lines and lines starting with a hash mark (#) will be ignored. Quite simple, and used similiarly in many projects – so this shouldn't be the big deal.

Configuration

Histview

Extract from script file (click to enlarge) The configuration itself takes place in the calling PHP script file. Since histview is provided as PHP class, you can define the handling yourself. As a base, you can of course use the provided examples.

Basically, there is not that much to do: Create a new instance of the class, providing it the ChangeLog file to parse and (optionally) the base name of the download files (which usually is the project name – e.g. "histview" for all the histview-0.1.4.tar.gz, histview_0.1.4-izzy1_all.deb, etc.pp.). Now you can optionally define to use the download class by calling the use_dlclass() method, and tell histview where to find the files using the set_basedir() method. If you want to distribute RPM/Debian packages, you should also define your release name prefix (if you use any) by the set_relname() method, and you are done. Calling the process() method will process your ChangeLog file, and the out() method returns the resulting HTML code you then can embed in your script.

More optional settings are possible – like using different icons for the download files, just to mention one. Please refer to the above mentioned Api reference for details on this.

The Download Class

Already mentioned before as optionally being used by the histview class, the download class offers some more facilities you may want to use aside from histview. On one hand, it can setup download directory listings for different (projects) files following the same naming conventions. But more interesting is possibly the chance to setup an "alias" to always retrieve the latest version from your projects releases – so you simply can point your users to some file, say project_release.php, and they always get the latest version of your code without the need for you to always update this file. This is done by the method send_latest() provided by this download class. This method takes 3 parameters: The base name of the file, the file type (e.g. "tar" or "rpm"), and the directory to look for this. From that directory it checks for the file with the given base name and of the specified type having the highest release number, and sends this to the requestor.

Sources

To find out more about histview, you can visit the following resources:

• Histview Homepage
• Histview Project page at Freshmeat
• Histview ChangeLog and Download directory
• The Software List of this site