spwhitton/ blog/ entry/ Using Propellor to provision your Debian development laptop
← Debian Policy call for participation -- November 2017, pt. 1 | Notes from the Library | Debian Policy call for participation -- October 2017 →

sbuild is a tool used by those maintaining packages in Debian, and derived distributions such as Ubuntu. When used correctly, it can catch a lot of categories of bugs before packages are uploaded. It does this by building the package in a clean environment, and then running the package through the Lintian, piuparts, adequate and autopkgtest tools. However, configuring sbuild so that it makes use of all of these tools is cumbersome.

In response to this complexity, I wrote a module for the Propellor configuration management system to prepare a system such that a user can just go ahead and run the sbuild(1) command.

This module is useful on one’s development laptop – if you need to reinstall your OS, you don’t have to look up the instructions for setting up sbuild again. But it’s also useful on throwaway build boxes. I can instruct propellor to provision a new virtual machine to build packages with sbuild, and all the different tools mentioned above will be connected together for me.

I just uploaded Propellor version 5.1.0 to Debian unstable. The version overhauls the API and internals of the Sbuild module to take better advantage of Propellor’s design. I won’t get into those details in this post. What I’d like to do is demonstrate how you can set up sbuild on your own machines, using Propellor.

Getting started with Propellor

apt-get install propellor, and then propellor --init.

As mentioned, at the time of writing you’ll need to install from Debian unstable. For this tutorial you need version 5.1.0 or greater.

You’ll be offered two setups, options A and B. I suggest starting with option B.

If you never use Propellor for anything other than provisioning sbuild, you can stick with option B. If this tutorial makes you want to check out more features of Propellor, you might consider switching to option A and importing your old configuration.

Open ~/.propellor/config.hs. You will see something like this:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ mybox
        ]

-- An example host.
mybox :: Host
mybox = host "mybox.example.com" $ props
        & osDebian Unstable X86_64
        & Apt.stdSourcesList
        & Apt.unattendedUpgrades
        & Apt.installed ["etckeeper"]
        & Apt.installed ["ssh"]
        & User.hasSomePassword (User "root")
        & File.dirExists "/var/www"
        & Cron.runPropellor (Cron.Times "30 * * * *")

You’ll want to customise this so that it reflects your computer. My laptop is called iris, so I might replace the above with this:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ iris
        ]

-- My laptop.
iris :: Host
iris = host "iris.silentflame.com" $ props
        & osDebian Testing X86_64

The list of lines beginning with & are the properties of the host iris. Here, I’ve removed all properties except the osDebian property, which informs propellor that iris runs Debian testing and has the amd64 architecture.

The effect of this is that Propellor will not try to change anything about iris. In this tutorial, we are not going to let Propellor configure anything about iris other than setting up sbuild.

(The osDebian property is a pure info property, which means that it tells Propellor information about the host to which other properties might refer, but it doesn’t itself change anything about iris.)

Telling Propellor to configure sbuild

First, add to the import lines at the top of config.hs the lines:

import qualified Propellor.Property.Sbuild as Sbuild
import qualified Propellor.Property.Schroot as Schroot

to enable use of the Sbuild module. Here is the full config for iris, which I’ll go through line-by-line:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ iris
        ]

-- My laptop.
iris :: Host
iris = host "iris.silentflame.com" $ props
        & osDebian Testing X86_64
        & Apt.useLocalCacher
        & sidSchrootBuilt
        & Sbuild.usableBy (User "spwhitton")
        & Schroot.overlaysInTmpfs
        & Cron.runPropellor (Cron.Times "30 * * * *")
  where
        sidSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian Unstable X86_64
                & Sbuild.update `period` Daily
                & Sbuild.useHostProxy iris
  • Apt.useLocalCacher set up apt-cacher-ng and points apt on iris at the cacher. This is the most efficient way to share a cache of packages between apt on iris, and apt within the sbuild chroot.
  • sidSchrootBuilt builds the sbuild schroot. Unfortunately, we have to use a where clause because of Haskell’s rules about operator precedence. But it’s just a simple substitution: imagine that & Sbuild.built ... replaces & sidSchrootBuilt.
    • osDebian specifies that this is a Debian unstable chroot. You can easily change this, or add another chroot, for building stable backports, etc.
    • Sbuild.UseCcache enables ccache for builds in this chroot. You can replace this with Sbuild.NoCcache when building a package which is broken by ccache, which happens from time-to-time.
    • Sbuild.update updates the chroot once per day.
    • Sbuild.useHostProxy iris causes Propellor to propagate iris’s apt proxy into the chroot, so that apt in the chroot will also use iris’s apt cacher.
  • Sbuild.usableBy adds spwhitton to the right group, so that he is allowed to invoke sbuild(1).
  • Schroot.overlaysInTmpfs configures sbuild to install build dependencies and build packages in tmpfs. You can omit this property on machines with low amounts of memory.
  • Cron.runPropellor sets up a cron job to re-run propellor once per hour. This is needed to ensure that things like Sbuild.update actually happen. It will also normalise sbuild configuration files, replace chroots that you accidently deleted, etc.

Running Propellor to configure your laptop

$ propellor iris.silentflame.com

In this configuration, you don’t need to worry about whether the hostname iris.silentflame.com actually resolves to your laptop. However, it must be possible to ssh root@localhost.

This should be enough that spwhitton can:

$ sbuild -A --run-lintian --run-autopkgtest --run-piuparts foo.dsc

Further configuration

It is easy to add new schroots; for example, for building backports:

        ...
        & stretchSchrootBuilt
        ...
  where
        ...
        stretchSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian (Stable "stretch") X86_64
                & Sbuild.update `period` Daily
                & Sbuild.useHostProxy iris

You can even use architectures other than X86_64. Propellor knows how to invoke qemu when it needs to do this to build the chroot, though sbuild does not know how to actually use chroots built in this way.

You can also add additional properties to configure your chroot. Perhaps on your LAN you need sbuild to install packages via https, and you already have an apt cacher available. You can replace the apt-cacher-ng configuration like this:

  where
        sidSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian Unstable X86_64
                & Sbuild.update `period` Daily
                & Apt.mirror "https://foo.mirror/debian/"
                & Apt.installed ["apt-transport-https"]

Thanks

Thanks to Propellor’s author, Joey Hess, for help navigating Propellor’s type system while performing the overhaul included in version 5.1.0. Also for a conversation at DebConf17 which enabled this work by clearing some misconceptions of mine.