sbuild is a tool used by those maintaining packages in Debian, and derived distributions such as Ubuntu. When used correctly, it can catch a lot of categories of bugs before packages are uploaded. It does this by building the package in a clean environment, and then running the package through the Lintian, piuparts, adequate and autopkgtest tools. However, configuring sbuild so that it makes use of all of these tools is cumbersome.

In response to this complexity, I wrote a module for the Propellor configuration management system to prepare a system such that a user can just go ahead and run the sbuild(1) command.

This module is useful on one’s development laptop – if you need to reinstall your OS, you don’t have to look up the instructions for setting up sbuild again. But it’s also useful on throwaway build boxes. I can instruct propellor to provision a new virtual machine to build packages with sbuild, and all the different tools mentioned above will be connected together for me.

I just uploaded Propellor version 5.1.0 to Debian unstable. The version overhauls the API and internals of the Sbuild module to take better advantage of Propellor’s design. I won’t get into those details in this post. What I’d like to do is demonstrate how you can set up sbuild on your own machines, using Propellor.

Getting started with Propellor

apt-get install propellor, and then propellor --init.

As mentioned, at the time of writing you’ll need to install from Debian unstable. For this tutorial you need version 5.1.0 or greater.

You’ll be offered two setups, options A and B. I suggest starting with option B.

If you never use Propellor for anything other than provisioning sbuild, you can stick with option B. If this tutorial makes you want to check out more features of Propellor, you might consider switching to option A and importing your old configuration.

Open ~/.propellor/config.hs. You will see something like this:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ mybox
        ]

-- An example host.
mybox :: Host
mybox = host "mybox.example.com" $ props
        & osDebian Unstable X86_64
        & Apt.stdSourcesList
        & Apt.unattendedUpgrades
        & Apt.installed ["etckeeper"]
        & Apt.installed ["ssh"]
        & User.hasSomePassword (User "root")
        & File.dirExists "/var/www"
        & Cron.runPropellor (Cron.Times "30 * * * *")

You’ll want to customise this so that it reflects your computer. My laptop is called iris, so I might replace the above with this:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ iris
        ]

-- My laptop.
iris :: Host
iris = host "iris.silentflame.com" $ props
        & osDebian Testing X86_64

The list of lines beginning with & are the properties of the host iris. Here, I’ve removed all properties except the osDebian property, which informs propellor that iris runs Debian testing and has the amd64 architecture.

The effect of this is that Propellor will not try to change anything about iris. In this tutorial, we are not going to let Propellor configure anything about iris other than setting up sbuild.

(The osDebian property is a pure info property, which means that it tells Propellor information about the host to which other properties might refer, but it doesn’t itself change anything about iris.)

Telling Propellor to configure sbuild

First, add to the import lines at the top of config.hs the lines:

import qualified Propellor.Property.Sbuild as Sbuild
import qualified Propellor.Property.Schroot as Schroot

to enable use of the Sbuild module. Here is the full config for iris, which I’ll go through line-by-line:

-- The hosts propellor knows about.
hosts :: [Host]
hosts =
        [ iris
        ]

-- My laptop.
iris :: Host
iris = host "iris.silentflame.com" $ props
        & osDebian Testing X86_64
        & Apt.useLocalCacher
        & sidSchrootBuilt
        & Sbuild.usableBy (User "spwhitton")
        & Schroot.overlaysInTmpfs
        & Cron.runPropellor (Cron.Times "30 * * * *")
  where
        sidSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian Unstable X86_64
                & Sbuild.update `period` Daily
                & Sbuild.useHostProxy iris

• Apt.useLocalCacher set up apt-cacher-ng and points apt on iris at the cacher. This is the most efficient way to share a cache of packages between apt on iris, and apt within the sbuild chroot.
• sidSchrootBuilt builds the sbuild schroot. Unfortunately, we have to use a where clause because of Haskell’s rules about operator precedence. But it’s just a simple substitution: imagine that & Sbuild.built ... replaces & sidSchrootBuilt.
  • osDebian specifies that this is a Debian unstable chroot. You can easily change this, or add another chroot, for building stable backports, etc.
  • Sbuild.UseCcache enables ccache for builds in this chroot. You can replace this with Sbuild.NoCcache when building a package which is broken by ccache, which happens from time-to-time.
  • Sbuild.update updates the chroot once per day.
  • Sbuild.useHostProxy iris causes Propellor to propagate iris’s apt proxy into the chroot, so that apt in the chroot will also use iris’s apt cacher.
• Sbuild.usableBy adds spwhitton to the right group, so that he is allowed to invoke sbuild(1).
• Schroot.overlaysInTmpfs configures sbuild to install build dependencies and build packages in tmpfs. You can omit this property on machines with low amounts of memory.
• Cron.runPropellor sets up a cron job to re-run propellor once per hour. This is needed to ensure that things like Sbuild.update actually happen. It will also normalise sbuild configuration files, replace chroots that you accidently deleted, etc.

Running Propellor to configure your laptop

$ propellor iris.silentflame.com

In this configuration, you don’t need to worry about whether the hostname iris.silentflame.com actually resolves to your laptop. However, it must be possible to ssh root@localhost.

This should be enough that spwhitton can:

$ sbuild -A --run-lintian --run-autopkgtest --run-piuparts foo.dsc

Further configuration

It is easy to add new schroots; for example, for building backports:

        ...
        & stretchSchrootBuilt
        ...
  where
        ...
        stretchSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian (Stable "stretch") X86_64
                & Sbuild.update `period` Daily
                & Sbuild.useHostProxy iris

You can even use architectures other than X86_64. Propellor knows how to invoke qemu when it needs to do this to build the chroot, though sbuild does not know how to actually use chroots built in this way.

You can also add additional properties to configure your chroot. Perhaps on your LAN you need sbuild to install packages via https, and you already have an apt cacher available. You can replace the apt-cacher-ng configuration like this:

  where
        sidSchrootBuilt = Sbuild.built Sbuild.UseCcache $ props
                & osDebian Unstable X86_64
                & Sbuild.update `period` Daily
                & Apt.mirror "https://foo.mirror/debian/"
                & Apt.installed ["apt-transport-https"]

Thanks

Thanks to Propellor’s author, Joey Hess, for help navigating Propellor’s type system while performing the overhaul included in version 5.1.0. Also for a conversation at DebConf17 which enabled this work by clearing some misconceptions of mine.

Here’s are some of the bugs against the Debian Policy Manual. In particular, there really are quite a few patches needing seconds from DDs. Please consider getting involved.

Consensus has been reached and help is needed to write a patch

#172436 BROWSER and sensible-browser standardization

#273093 document interactions of multiple clashing package diversions

#299007 Transitioning perms of /usr/local

#314808 Web applications should use /usr/share/package, not /usr/share/doc/…

#425523 Describe error unwind when unpacking a package fails

#452393 Clarify difference between required and important priorities

#476810 Please clarify 12.5, “Copyright information”

#484673 file permissions for files potentially including credential informa…

#491318 init scripts “should” support start/stop/restart/force-reload - why…

#556015 Clarify requirements for linked doc directories

Wording proposed, awaiting review from anyone and/or seconds by DDs

#756835 Extension of the syntax of the Packages-List field.

#786470 [copyright-format] Add an optional “License-Grant” field

#835451 Building as root should be discouraged

#845255 Include best practices for packaging database applications

#846970 Proposal for a Build-Indep-Architecture: control file field

#864615 please update version of posix standard for scripts (section 10.4)

#874090 Clarify wording of some passages

#874095 copyright-format: Use the “synopsis” term established in the de…

#877674 [debian-policy] update links to the pdf and other formats of the do…

#882445 Proposed change of offensive packages to -offensive

Merged for the next release

#683495 perl scripts: ”#!/usr/bin/perl” MUST or SHOULD?

#877674 [debian-policy] update links to the pdf and other formats of the do…

#878523 [PATCH] Spelling fixes

Here’s are some more of the bugs against the Debian Policy Manual. In particular, there really are quite a few patches needing seconds from DDs. Please consider getting involved.

Consensus has been reached and help is needed to write a patch

#568313 Suggestion: forbid the use of dpkg-statoverride when uid and gid ar…

#578597 Recommend usage of dpkg-buildflags to initialize CFLAGS and al.

#582109 document triggers where appropriate

#587991 perl-policy: /etc/perl missing from Module Path

#592610 Clarify when Conflicts + Replaces et al are appropriate

#613046 please update example in 4.9.1 (debian/rules and DEB_BUILD_OPTIONS)

#614807 Please document autobuilder-imposed build-dependency alternative re…

#628515 recommending verbose build logs

#664257 document Architecture name definitions

#682347 mark ‘editor’ virtual package name as obsolete

Wording proposed, awaiting review from anyone and/or seconds by DDs

#582109 document triggers where appropriate

#610083 Remove requirement to document upstream source location in debian/c…

#636383 10.2 and others: private libraries may also be multi-arch-ified

#645696 [copyright-format] clearer definitions and more consistent License:…

#649530 [copyright-format] clearer definitions and more consistent License:…

#662998 stripping static libraries

#682347 mark ‘editor’ virtual package name as obsolete

#683495 perl scripts: ”#!/usr/bin/perl” MUST or SHOULD?

#688251 Built-Using description too aggressive

#737796 copyright-format: support Files: paragraph with both abbreviated na…

Merged for the next release

#683495 perl scripts: ”#!/usr/bin/perl” MUST or SHOULD?

#877674 [debian-policy] update links to the pdf and other formats of the do…

#878523 [PATCH] Spelling fixes