Building from Source Has Never Been Easier

Overview

For me, one of the greatest things Debian gave to the world was apt and dpkg, ie. Debian’s package management system. It does a brilliant job of almost everything and is very easy to use. What I’ll be explaining in this post is how you would use these tools to customize a package at the source level.

If you wanted to change something in the source code of a package, you could always go check it out from the project’s revision control system, or download it from the project’s web site. Though this won’t necessarily be the same version you received through the repositories, and will most probably not have all the patches applied by the distribution’s authors.

There are benefits in getting the latest vanilla version, though there are more cons than pros when compared to using apt and dpkg to get and build the source. Provided one is available, some of the benefits of using the source package from the repositories are:

1. The source code you will be editing will be for the same version as the package you have installed.
2. The source will have all the patches as applied by the distribution’s authors. Some of these patches are sometimes extra functionality which would be lost if you use the vanilla source code.
3. The package version and patches from the distribution is what was tested within that environment.
4. You are building a .deb package file, which can be installed and/or added to a repository to easily use on multiple installations
5. When using a .deb file you can benefit from the dependency management
6. Having a .deb you can control how new versions of the package are handled (like preventing new installations, safely overriding with new versions, etc.)
7. By having a .deb it’s easy to remove the package and install the original again

Points 4 to 7 are also possible to achieve when downloading the vanilla source, though requires many more steps and is far more complicated than the technique I’m describing in this post.

Above all of these benefits, the biggest reason of all why I like to follow this approach when hacking the source of packages from my installations, is definitely the simplicity of it. When summarized, it all comes down to 3 commands I’ll list in the conclusion of this post.

Getting the Source

Before you can start building you need to prepare your environment for it. Run the following command to install the necessary packages:

quintin:~$ sudo apt-get install build-essential fakeroot dpkg-dev

So, for all the examples I’ll be using blueproximity as the package to be built. It’s a python script, so you don’t really need to download separate source code to modify it. Though to demonstrate this technique I figured it’s a good candidate given it’s small size and simple structure.

So to get the source, I’ll make a directory called src and change into it.

quintin:~$ mkdir src
quintin:~$ cd src/

Then instruct apt to download the source code for the project named blueproximity.

quintin:~/src$ apt-get source blueproximity
Reading package lists... Done
Building dependency tree
Reading state information... Done
Need to get 309kB of source archives.
Get:1 http://repo/ubuntu/ lucid/universe blueproximity 1.2.5-4 (dsc) [1,377B]
Get:2 http://repo/ubuntu/ lucid/universe blueproximity 1.2.5-4 (tar) [301kB]
Get:3 http://repo/ubuntu/ lucid/universe blueproximity 1.2.5-4 (diff) [6,857B]
Fetched 309kB in 9s (32.6kB/s)
gpgv: Signature made Mon 24 Aug 2009 00:52:04 SAST using DSA key ID 7ADF9466
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./blueproximity_1.2.5-4.dsc
dpkg-source: info: extracting blueproximity in blueproximity-1.2.5
dpkg-source: info: unpacking blueproximity_1.2.5.orig.tar.gz
dpkg-source: info: applying blueproximity_1.2.5-4.diff.gz

As you can see, apt

1. Downloaded the source tarball blueproximity-1.2.5.orig.tar.gz
2. Downloaded a patch file blueproximity_1.2.5-4.diff.gz.
3. It extracted the source code into a directory blueproximity-1.2.5.
4. And then applied the patch to this directory.

At this stage the source is ready for editing.

Building the Source

In order for your build to complete successfully you might need some development dependencies. These are usually the header files or link libraries, and often named after the package with a -dev suffix. Apt can install anything needed to build a specific package using the build-dep command.

To make sure we have all these dependencies for building blueproximity, we run:

quintin:~/src$ sudo apt-get build-dep blueproximity
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

In my case these wasn’t any build dependencies needed, and thus nothing was installed.

Once you’re happy with your changes and want to build the .deb file, you simply need to

1. Change into the root of the extracted project source code

   quintin:~/src$ cd blueproximity-1.2.5

2. And run the build.

   quintin:~/src/blueproximity-1.2.5$ dpkg-buildpackage -rfakeroot -uc -b
   [truncated output]
   dpkg-deb: building package `blueproximity'
   in `../blueproximity_1.2.5-4_all.deb'.
   dpkg-deb: warning: ignoring 1 warnings about the control file(s)
   
   dpkg-genchanges -b >../blueproximity_1.2.5-4_i386.changes
   dpkg-genchanges: binary-only upload - not including any source code
   dpkg-buildpackage: binary only upload (no source included)

You’ll see a lot of output which I truncated here. The result will be one or more .deb files in the directory where you downloaded the source (in the example, the one named src).

Conclusion

As you can see, there is very little needed to get a package’s source code and build a .deb from it. I’ve done this a few times, and all the packages I’ve modified I’ve added to a repository of my own, and have found this approach to work very well for integrating my changes into the existing system as seamlessly as possible.

There are basically 3 commands needed to build a package from source (excluding setting up your environment for this). These are:

apt-get source [package name]
apt-get build-dep [package name]
dpkg-buildpackage -rfakeroot -uc -b

The first 2 commands you would run in a dedicated directory created for this purpose. The first command will then create a sub directory where it extracts the source code to. It’s in this sub directory where you would run the last command. The first 2 commands you would also only need to run once. The last command you can run each time you want to build a package from the same source code, perhaps when making changes for a second or third time.

It’s really that simple.

So Why Love Linux? Because apt opens up a very easy way of customizing your system at as low a level as the source code.

Managed Packages

There are tons and tons of open source projects out there. Something for almost every topic or task. From general purpose, common or popular down to highly specialized or unheard of software. This is one of Linux’s strengths, especially with distributions like Ubuntu which have package repositories with thousands of options readily available to the user.

Package Manager

Synaptic Package Manager is Ubuntu’s user interface to the underlying apt package management system. Whenever I want to install something I would first go check if I can’t find it in Synaptic before I go look to download it manually. More often than not I would find the package in Synaptic, and can have it then installed with just 2 more clicks of the mouse.

This saves a lot of time, and never goes unappreciated.

Ubuntu Repositories

The package management software for Ubuntu is brilliant. But without thorough repositories they’re nothing more than just that, package management.

Ubuntu has multiple levels of repositories by default, nl. main, universe, multiverse and restricted.

• The main repository is maintained by, and contains software officially supported by Canonicle themselves.
• The universe repository is maintained by the community and isn’t officially supported by Canonicle.
• The restricted repository contains packages that isn’t available under a completely free license. A popular example is for proprietary drivers, like the Nvidia or ATI graphics drivers.
• The multiverse repository contain software that isn’t free.

Canonicle is doing a great job with the main repository, having a decent variety of packages available and kept up to date. On top of this the community is doing a fantastic job to keep the universe repository filled up. With these two I rarely have the need to go looking for software on the internet.

Easy Repository Integration

For the few cases where the default repositories don’t have what you need, you need to get it from the internet.

There are a few ways to install packages from the internet.

• Download an installer and run it.
• Download an archive and either build from source or install it some manual way.
• Download a .deb package and install via dpkg.
• Add a 3rd party repository to your package management system and then install via Synaptic.

The Ubuntu system makes it very easy to add a 3rd party repository. This means that if you come across a site that offers an Ubuntu (or apt) repository, it can usually come in the form of

1. A string called an “APT line”, which you can just add using the supplied GUI in Synaptic Package Manager, or
2. A .deb file which you install via dpkg. This will then set up the repository for you. You can usually just double click on the .deb and it will start up the installation for you.

After you’ve got their repository set up you can go into Synaptic, search for the package you want, and install it.

Standardized Maintenance and Management

One of the biggest benefits of installing packages via the repositories (other than it making your life easier), is that the program is now maintained by the package management system. This means that your system has a standardized way of

1. Having on record what is installed and what files are owned by the package
2. Reinstalling if files go missing or become corrupted
3. Cleanly removing the package
4. Finding and installing updates for the package.

For packages installed via other methods there is usually no uninstall or automated update support.

Some of the more advanced programs have built in support for this. But if you installed it into a shared location owned by root, you won’t be able to update. I usually get around this by temporarily changing the ownership of the directory, doing the update and restoring the ownership.

Exploring

With the large variety of packages available via the Ubuntu repositories, you have an endless number of programs to try out if you feel like exploring. I have had some of these moment where I just pick some random location and start reading the description of each package until I find something that pokes seems interesting. I will then install it, play around and return to the list to find another one.

It’s a very good way of learning about new programs and projects, and certainly an amusing exercise.

Conclusion

So Why Love Linux? Being the result of open source communities there are tons of projects out there and decent repositories and package management systems make these easily available.