Introduction

I hear you saying: "Yeah, yeah, another marvelous make replacement with built-in support for various compilers, automatic dependency scanning, parallel and distributed builds, and XML instead of makefiles!" On the contrary, build is a software build system that is implemented on top of GNU make. It defines extensible framework for translators, delegates most of its tasks to existing tools, and uses GNU make syntax for makefiles. Build was designed with the following tasks in mind:

configuration
building
testing
installation

As distribution of software in source code is being replaced by precompiled package distributions, greater emphasis was placed on development-time conveniences. Some of the features of build include:

position-independent makefiles
non-recursive multi-makefile include-based structure
leaf makefiles are full-fledged GNU makefiles, not just variable definitions
complete dependency graph
inter-project dependency tracking

To understand leaf makefiles in this manual you will need basic knowledge of how make works. However, inside, build uses some of the more advanced features of GNU make including

order-only prerequisites
pattern and target-specific variables
$(call ) and $(eval )
different flavors of include directive

If you find usage of some of the make techniques obscure in this text you can always consult with the GNU make manual.

Fundamentals

Build defines a framework in terms of which user makefiles are written. Below is the list of fundamental concepts.

Project: is a separately distributable collection of source files. A more intuitive way of defining a project would be to say it's what you normally assign a version number to (e.g., libhello-0.0.1) and package as a tar ball (e.g., libhello-0.0.1.tar.gz). A reasonably complex project usually has a hierarchy of sub-directories under its root directory.
Component: is part of a project for which you would normally write a makefile. For example our libhello could consist of two components: the library itself and a test driver.
Source root: is a root directory of a project. For example, if we unpack our libhello-0.0.1.tar.gz to /tmp and end up with a directory /tmp/libhello-0.0.1 then that would be this project's source root.
Source base: is a directory of a component in a project. Continuing our example, if our libhello project had a subdirectory test then the source base of the test component would be /tmp/libhello-0.0.1/test. By definition, any source base is a sub-directory of a source root for any particular project.
Out root: is a root directory of a project's build (or output) hierarchy. This directory signifies where generated files for the project will be placed. For example, if we decide to build libhello in /tmp/libhello-i686-pc-linux-gnu then that would be one of the possible out roots.
Out base: is a build (or output) directory of a component in a project. If we continue our example, the out base for the test component would be /tmp/libhello-i686-pc-linux-gnu/test.

The build runtime defines four make variables that correspond to the last four definitions:

src_root
src_base
out_root
out_base

Below is an example of how you could use some of them.

driver := $(out_base)/driver

$(driver): $(out_base)/driver.o $(out_root)/libhello/libhello.so

Additionally, the build framework defines three more variables:

bld_root
scf_root := $(src_root)/build
dcf_root := $(out_root)/build

bld_root is a root directory of the build runtime. Normally you would use it to include one of the build's files.

scf_root stands for static configuration root and points to a directory where project's static configuration is kept. dcf_root stands for dynamic configuration root and points to a directory where project's dynamic configuration is kept. We will talk more about those two variables later.

Also note that having src_root the same as out_root is perfectly valid and indicates that we are building in a source directory.

First Example

Now we are ready to examine our first simple example. We will skip one-file C-based "hello world", however, because it is not very practical and there is not much room for improvement. Rather we will start from a libhello library, a test driver for it and a hello program that uses the library. To make our example even more realistic the libhello library and the hello program are two separate projects. And, did I mention, they are written in C++. See examples/cxx/hello.

Invocation

Let's first get some user experience without looking into makefiles. Suppose we unpacked build source into /tmp/build-0.1.12. First let's try to build libhello:

$ cd /tmp/build-0.1.12/examples/cxx/hello/libhello
$ make


configuring 'libhello'



Please select the C++ compiler you would like to use:

(1) GNU C++ (g++)
(2) Intel C++ (icc)

[1]: 1

Would you like the C++ compiler to optimize generated code?

[y]: y

Would you like the C++ compiler to generate debug information?

[y]: y


configuring 'libhello'



Please select the default library type:

(1) archive
(2) shared object

[2]: 2


configuring 'libhello'



Please enter the g++ binary you would like to use, for example 'g++-3.4',
'/usr/local/bin/g++' or 'distcc g++'. You can use path auto-completion.

[g++]: g++

Please select the optimization level you would like to use:

(1) -O1 [Tries to reduce code size and execution time, without
         performing any optimizations that take a great deal of
         compilation time.]
(2) -O2 [Performs nearly all supported optimizations that do not
         involve a space-speed tradeoff.]
(3) -O3 [Optimize even more.]
(4) -Os [Optimize for size.]

[2]: 2
c++ /tmp/build-0.1.12/examples/cxx/hello/libhello/libhello/hello.cxx
ld  /tmp/build-0.1.12/examples/cxx/hello/libhello/libhello/hello.l
c++ /tmp/build-0.1.12/examples/cxx/hello/libhello/test/driver.cxx
ld  /tmp/build-0.1.12/examples/cxx/hello/libhello/test/driver

Here we were presented with a bunch of configuration questions. If you have g++ in your default path you can just keep hitting enter to select defaults. If you would like to see real commands that are being executed (useful when something goes wrong) you can say make verbose=1.

From the messages above we can deduce that we have built the libhello library (hello.l) and the test driver for it (test/driver). So far so good. Now let's try to build the hello program:

$ cd /tmp/build-0.1.12/examples/cxx/hello/hello
$ make

Again you will be asked a bunch of questions you have already seen, except these two:

Configuring external dependency on 'libhello' for 'hello driver'.


Would you like to configure dependency on the installed
version as opposed to the development build?

[y]: n

Please enter the out_root for 'libhello'.

[]: ../libhello

Please enter the src_root for 'libhello'.

[]: ../libhello

The hello program depends on libhello. Since they are distributed as separate projects hello needs to know where to look for libhello. That's why user intervention is required. The first question is whether we would like to use an installed version of libhello as opposed to the not installed build. Since we haven't installed libhello the answer is no. The next two questions determine where the out_root of the build and the src_root of the source code are. Since we built libhello in its source directory both become /tmp/build-0.1.12/examples/cxx/hello/libhello or ../libhello if we are in /tmp/build-0.1.12/examples/cxx/hello/hello. After answering those questions you will see something like this:

c++ /tmp/build-0.1.12/examples/cxx/hello/hello/hello.cxx
ld  /tmp/build-0.1.12/examples/cxx/hello/hello/hello

And that's it.

One of the nice features of build is that you can have several builds in separate directories all from the same source base. Suppose we would like to build an -O3-optimized statically-linked version of hello without debug information:

$ mkdir /tmp/hello-i686-pc-linux-gnu
$ cd /tmp/hello-i686-pc-linux-gnu
$ make -f ../build-0.1.12/examples/cxx/hello/hello/makefile


configuring 'hello driver'



Please select the C++ compiler you would like to use:

(1) GNU C++ (g++)
(2) Intel C++ (icc)

[1]: 1

Would you like the C++ compiler to optimize generated code?

[y]: y

Would you like the C++ compiler to generate debug information?

[y]: n


Configuring external dependency on 'libhello' for 'hello driver'.



Would you like to configure dependency on the installed
version as opposed to the development build?

[y]: n

Please enter the out_root for 'libhello'.

[]: /tmp/libhello-i686-pc-linux-gnu

Please enter the src_root for 'libhello'.

[]: ../build-0.1.12/examples/cxx/hello/libhello


configuring 'hello driver'



Please enter the g++ binary you would like to use, for example 'g++-3.4',
'/usr/local/bin/g++' or 'distcc g++'. You can use path auto-completion.

[g++]: g++

Please select the optimization level you would like to use:

(1) -O1 [Tries to reduce code size and execution time, without
         performing any optimizations that take a great deal of
         compilation time.]
(2) -O2 [Performs nearly all supported optimizations that do not
         involve a space-speed tradeoff.]
(3) -O3 [Optimize even more.]
(4) -Os [Optimize for size.]

[2]: 3


configuring 'libhello'



Please select the C++ compiler you would like to use:

(1) GNU C++ (g++)
(2) Intel C++ (icc)

[1]: 1

Would you like the C++ compiler to optimize generated code?

[y]: y

Would you like the C++ compiler to generate debug information?

[y]: n


configuring 'libhello'



Please select the default library type:

(1) archive
(2) shared object

[2]: 1


configuring 'libhello'



Please enter the g++ binary you would like to use, for example 'g++-3.4',
'/usr/local/bin/g++' or 'distcc g++'. You can use path auto-completion.

[g++]: g++

Please select the optimization level you would like to use:

(1) -O1 [Tries to reduce code size and execution time, without
         performing any optimizations that take a great deal of
         compilation time.]
(2) -O2 [Performs nearly all supported optimizations that do not
         involve a space-speed tradeoff.]
(3) -O3 [Optimize even more.]
(4) -Os [Optimize for size.]

[2]: 3
c++ /tmp/build-0.1.12/examples/cxx/hello/hello/hello.cxx
c++ /tmp/build-0.1.12/examples/cxx/hello/libhello/libhello/hello.cxx
ar  /tmp/libhello-i686-pc-linux-gnu/libhello/hello.l
ld  /tmp/hello-i686-pc-linux-gnu/hello

Makefiles

Now let's take a look at the makefiles. We will start from the makefile for libhello (hello/libhello/libhello/makefile). Note, that I removed support for install target for now.

include $(dir $(lastword $(MAKEFILE_LIST)))../build/bootstrap.make

cxx_tun   := hello.cxx
cxx_obj   := $(addprefix $(out_base)/,$(cxx_tun:.cxx=.o))
cxx_od    := $(cxx_obj:.o=.o.d)

hello.l             := $(out_base)/hello.l
hello.l.cpp-options := $(out_base)/hello.l.cpp-options

clean     := $(out_base)/.clean


# Build.
#
$(hello.l): $(cxx_obj)

$(cxx_obj): $(hello.l.cpp-options)

$(hello.l.cpp-options): value := -I$(src_root)

$(call -include,$(cxx_od))


# Clean.
#
.PHONY: $(clean)

$(clean): $(hello.l).clean \
          $(addsuffix .clean,$(cxx_obj)) \
          $(hello.l.cpp-options).clean


# Aliases.
#
ifdef %interactive%

.PHONY: clean

clean: $(clean)

endif


# How to.
#
$(call include,$(bld_root)/cxx/o-l.make)
$(call include,$(bld_root)/cxx/cxx-o.make)

Let's examine this file line-by-line:

include $(dir $(lastword $(MAKEFILE_LIST)))../build/bootstrap.make

Every leaf makefile (i.e., one written to build a component) starts from a line that looks like this. Its sole purpose is to bootstrap the build system. In our case $(dir $(lastword ...)) will expand to something like

include .../hello/libhello/libhello/../build/bootstrap.make

Let's take a look at libhello/build/bootstrap.make:

project_name := libhello
include $(dir $(lastword $(MAKEFILE_LIST))).../build/bootstrap.make

The first line just initializes project_name with the name of the project. The second line includes the real bootstrap.make. See Bootstrapping section for more information on various ways of bootstrapping the build runtime.

What does bootstrap.make do? Besides other things (which are explained as we encounter them) it sets all those *_root and *_base variables.

Let's go back to our libhello/makefile. The first line should be clear by now so let's move on:

cxx_tun   := hello.cxx
cxx_obj   := $(addprefix $(out_base)/,$(cxx_tun:.cxx=.o))
cxx_od    := $(cxx_obj:.o=.o.d)

This should be pretty straightforward: cxx_tun contains a list of translation units, cxx_obj contains a list of object files that will be produced from those translation units, and cxx_od contains a list of automatically generated dependency files for those translation units. After expansion these variables could contain the following values (note, in this example out_root is the same as src_root):

cxx_tun   := hello.cxx
cxx_obj   := .../hello/libhello/libhello/hello.o
cxx_od    := .../hello/libhello/libhello/hello.o.d

Next chunk:

hello.l             := $(out_base)/hello.l
hello.l.cpp-options := $(out_base)/hello.l.cpp-options

clean     := $(out_base)/.clean

hello.l and clean are two variables that store names of targets. You are probably wondering what the heck is hello.l? This is a library abstraction the build system provides to deal (besides other things) with archives/shared objects uniformly. A user who builds your project can specify what type of library they want without any additional effort from you. I encourage you to take a look inside hello.l.

hello.l.cpp-options is a bit trickier. $(out_base)/hello.l.cpp-options keeps C preprocessor options that are required to compile library files as well as any piece of code that uses it. Hopefully, it will become clear once you see the rest of the makefile.

Let's move on:

# Build.
#
$(hello.l): $(cxx_obj)

$(cxx_obj): $(hello.l.cpp-options)

$(hello.l.cpp-options): value := -I$(src_root)

$(call -include,$(cxx_od))

The first line tells us that hello.l is built from object files listed in cxx_obj. The second line establishes dependency of object files on C preprocessor options - in order to build an object file we will need C++ source file and C preprocessor options - quite logical. The third line sets target-specific variable value for hello.l.cpp-options: that's how hello.l.cpp-options gets its content. You may be wondering why do we need to add this -I$(src_root). The first line of hello.cxx should clear things up:

#include "libhello/hello.hxx"

And finally the fourth line includes auto-generated dependency information for each object file. You can think of $(call -include ) as being equivalent to -include directive for now.

Next chunk:

.PHONY: $(clean)

$(clean): $(hello.l).clean \
          $(addsuffix .clean,$(cxx_obj)) \
          $(hello.l.cpp-options).clean

Let's concentrate on the last line. There we are essentially saying that cleaning libhello consists of cleaning the library, object files and C preprocessor options. How does this work? It is done using pattern rules. Part of the build system that defines how to build say %.l also defines how to clean after it: %.l.clean. You may want to look into build/cxx/gnu/o-l.make for details.

Moving on:

# Aliases.
#
ifdef %interactive%

.PHONY: clean

clean: $(clean)

endif

In this part we are creating a short alias (clean) for its long equivalent ($(out_base)/.clean). %interactive% is a system variable defined by the framework. It is initialized only if current makefile is used in interactive mode as opposed to being included.

And finally:

# How to.
#
$(call include,$(bld_root)/cxx/o-l.make)
$(call include,$(bld_root)/cxx/cxx-o.make)

In this makefile fragment we are including parts of the build system that actually know how to build. As you might have noticed, all the code that we have examined up until now was dealing with what to build. o-l.make defines a pattern rule to build libraries from object files. cxx-o.make defines a pattern rule to build object files from C++ translation units.

And this concludes our examination of the makefile for libhello. Next is the test driver( hello/libhello/test/makefile):

include $(dir $(lastword $(MAKEFILE_LIST)))../build/bootstrap.make

cxx_tun := driver.cxx
cxx_obj := $(addprefix $(out_base)/,$(cxx_tun:.cxx=.o))
cxx_od  := $(cxx_obj:.o=.o.d)

hello.l             := $(out_root)/libhello/hello.l
hello.l.cpp-options := $(out_root)/libhello/hello.l.cpp-options

driver  := $(out_base)/driver.e
clean   := $(out_base)/.clean
test    := $(out_base)/.test


# Build.
#
$(driver): $(cxx_obj) $(hello.l)

$(cxx_obj): $(hello.l.cpp-options)

$(call -include,$(cxx_od))


# Test.
#
.PHONY: $(test)

$(test): $(driver)
	$<


# Clean.
#
.PHONY: $(clean)

$(clean): $(driver).clean $(addsuffix .clean,$(cxx_obj))


# Aliases.
#
ifdef %interactive%

.PHONY: clean test

test: $(test)
clean: $(clean)

endif


# How to.
#
$(call include,$(bld_root)/cxx/o-e.make)
$(call include,$(bld_root)/cxx/cxx-o.make)


# Load build information.
#
$(call load,$(src_root)/libhello/makefile)

Do you see anything unknown or strange in this makefile? I hope you don't. Just to re-iterate, I will remove everything we definitely saw and leave only what's new:

hello.l             := $(out_root)/libhello/hello.l
hello.l.cpp-options := $(out_root)/libhello/hello.l.cpp-options


# Build.
#
$(driver): $(cxx_obj) $(hello.l)

$(cxx_obj): $(hello.l.cpp-options)


# Load build information.
#
$(call load,$(src_root)/libhello/makefile)

All of what's left deals in one way or the other with linking to libhello. The first two lines define variables that hold paths to the library and C preprocessor options (since we are in the same project we know what those paths are relative to out_root/src_root). The third line declares that driver consists of object files and should be linked with libhello. The next line says that we need C preprocessor options to build object files, just like we did for libhello. And finally the last line: what does the load function do? The load function loads rules and dependency information from the makefile specified. Note that it doesn't make variable definitions from that makefile available (or, even worse, override ones we set) - only dependencies and rules. The major benefit of doing this is in having complete dependency information along with the rules in case something gets out of date. If, for example, we change something in libhello and then execute make in test, make will be able to detect that libhello is out of date and will re-build it before building the test driver.

Now let's take a look at the makefile from the hello program (hello/hello/makefile). Remember that it is a separate project (I removed support for install target again):

include $(dir $(lastword $(MAKEFILE_LIST)))build/bootstrap.make

cxx_tun   := hello.cxx
cxx_obj   := $(addprefix $(out_base)/,$(cxx_tun:.cxx=.o))
cxx_od    := $(cxx_obj:.o=.o.d)

hello.e   := $(out_base)/hello.e
clean     := $(out_base)/.clean

# Secure default target.
#
$(hello.e):


# Import information about libhello.
#
$(call import,libhello,l: hello.l,cpp-options: hello.l.cpp-options)


# Build.
#
$(hello.e): $(cxx_obj) $(hello.l)

$(cxx_obj): $(hello.l.cpp-options)

$(call -include,$(cxx_od))


# Clean.
#
.PHONY: $(clean)

$(clean): $(hello.e).clean $(addsuffix .clean,$(cxx_obj))


# Aliases.
#
ifdef %interactive%

.PHONY: clean

clean: $(clean)

endif


# How to.
#
$(call include,$(bld_root)/cxx/o-e.make)
$(call include,$(bld_root)/cxx/cxx-o.make)

Again I will remove all the parts that we are familiar with leaving only what's new:

# Secure default target.
#
$(hello.e):


# Import information about libhello.
#
$(call import,libhello,l: hello.l,cpp-options: hello.l.cpp-options)

Let's see what's going on here. I will start from the call to the import function. It is similar to the load function with a few exceptions. First of all, since we are importing build information from a separate project we don't know where to look for its parts; this will require dynamic configuration. Secondly, we have no way to figure out where hello.l and hello.l.cpp-options are. Thus we have `l: hello.l' and `cpp-options: hello.l.cpp-options' which essentially means "initialize variable hello.l with the library path and variable hello.l.cpp-options with the C preprocessor options file path".

One side effect of the call to import function is the potential loss of the default target. Thus, the first line.

The inter-project dependency importing architecture is probably the most complicated part of the build system. It will be described in a separate section.

This concludes our step-by-step examination. It was not trivial but neither is building real software.

Inter-Project Dependencies

To be completed.

Administrivia

License

The build runtime (makefiles, scripts, etc.) is distributed under the terms of the GNU General Public License, version 2. Build documentation is distributed under the terms of the GNU Free Documentation License, version 1.2.

In particular, this means that any makefile, script, etc., that includes, calls, or otherwise links to the build runtime must be covered by a GPL-compatible license, should you decide to distribute them.

Bootstrapping

There are two ways to use the build system: your project can embed all necessary files or you can require the build runtime to be installed. The first approach allows you to make your project self-content, modulo other external dependencies it might have. The disadvantage of this approach manifests itself when your project participates in a build with inter-project dependencies established: your project and the rest of the build could use different (and potentially incompatible) versions of the runtime. There is no such problem with the second approach.

My advice is to go with the installed build unless you have a compelling reason to do otherwise.

When you go with the embedded runtime you can copy all necessary files to your project's build/ directory. When you use external build your build/bootstrap.make might look something like this:

project_name := libhello
include build-0.1/bootstrap.make

How will make find build-0.1/bootstrap.make? There are two ways this can happen: you have the runtime installed (e.g., into /usr/local/include) where make will look for it by default or you can tell make where to look using -I option in the command line or by setting MAKEFLAGS environment variable.

Versioning

Build uses a three-digit versioning scheme: X.Y.Z. X is a generation number; it is incremented when a completely new design is implemented. Y is an interface version; two build runtimes with different Y have incompatible interfaces even though the ideology is the same (if ideology changes X will most likely be incremented). Additionally, odd Y indicates that it's a development release (similar to the linux kernel). Finally, Z signifies a release number. Releases with the same X.Y have the same interface.

To allow co-existence of several versions of build on the same system, the X.Y pair is made part of the path (e.g., /usr/include/build-0.1) thus when you bootstrap the runtime (in a project-specific bootstrap.make) you specify interface version:

include build-0.1/bootstrap.make