Introduction
I hear you saying: "Yeah, yeah, another marvelous make replacement with built-in support for various compilers, automatic dependency scanning, parallel and distributed builds, and XML instead of makefiles!" On the contrary, build is a software build system that is implemented on top of GNU make. It defines extensible framework for translators, delegates most of its tasks to existing tools, and uses GNU make syntax for makefiles. Build was designed with the following tasks in mind:
- configuration
- building
- testing
- installation
As distribution of software in source code is being replaced by precompiled package distributions, greater emphasis was placed on development-time conveniences. Some of the features of build include:
- position-independent makefiles
- non-recursive multi-makefile include-based structure
- leaf makefiles are full-fledged GNU makefiles, not just variable definitions
- complete dependency graph
- inter-project dependency tracking
To understand leaf makefiles in this manual you will need basic
knowledge of how make
works. However, inside, build
uses some of the more advanced features of GNU make including
- order-only prerequisites
- pattern and target-specific variables
$(call )
and$(eval )
- different flavors of
include
directive
If you find usage of some of the make
techniques obscure
in this text you can always consult with the
GNU make
manual.
Fundamentals
Build defines a framework in terms of which user makefiles are written. Below is the list of fundamental concepts.
- Project
- is a separately distributable collection of source files. A more
intuitive way of defining a project would be to say it's what you
normally assign a version number to (e.g.,
libhello-0.0.1
) and package as a tar ball (e.g.,libhello-0.0.1.tar.gz
). A reasonably complex project usually has a hierarchy of sub-directories under its root directory. - Component
- is part of a project for which you would normally write a
makefile. For example our
libhello
could consist of two components: the library itself and a test driver. - Source root
- is a root directory of a project. For example, if we unpack our
libhello-0.0.1.tar.gz
to/tmp
and end up with a directory/tmp/libhello-0.0.1
then that would be this project's source root. - Source base
- is a directory of a component in a project. Continuing our example,
if our
libhello
project had a subdirectorytest
then the source base of thetest
component would be/tmp/libhello-0.0.1/test
. By definition, any source base is a sub-directory of a source root for any particular project. - Out root
- is a root directory of a project's build (or output) hierarchy. This
directory signifies where generated files for the project will be placed.
For example, if we decide to build
libhello
in/tmp/libhello-i686-pc-linux-gnu
then that would be one of the possible out roots. - Out base
- is a build (or output) directory of a component in a project.
If we continue our example, the out base for the
test
component would be/tmp/libhello-i686-pc-linux-gnu/test
.
The build runtime defines four make
variables that
correspond to the last four definitions:
src_root src_base out_root out_base
Below is an example of how you could use some of them.
driver := $(out_base)/driver $(driver): $(out_base)/driver.o $(out_root)/libhello/libhello.so
Additionally, the build framework defines three more variables:
bld_root scf_root := $(src_root)/build dcf_root := $(out_root)/build
bld_root
is a root directory of the build runtime.
Normally you would use it to include one of the build's files.
scf_root
stands for static configuration root and
points to a directory where project's static configuration is kept.
dcf_root
stands for dynamic configuration root and
points to a directory where project's dynamic configuration is kept.
We will talk more about those two variables later.
Also note that having src_root
the same as
out_root
is perfectly valid and indicates that we
are building in a source directory.
First Example
Now we are ready to examine our first simple example. We
will skip one-file C-based "hello world", however, because it
is not very practical and there is not much room for improvement.
Rather we will start from a libhello
library, a test
driver for it and a hello
program that uses the library.
To make our example even more realistic the libhello
library and the hello
program are two separate projects.
And, did I mention, they are written in C++. See
examples/cxx/hello
.
Invocation
Let's first get some user experience without looking into makefiles.
Suppose we unpacked build source into /tmp/build-0.1.12
.
First let's try to build libhello
:
$ cd /tmp/build-0.1.12/examples/cxx/hello/libhello $ make configuring 'libhello' Please select the C++ compiler you would like to use: (1) GNU C++ (g++) (2) Intel C++ (icc) [1]: 1 Would you like the C++ compiler to optimize generated code? [y]: y Would you like the C++ compiler to generate debug information? [y]: y configuring 'libhello' Please select the default library type: (1) archive (2) shared object [2]: 2 configuring 'libhello' Please enter the g++ binary you would like to use, for example 'g++-3.4', '/usr/local/bin/g++' or 'distcc g++'. You can use path auto-completion. [g++]: g++ Please select the optimization level you would like to use: (1) -O1 [Tries to reduce code size and execution time, without performing any optimizations that take a great deal of compilation time.] (2) -O2 [Performs nearly all supported optimizations that do not involve a space-speed tradeoff.] (3) -O3 [Optimize even more.] (4) -Os [Optimize for size.] [2]: 2 c++ /tmp/build-0.1.12/examples/cxx/hello/libhello/libhello/hello.cxx ld /tmp/build-0.1.12/examples/cxx/hello/libhello/libhello/hello.l c++ /tmp/build-0.1.12/examples/cxx/hello/libhello/test/driver.cxx ld /tmp/build-0.1.12/examples/cxx/hello/libhello/test/driver
Here we were presented with a bunch of configuration questions.
If you have g++
in your default path you can just keep
hitting enter to select defaults. If you would like to see real commands
that are being executed (useful when something goes wrong) you can
say make verbose=1
.
From the messages above we can deduce that we have built the
libhello
library (hello.l
) and the test driver
for it (test/driver
). So far so good. Now let's try to build
the hello
program:
$ cd /tmp/build-0.1.12/examples/cxx/hello/hello $ make
Again you will be asked a bunch of questions you have already seen, except these two:
Configuring external dependency on 'libhello' for 'hello driver'. Would you like to configure dependency on the installed version as opposed to the development build? [y]: n Please enter the out_root for 'libhello'. []: ../libhello Please enter the src_root for 'libhello'. []: ../libhello
The hello
program depends on libhello
. Since
they are distributed as separate projects hello
needs to
know where to look for libhello
. That's why user intervention
is required. The first question is whether we would like to use an installed
version of libhello
as opposed to the not installed build.
Since we haven't installed libhello
the answer is
no
. The next two questions determine where the
out_root
of the build and the src_root
of the source
code are. Since we built libhello
in its source directory both
become /tmp/build-0.1.12/examples/cxx/hello/libhello
or
../libhello
if we are in
/tmp/build-0.1.12/examples/cxx/hello/hello
. After answering
those questions you will see something like this:
c++ /tmp/build-0.1.12/examples/cxx/hello/hello/hello.cxx ld /tmp/build-0.1.12/examples/cxx/hello/hello/hello
And that's it.
One of the nice features of build is that you can have several
builds in separate directories all from the same source base. Suppose we
would like to build an -O3
-optimized statically-linked version
of hello
without debug information:
$ mkdir /tmp/hello-i686-pc-linux-gnu $ cd /tmp/hello-i686-pc-linux-gnu $ make -f ../build-0.1.12/examples/cxx/hello/hello/makefile configuring 'hello driver' Please select the C++ compiler you would like to use: (1) GNU C++ (g++) (2) Intel C++ (icc) [1]: 1 Would you like the C++ compiler to optimize generated code? [y]: y Would you like the C++ compiler to generate debug information? [y]: n Configuring external dependency on 'libhello' for 'hello driver'. Would you like to configure dependency on the installed version as opposed to the development build? [y]: n Please enter the out_root for 'libhello'. []: /tmp/libhello-i686-pc-linux-gnu Please enter the src_root for 'libhello'. []: ../build-0.1.12/examples/cxx/hello/libhello configuring 'hello driver' Please enter the g++ binary you would like to use, for example 'g++-3.4', '/usr/local/bin/g++' or 'distcc g++'. You can use path auto-completion. [g++]: g++ Please select the optimization level you would like to use: (1) -O1 [Tries to reduce code size and execution time, without performing any optimizations that take a great deal of compilation time.] (2) -O2 [Performs nearly all supported optimizations that do not involve a space-speed tradeoff.] (3) -O3 [Optimize even more.] (4) -Os [Optimize for size.] [2]: 3 configuring 'libhello' Please select the C++ compiler you would like to use: (1) GNU C++ (g++) (2) Intel C++ (icc) [1]: 1 Would you like the C++ compiler to optimize generated code? [y]: y Would you like the C++ compiler to generate debug information? [y]: n configuring 'libhello' Please select the default library type: (1) archive (2) shared object [2]: 1 configuring 'libhello' Please enter the g++ binary you would like to use, for example 'g++-3.4', '/usr/local/bin/g++' or 'distcc g++'. You can use path auto-completion. [g++]: g++ Please select the optimization level you would like to use: (1) -O1 [Tries to reduce code size and execution time, without performing any optimizations that take a great deal of compilation time.] (2) -O2 [Performs nearly all supported optimizations that do not involve a space-speed tradeoff.] (3) -O3 [Optimize even more.] (4) -Os [Optimize for size.] [2]: 3 c++ /tmp/build-0.1.12/examples/cxx/hello/hello/hello.cxx c++ /tmp/build-0.1.12/examples/cxx/hello/libhello/libhello/hello.cxx ar /tmp/libhello-i686-pc-linux-gnu/libhello/hello.l ld /tmp/hello-i686-pc-linux-gnu/hello
Makefiles
Now let's take a look at the makefiles. We will start from the
makefile for libhello
(hello/libhello/libhello/makefile
).
Note, that I removed support for install
target for now.
include $(dir $(lastword $(MAKEFILE_LIST)))../build/bootstrap.make cxx_tun := hello.cxx cxx_obj := $(addprefix $(out_base)/,$(cxx_tun:.cxx=.o)) cxx_od := $(cxx_obj:.o=.o.d) hello.l := $(out_base)/hello.l hello.l.cpp-options := $(out_base)/hello.l.cpp-options clean := $(out_base)/.clean # Build. # $(hello.l): $(cxx_obj) $(cxx_obj): $(hello.l.cpp-options) $(hello.l.cpp-options): value := -I$(src_root) $(call -include,$(cxx_od)) # Clean. # .PHONY: $(clean) $(clean): $(hello.l).clean \ $(addsuffix .clean,$(cxx_obj)) \ $(hello.l.cpp-options).clean # Aliases. # ifdef %interactive% .PHONY: clean clean: $(clean) endif # How to. # $(call include,$(bld_root)/cxx/o-l.make) $(call include,$(bld_root)/cxx/cxx-o.make)
Let's examine this file line-by-line:
include $(dir $(lastword $(MAKEFILE_LIST)))../build/bootstrap.make
Every leaf makefile (i.e., one written to build a component) starts from a
line that looks like this. Its sole purpose is to bootstrap the build system.
In our case $(dir $(lastword ...))
will expand to something
like
include .../hello/libhello/libhello/../build/bootstrap.make
Let's take a look at libhello/build/bootstrap.make
:
project_name := libhello include $(dir $(lastword $(MAKEFILE_LIST))).../build/bootstrap.make
The first line just initializes project_name
with the name
of the project. The second line includes the real bootstrap.make
.
See Bootstrapping section for more information on
various ways of bootstrapping the build runtime.
What does bootstrap.make
do? Besides other things (which are
explained as we encounter them) it sets all those *_root
and
*_base
variables.
Let's go back to our libhello/makefile
. The first line
should be clear by now so let's move on:
cxx_tun := hello.cxx cxx_obj := $(addprefix $(out_base)/,$(cxx_tun:.cxx=.o)) cxx_od := $(cxx_obj:.o=.o.d)
This should be pretty straightforward: cxx_tun
contains
a list of translation units, cxx_obj
contains a list of
object files that will be produced from those translation units, and
cxx_od
contains a list of automatically generated dependency
files for those translation units. After expansion these variables could
contain the following values (note, in this example out_root
is the same as src_root
):
cxx_tun := hello.cxx cxx_obj := .../hello/libhello/libhello/hello.o cxx_od := .../hello/libhello/libhello/hello.o.d
Next chunk:
hello.l := $(out_base)/hello.l hello.l.cpp-options := $(out_base)/hello.l.cpp-options clean := $(out_base)/.clean
hello.l
and clean
are two variables that
store names of targets. You are probably wondering what the heck
is hello.l
? This is a library abstraction the build system
provides to deal (besides other things) with archives/shared objects
uniformly. A user who builds your project can specify what type of
library they want without any additional effort from you. I encourage
you to take a look inside hello.l
.
hello.l.cpp-options
is a bit trickier.
$(out_base)/hello.l.cpp-options
keeps C preprocessor options
that are required to compile library files as well as any piece of code
that uses it. Hopefully, it will become clear once you see the rest of
the makefile.
Let's move on:
# Build. # $(hello.l): $(cxx_obj) $(cxx_obj): $(hello.l.cpp-options) $(hello.l.cpp-options): value := -I$(src_root) $(call -include,$(cxx_od))
The first line tells us that hello.l
is built from object
files listed in cxx_obj
. The second line establishes dependency
of object files on C preprocessor options - in order to build an object file
we will need C++ source file and C preprocessor options - quite logical. The
third line sets target-specific variable value
for
hello.l.cpp-options
: that's how hello.l.cpp-options
gets its content. You may be wondering why do we need to add this
-I$(src_root)
. The first line of hello.cxx
should clear things up:
#include "libhello/hello.hxx"
And finally the fourth line includes auto-generated dependency
information for each object file. You can think of
$(call -include )
as being equivalent to -include
directive for now.
Next chunk:
.PHONY: $(clean) $(clean): $(hello.l).clean \ $(addsuffix .clean,$(cxx_obj)) \ $(hello.l.cpp-options).clean
Let's concentrate on the last line. There we are essentially saying that
cleaning libhello
consists of cleaning the library, object
files and C preprocessor options. How does this work? It is done using
pattern rules. Part of the build system that defines how to build say
%.l
also defines how to clean after it: %.l.clean
.
You may want to look into build/cxx/gnu/o-l.make
for details.
Moving on:
# Aliases. # ifdef %interactive% .PHONY: clean clean: $(clean) endif
In this part we are creating a short alias (clean
)
for its long equivalent ($(out_base)/.clean
).
%interactive%
is a system variable defined by
the framework. It is initialized only if current makefile is
used in interactive mode as opposed to being included.
And finally:
# How to. # $(call include,$(bld_root)/cxx/o-l.make) $(call include,$(bld_root)/cxx/cxx-o.make)
In this makefile fragment we are including parts of the build
system that actually know how to build. As you might have
noticed, all the code that we have examined up until now was dealing
with what to build. o-l.make
defines a pattern
rule to build libraries from object files. cxx-o.make
defines a pattern rule to build object files from C++ translation units.
And this concludes our examination of the makefile for
libhello
. Next is the test driver(
hello/libhello/test/makefile
):
include $(dir $(lastword $(MAKEFILE_LIST)))../build/bootstrap.make cxx_tun := driver.cxx cxx_obj := $(addprefix $(out_base)/,$(cxx_tun:.cxx=.o)) cxx_od := $(cxx_obj:.o=.o.d) hello.l := $(out_root)/libhello/hello.l hello.l.cpp-options := $(out_root)/libhello/hello.l.cpp-options driver := $(out_base)/driver.e clean := $(out_base)/.clean test := $(out_base)/.test # Build. # $(driver): $(cxx_obj) $(hello.l) $(cxx_obj): $(hello.l.cpp-options) $(call -include,$(cxx_od)) # Test. # .PHONY: $(test) $(test): $(driver) $< # Clean. # .PHONY: $(clean) $(clean): $(driver).clean $(addsuffix .clean,$(cxx_obj)) # Aliases. # ifdef %interactive% .PHONY: clean test test: $(test) clean: $(clean) endif # How to. # $(call include,$(bld_root)/cxx/o-e.make) $(call include,$(bld_root)/cxx/cxx-o.make) # Load build information. # $(call load,$(src_root)/libhello/makefile)
Do you see anything unknown or strange in this makefile? I hope you don't. Just to re-iterate, I will remove everything we definitely saw and leave only what's new:
hello.l := $(out_root)/libhello/hello.l hello.l.cpp-options := $(out_root)/libhello/hello.l.cpp-options # Build. # $(driver): $(cxx_obj) $(hello.l) $(cxx_obj): $(hello.l.cpp-options) # Load build information. # $(call load,$(src_root)/libhello/makefile)
All of what's left deals in one way or the other with linking
to libhello
. The first two lines define variables that
hold paths to the library and C preprocessor options (since we are in
the same project we know what those paths are relative to
out_root
/src_root
). The third line declares that
driver consists of object files and should be linked with
libhello
. The next line says that we need C preprocessor options
to build object files, just like we did for libhello
. And finally
the last line: what does the load
function do? The
load
function loads rules and dependency information from the
makefile specified. Note that it doesn't make variable definitions from that
makefile available (or, even worse, override ones we set) - only dependencies
and rules. The major benefit of doing this is in having complete dependency
information along with the rules in case something gets out of date. If, for
example, we change something in libhello
and then execute
make
in test
, make
will be able to
detect that libhello
is out of date and will re-build it before
building the test driver.
Now let's take a look at the makefile from the hello
program
(hello/hello/makefile
). Remember that it is a separate project
(I removed support for install
target again):
include $(dir $(lastword $(MAKEFILE_LIST)))build/bootstrap.make cxx_tun := hello.cxx cxx_obj := $(addprefix $(out_base)/,$(cxx_tun:.cxx=.o)) cxx_od := $(cxx_obj:.o=.o.d) hello.e := $(out_base)/hello.e clean := $(out_base)/.clean # Secure default target. # $(hello.e): # Import information about libhello. # $(call import,libhello,l: hello.l,cpp-options: hello.l.cpp-options) # Build. # $(hello.e): $(cxx_obj) $(hello.l) $(cxx_obj): $(hello.l.cpp-options) $(call -include,$(cxx_od)) # Clean. # .PHONY: $(clean) $(clean): $(hello.e).clean $(addsuffix .clean,$(cxx_obj)) # Aliases. # ifdef %interactive% .PHONY: clean clean: $(clean) endif # How to. # $(call include,$(bld_root)/cxx/o-e.make) $(call include,$(bld_root)/cxx/cxx-o.make)
Again I will remove all the parts that we are familiar with leaving only what's new:
# Secure default target. # $(hello.e): # Import information about libhello. # $(call import,libhello,l: hello.l,cpp-options: hello.l.cpp-options)
Let's see what's going on here. I will start from the call to
the import
function. It is similar to the load
function with a few exceptions. First of all, since we are importing
build information from a separate project we don't know where to look
for its parts; this will require dynamic configuration. Secondly, we
have no way to figure out where hello.l
and
hello.l.cpp-options
are. Thus we have
`l: hello.l'
and `cpp-options: hello.l.cpp-options'
which essentially means "initialize variable hello.l
with
the library path and variable hello.l.cpp-options
with
the C preprocessor options file path".
One side effect of the call to import
function is the
potential loss of the default target. Thus, the first line.
The inter-project dependency importing architecture is probably the most complicated part of the build system. It will be described in a separate section.
This concludes our step-by-step examination. It was not trivial but neither is building real software.
Inter-Project Dependencies
To be completed.
Administrivia
License
The build runtime (makefiles, scripts, etc.) is distributed under the terms of the GNU General Public License, version 2. Build documentation is distributed under the terms of the GNU Free Documentation License, version 1.2.
In particular, this means that any makefile, script, etc., that includes, calls, or otherwise links to the build runtime must be covered by a GPL-compatible license, should you decide to distribute them.
Bootstrapping
There are two ways to use the build system: your project can embed all necessary files or you can require the build runtime to be installed. The first approach allows you to make your project self-content, modulo other external dependencies it might have. The disadvantage of this approach manifests itself when your project participates in a build with inter-project dependencies established: your project and the rest of the build could use different (and potentially incompatible) versions of the runtime. There is no such problem with the second approach.
My advice is to go with the installed build unless you have a compelling reason to do otherwise.
When you go with the embedded runtime you can copy all necessary files
to your project's build/
directory. When you use external
build your build/bootstrap.make
might look something
like this:
project_name := libhello include build-0.1/bootstrap.make
How will make
find build-0.1/bootstrap.make
?
There are two ways this can happen: you have the runtime installed (e.g.,
into /usr/local/include
) where make
will look
for it by default or you can tell make
where to look using
-I
option in the command line or by setting
MAKEFLAGS
environment variable.
Versioning
Build uses a three-digit versioning scheme: X.Y.Z
.
X
is a generation number; it is incremented when a completely
new design is implemented. Y
is an interface version; two
build runtimes with different Y
have incompatible
interfaces even though the ideology is the same (if ideology changes
X
will most likely be incremented). Additionally, odd
Y
indicates that it's a development release (similar to
the linux kernel). Finally, Z
signifies a release number.
Releases with the same X.Y
have the same interface.
To allow co-existence of several versions of build on the
same system, the X.Y
pair is made part of the path (e.g.,
/usr/include/build-0.1
) thus when you bootstrap the runtime
(in a project-specific bootstrap.make
) you specify interface
version:
include build-0.1/bootstrap.make