Aquefir

Massively multipurpose software.

Slick Makefiles

Penned on the 4th day of September, 2020. It was a Friday.

Last updated on the 21st day of February, 2021. It was a Sunday.

What are these?

These Makefiles are used for building systems software projects. They support compilation of assembly, C, C++ and C* code as well as a variety of data/asset formats into object code. The Slick Makefiles are a living example showing how meta-build systems like CMake and SCons are unnecessary to target many platforms. They are a vastly simpler answer to the problem of building software, and they make no sacrifices in ability to output.

There are two files comprising the distribution: base.mk and targets.mk. In the project root where the Makefile resides, a folder etc/ is created to contain these two files. In the Makefile, base.mk is included at the top, before anything else; targets.mk is included last, after the declaration of sources, flags and such.

Provisions of the Makefiles

There are a few kinds of variables provided by the Makefiles: environment identifiers, programs, program flags, and suffixes. Each of these categories has different properties of modifiability and configuration.

As variables are listed, they will have a few attribute markers displayed next to them, showing whether the variables are mutable (i.e. modifiable) using a Greek delta Δ, whether they are cascading (change other variable defaults) using a Greek xi Ξ, and whether they are “shelled”, i.e. modifiable from the shell command line or shell environment using a Greek sigma Σ.

There are two special environment variables the Slick Makefiles will recognise for altering behaviour: SLICK_PRINT and SLICK_OVERRIDE. The former will enable the printing of build variable values for diagnostics, and the latter will cause the definitions of flag variables to fully override the default values instead of appending to them, the default behaviour.

Environment identifiers

Environment identifiers, or envidents for short, are variables provided to application developers to dictate the desired toolchain and the machine being targeted. This is more robust than having developers modify the other kinds of variables directly, as it is meant to automatically figure out desired defaults.

UNAME
Δ
Ξ
Σ
The host platform. This is auto-detected and cannot be overridden. The default value for $TP is set to this variable.
Valid values: Darwin, Linux
TP
Δ
Ξ
Σ
The target platform. The default value for $TC is discerned from this variable.
Valid values: Darwin, GBA, Linux, Win32, Win64
TC
Δ
Ξ
Σ
Toolchain in use. This is used to default compilers and the linker.
Valid values: gnu, llvm
TROOT
Δ
Ξ
Σ
Target sysroot. This is the folder containing the target platform’s lib/ and include/ subdirectories for use in portable variable definitions in the user’s Makefile.
Valid values: Any absolute path to a valid directory containing at least lib/ and include/ as subdirectories.

Programs

These are pretty self-explanatory. Compilers, assemblers, transmogrifiers, linkers, script interpreters, and any other utilities with canonical names are on this list.

AS
Δ
Ξ
Σ
The native assembler. Note that this is undefined when $TC is llvm, as LLVM’s assembler is for IR. When using LLVM with projects depending on assembly, it is recommended to set this variable to the appropriate assembler in the Makefile after including base.mk.
Valid values: Any absolute path to a valid executable.
CC
Δ
Ξ
Σ
The C compiler. When $TP is Linux, setting this to a path where $(basename) evaluates to tcc causes the Makefiles to adjust for using the Tiny C Compiler. This is used as the default $LD, unless C++ sources are present.
Valid values: Any absolute path to a compliant C89 compiler.
CXX
Δ
Ξ
Σ
The C++ compiler. When C++ sources are present, this variable is used as the default $LD.
Valid values: Any absolute path to a compliant C++11 compiler.
CXC
Δ
Ξ
Σ
The C* compiler.
Valid values: Any absolute path to a compliant C* compiler.
AR
Δ
Ξ
Σ
The static library archiver.
Valid values: Any absolute path to a valid executable.
LD
Δ
Ξ
Σ
The program linker.
Valid values: Any absolute path to a valid executable.
OCPY
Δ
Ξ
Σ
The object file copier.
Valid values: Any absolute path to a valid executable.
STRIP
Δ
Ξ
Σ
The symbol stripper. This is only used on release builds.
Valid values: Any absolute path to a valid executable.
PY
Δ
Ξ
Σ
The Python 3 interpreter.
Valid values: Any absolute path to a compliant Python 3.4+ interpreter.
PL
Δ
Ξ
Σ
The Perl interpreter.
Valid values: Any absolute path to a compliant Perl 5 interpreter.
FMT
Δ
Ξ
Σ
The C/C++ auto-formatter
Valid values: Any absolute path to a clang-format command-compatible code auto-formatter.

Program flags

These are options, in the form of space-separated array-strings, passed to many of the above programs. When these are set in the main Makefile or in the shell, the contents of those definitions will be appended to the default values; this behaviour can be undone by setting the SLICK_OVERRIDE variable on invocation. This category also includes file parameters for various programs, including the commonly named LIBS, LIBDIRS and DEFINES.

ASFLAGS
Δ
Ξ
Σ
Flags for the native assembler.
Valid values: See the program manual for details.
CFLAGS
Δ
Ξ
Σ
Flags for the C compiler.
Valid values: See the program manual for details.
CPPFLAGS
Δ
Ξ
Σ
Flags for the C preprocessor.
Valid values: See the program manual for details.
CXXFLAGS
Δ
Ξ
Σ
Flags for the C++ compiler.
Valid values: See the program manual for details.
CXCFLAGS
Δ
Ξ
Σ
Flags for the C* compiler.
Valid values: See the program manual for details.
ARFLAGS
Δ
Ξ
Σ
Flags for the static library archiver.
Valid values: See the program manual for details.
LDFLAGS
Δ
Ξ
Σ
Flags for the program linker.
Valid values: See the program manual for details.
LIBS
Δ
Ξ
Σ
Space-separated list of libraries to link to.
Valid values: Any name that resolves to a library file in one of the $LIBDIRS.
LIBDIRS
Δ
Ξ
Σ
Space-separated list of directories to have the linker search for libraries in.
Valid values: Any path to a valid directory.
DEFINES
Δ
Ξ
Σ
Space-separated list of macro constants to define for the C preprocessor and the assembler.
Valid values: Any alphanumeric string that does not begin with a number. It can also include underscores.
UNDEFINES
Δ
Ξ
Σ
Space-separated list of macro constants to explicitly undefine for the C preprocessor. Note that unlike $DEFINES this does not propagate to the assembler, as the default lacks a flag for undefining macro constants.
Valid values: Any alphanumeric string that does not begin with a number. It can also include underscores.
SYNDEFS
Δ
Ξ
Σ
Synthetic #defines are providedto the $CPP and the assembler to allow code to be made more amicable to the specifics of different machines. They are always prefixed with CFG_. The glossary of such definitions is provided in the appendix.
Valid values:

Suffixes

This group of variables serve as glue for portability of files, as these Makefiles support cross-compilation as the only strategy for many targets. By design, these are not modifiable directly; if changing them is necessary, change the target platform instead.

SO
Δ
Ξ
Σ
The shared library file extension, including the dot.
Valid values: Any number of lowercase letters following a dot, e.g. .so, .dll, …
EXE
Δ
Ξ
Σ
The executable file extension, including the dot. Usually blank, except when targeting certain platforms like Windows where it is .exe, or GBA where it is .elf.
Valid values: Any number of lowercase letters following a dot, e.g. .exe, .elf, …

Creating a Slick Makefile

This section covers how to write the actual project Makefile, using the Slick scaffolding detailed here.

Note
Other than the sources, all of the variables mentioned in this section must be defined for the Makefile to function, even if they are ultimately blank. This section shows all of the possible source file types and their canonical names, which currently includes assembly, C, C++, and C* code.

First, include base.mk. If the reference packages are in use, it can be pathed relative to the $AQ environment variable:

include $(AQ)/lib/slick/base.mk

The project needs to be named with an identifier. This can be anything, but it is best to pick something concise and alphanumeric only, as it is used to construct the output file name(s).

PROJECT := mycommand

Next, define the desired targets. For instance:

# put a ‘1’ for the desired target types to compile
EXEFILE := 1
SOFILE  :=
AFILE   :=

Define system and local #includes, as well as any libraries and the folders to find them in. Usually, working defaults will be set up automatically for these, but values can be set here to get appended to the ultimate values used.

# space-separated path list for #includes
# <system> includes
INCLUDES :=
# "local" includes
INCLUDEL := src

# space-separated library name list
LIBS    :=
LIBDIRS :=

Now define all of the sources. These are each space-separated lists of relative file paths, and the conventional way to organise them is using backslashes to have one file per line, like so:

# sources
SFILES    :=
CFILES    := \
	src/errors.c \
	src/extra.c \
	src/main.c
CPPFILES  :=
CSTFILES  :=
PUBHFILES := \
	include/extra.h
PRVHFILES := \
	src/errors.h \
	src/extra.h

# test suite sources
TES_SFILES    :=
TES_CFILES    :=
TES_CPPFILES  :=
TES_CSTFILES  :=
TES_PUBHFILES :=
TES_PRVHFILES :=

In order for them to be picked up by the auto-formatter, header files may be listed as well. At present there is no other use for the variables. There are also a second set of sources for test batteries.

Finally, include targets.mk to make the thing work:

include $(AQ)/lib/slick/targets.mk

Configuring the Slick Makefiles

There are several techniques that can be used to modify the behaviour of the Slick Makefiles to work differently. This section goes over these one-by-one with explanations and rationales.

Public and private headers as APIs

Slick provides distinct variables $INCLUDES and $INCLUDEL for public and private include folders, respectively. This makes it easy to separate private header files that are only consumed internally from the public API provided to users in library code. This also combines with ADP 1’s provision of a distinct include/ folder for public headers, so it is clear that headers in src/ are private. Since $INCLUDES translates to angle-bracket includes and $INCLUDEL translates to quoted includes, it becomes obvious and consistent in source code what files are coming from where, and what duties are imposed upon them by the project’s constraints.

Overriding the linker

Normally, the linker is automatically set as needed based on the composition of a project’s sources. However, it may be necessary to override the linker anyway in order for a project to build. One situation where this is needed is with a project using only C sources that statically links into a C++ library; the linker will not auto-detect to use C++ because there are no $CPPFILES, but there are C++ ofiles inside the static library which will fail to link otherwise. This can be done with a line like so, written just before targets.mk is included:

LD := $(CXX)

Choosing a target type

The Slick Makefiles provide six (6) different targets to choose from. The default is debug, but there are also release, check, cov, asan and ubsan as well. Here are the differences:

Advanced tools & concepts

There are many features provided by the Slick Makefiles that provide for advanced use cases.

Dot-infixing for platform-specific sources

The Makefiles provide a novel dot-infix notation for specifying sources only intended to be compiled for a specific target platform. This is especially needed in projects that use assembly code, as it is a non-portable source code form, but it is provided for all types of sources as well as test batteries. Simply affix the desired $TP identifier in all uppercase to the sources with a dot, like so:

SFILES.GBA := \
	src/gbabios.s
CFILES.DARWIN := \
	src/macstuff.c \
	src/swifty.c

In-tree dependency management

The Slick Makefiles only run on Unix-like operating systems. Resultantly, the use of sysroots for dependency management is highly encouraged, even for all kinds of cross-compilation endeavours. However, it can often prove wiser to install dependencies in-tree, into a dedicated subdirectory of the project’s repository. The Slick Makefiles have a system to support this as well, using a couple of variables and some additional logic in the backend.

The two variables that need to be set for in-tree dependency management are $3PLIBDIR and $3PLIBS. $3PLIBDIR is a directory path, relative to the repository root, in which subfolders for each dependency are kept. ADP 1 recommends that this be set to 3rdparty, as the spec makes a special allowance for that name, but it can be set to point anywhere, including absolute paths and symbolic links. $3PLIBS is a space-separated list of names of each library that is an in-tree dependency, without the lib prefix.

Note
For in-tree dependency management to function, it is required that the child repositories follow ADP 1, and that their folder names are of the form project_namelib while their compiled binaries are placed into their own repository roots.

Targeting APE with Cosmopolitan

Slick supports targeting the Actually Portable Executable polyglot using its accompanying Cosmopolitan libc. To do this, five (5) file paths to the amalgamated sources of Cosmopolitan must be specified before the inclusion of targets.mk:

  1. APE_LDSCR: path to ape.lds
  2. APE_AFILE: path to cosmopolitan.a
  3. APE_HFILE: path to cosmopolitan.h
  4. APE_CRTO: path to crt.o
  5. APE_APEO: path to ape.o

Usually, it suffices to have these extracted into a subfolder of etc/ in compliance with ADP 1. Many Aquefir projects use an automated getcosmo.sh script to fetch and verify the amalgamated sources, along with .gitignore usage to block out the downloads from the source tree. It is also possible to exclusively target APE by explicitly setting TP := APE before including base.mk.

Slick takes care of applying the appropriate flags where they’re needed, as well as fixing the ordering of the various options. It should “just work” if all five variables are provided and present.

Appendix & references

List of synthetic definitions

DARWIN
Apple macOS.
LINUX
GNU/Linux.
WINDOWS
Microsoft Windows, 32-bit and 64-bit.
GBA
The Nintendo Game Boy Advance.
IBMPC
Baremetal x86-based IBM-PC compatibles.
AMD64
The AMD 64-bit extensions to Intel’s IA-32 architecture.
IA32
Intel’s i386 architecture, and later updates up to and including Pentium Pro (i686).
I86
Intel’s 8086 architecture.
I186
Intel’s iAPX 186 architecture.
I286
Intel’s i286 architecture.
ARMV4T
ARMv4 architecture with Thumb-1 extensions.
LILENDIAN
Little endian word arrangement.
BIGENDIAN
Big endian word arrangement.
WIN32
32-bit version of the Win32 APIs.
WIN64
64-bit version of the Win32 APIs.
WORDSZ_16
The size of a CPU word (register) is 16 bits.
WORDSZ_32
The size of a CPU word (register) is 32 bits.
WORDSZ_64
The size of a CPU word (register) is 64 bits.
HAVE_I32
Machine has 32-bit integer primitives.
HAVE_I64
Machine has 64-bit integer primitives.
HAVE_FP
Target supports floating-point instructions.
FP_HARD
Machine has hardware floating-point support.
FP_SOFT
Target has software floating-point support.
LONGSZ_32
The size of a long is 32 bits.
LONGSZ_64
The size of a long is 64 bits.

Referenced materials & suggested reading

  1. Aquefir. “ADP 1.” Project repository filesystem schema. <https://aquefir.co/adp1>
  2. Aquefir on GitHub. “Slick.” The Slick Makefiles. <https://github.com/aquefir/slick>
  3. Aquefir on GitHub. “Test Engineering Suite (TES).” Aquefir Test Engineering Suite. <https://github.com/aquefir/teslib>
  4. Tunney, Justine. Actually Portable Executables. A polyglot for portable native executables. <https://justine.lol/ape.html>
  5. Tunney, Justine. “Cosmopolitan libc.” A libc implementation for Actually Portable Executables. <https://justine.lol/cosmopolitan/index.html>