FetchContent_MakeHermetic ¶

Set the configuration variables of Hermetic FetchContent for the content

Overview ¶

This module allows to setup an Hermetic FetchContent configuration for a given content, augmenting what FetchContent can do by enabling consuming dependencies that are configured in a separate cmake execution.

The following shows a typical example of declaring content detail for an examplary dependency:

include(HermeticFetchContent)

# Hermetic FetchContent consumes information of FetchContent contents declared using the same name
FetchContent_Declare(
  zstd
  GIT_REPOSITORY https://github.com/facebook/zstd.git
  GIT_TAG 63779c798237346c2b245c546c40b72a5a5913fe
  SOURCE_SUBDIR build/cmake
)

# additional Hermetic FetchContent configuration
FetchContent_MakeHermetic(
  zstd
  HERMETIC_BUILD_SYSTEM cmake
)

# choose when this content gets build
HermeticFetchContent_MakeAvailableAtConfigureTime(zstd)
# OR
HermeticFetchContent_MakeAvailableAtBuildTime(zstd)

Hermetic FetchContent does consume information defined in FetchContent_Declare calls whenever possible but enables additional configuration through the FetchContent_MakeHermetic() command.

Commands ¶

FetchContent_MakeHermetic¶

FetchContent_MakeHermetic(
  <name>
  [HERMETIC_BUILD_SYSTEM cmake | autotools | openssl]
  [HERMETIC_TOOLCHAIN_EXTENSION <cmake code>]
  [HERMETIC_CONFIG_EXTRA_ARGS <configure flags>...]
  [HERMETIC_CONFIG_LANGUAGE C | CXX]
  [HERMETIC_FIND_PACKAGES <list of hermetic content names>]
  [HERMETIC_ADDITIONAL_TOOLCHAIN_FINGERPRINT_VARIABLES <list of variable names>]
  [HERMETIC_CREATE_TARGET_ALIASES <cmake code>]
  [HERMETIC_PREPATCHED_RESOLVER <cmake code>]
  [HERMETIC_CMAKE_EXPORT_LIBRARY_DECLARATION <cmake code>]
  [HERMETIC_DISCOVER_TARGETS_FILE_PATTERN <regex pattern>]
)

The FetchContent_MakeHermetic() function records options that describe the additional parameters required to populate and consume the specified content in a hermetic way.

The content <name> can be any string without spaces, but good practice would be to use only letters, numbers and underscores.

The HERMETIC_BUILD_SYSTEM allows selecting which build system scheme will be used to configure and build the specified content. Currently the available choices are:

cmake
For contents that come with a working CMake buildsystem

autotools
For contents using the GNU Autotools as build system

openssl
To consume the native openssl buildsystem

The HERMETIC_TOOLCHAIN_EXTENSION option enables injecting code into the toolchain used in the isolated build of the specified content, enabling to set the required build configuration without interfering with the build configuration of other parts of your project.

FetchContent_MakeHermetic(
  rapidjson
  HERMETIC_BUILD_SYSTEM cmake
  HERMETIC_TOOLCHAIN_EXTENSION [=[
    # don't build doc, examples & tests
    set(RAPIDJSON_BUILD_DOC OFF CACHE BOOL "" FORCE)
    set(RAPIDJSON_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
    set(RAPIDJSON_BUILD_TESTS OFF CACHE BOOL "" FORCE)

    # use std::string
    set(RAPIDJSON_HAS_STDSTRING ON CACHE BOOL "" FORCE)
  ]=]
)

Additionally to these dependency specific toolchain fragments Hermetic FetchContent is able to forward a defined set of CMake variables to the sub-builds by injecting values of the variables specified in the global HERMETIC_FETCHCONTENT_FORWARDED_CMAKE_VARIABLES into the generated proxy toolchain files:

set(HERMETIC_FETCHCONTENT_FORWARDED_CMAKE_VARIABLES
  "CMAKE_BUILD_TYPE"
  "CMAKE_EXE_LINKER_FLAGS"
  "CMAKE_CXX_FLAGS"
  "CMAKE_C_FLAGS"
)

The HERMETIC_CONFIG_EXTRA_ARGS option allows passing additional flags to the configure script for dependencies using autotools or openssl build systems. These flags are appended to the configure command after all standard flags (CFLAGS, CXXFLAGS, LDFLAGS, etc.).

This option only applies to HERMETIC_BUILD_SYSTEM autotools and HERMETIC_BUILD_SYSTEM openssl. For CMake-based dependencies, use HERMETIC_TOOLCHAIN_EXTENSION instead to control build configuration.

FetchContent_MakeHermetic(
  OpenSSL
  HERMETIC_BUILD_SYSTEM openssl
  HERMETIC_CONFIG_EXTRA_ARGS
    no-asm        # Disable assembly optimization
    no-shared     # Build only static libraries
)

For Autotools projects, you can pass parameters to the standard configure script using HERMETIC_CONFIG_EXTRA_ARGS :

FetchContent_MakeHermetic(
  libev
  HERMETIC_BUILD_SYSTEM autotools
  HERMETIC_CONFIG_EXTRA_ARGS
    --enable-maintainer-mode
)

The HERMETIC_CONFIG_LANGUAGE option selects which language's resolved view of the toolchain flags is forwarded to the configure script for dependencies using the autotools build system. Valid values are C (the default) and CXX.

Toolchain directives such as add_compile_options, add_compile_definitions, add_link_options, include_directories and link_directories may contain generator expressions like $<COMPILE_LANGUAGE:C> or $<LINK_LANGUAGE:CXX>. Hermetic FetchContent resolves those expressions separately for each language, then uses HERMETIC_CONFIG_LANGUAGE to decide which resolved view feeds CPPFLAGS, CFLAGS / CXXFLAGS and LDFLAGS on the configure command line. Use CXX when the imported target is primarily consumed from C++ code and you want the CXX branches of your language-conditional toolchain flags to apply.

This option only applies to HERMETIC_BUILD_SYSTEM autotools. OpenSSL is a pure C project and always uses the C-language resolved view. For CMake-based dependencies, per-language flags are already handled natively by CMake and no selection is needed.

FetchContent_MakeHermetic(
  Iconv
  HERMETIC_BUILD_SYSTEM autotools
  HERMETIC_CONFIG_LANGUAGE CXX
)

The HERMETIC_FIND_PACKAGES option enables specifying which packages that are provided by other Hermetic FetchContent declarations will be able to be find_package-ed by the content being specified. This enables controlling the packages available in the individual builds by other means than by just order of declaration.

In this example, the library libxml2 gets to find_package() Ìconv and ZLIB. These libraries have to be made available using Hermetic FetchContent in the same build. Such a call to find_package() will fail for other packages that are not explicitely whitelisted, either using the HERMETIC_FIND_PACKAGES option OR by adding said library to the global HERMETIC_FETCHCONTENT_BYPASS_PROVIDER_FOR_PACKAGES setting which allows find_package() to fall-back to CMake's native implementation if it cannot resolve the package among the whitelisted Hermetic FetchContent builds.

# omitting the FetchContent_Declare() call

FetchContent_MakeHermetic(
  LibXml2
  HERMETIC_BUILD_SYSTEM cmake
  HERMETIC_FIND_PACKAGES "Iconv;ZLIB" # <- this
  HERMETIC_TOOLCHAIN_EXTENSION [=[
    set(LIBXML2_WITH_LZMA OFF)
    set(LIBXML2_WITH_PYTHON OFF)
  ]=]
)

The HERMETIC_ADDITIONAL_TOOLCHAIN_FINGERPRINT_VARIABLES option extends the toolchain fingerprint computed for this specific dependency with additional cmake variables or environment variables beyond those captured by default. This is useful when a dependency's rebuild should be triggered by a project-specific variable that is not tracked by HFC by default (as are standard toolchain relevant things like compiler or linker flags), or by an environment variable (note: as for environment variable captures that CMake project usually do, those are captured at configure time).

Environment variables are specified with the ENV{VAR_NAME} syntax:

FetchContent_MakeHermetic(
  MyLib
  HERMETIC_BUILD_SYSTEM cmake
  HERMETIC_ADDITIONAL_TOOLCHAIN_FINGERPRINT_VARIABLES
    MY_CUSTOM_FLAG           # cmake variable
    ENV{CI_BUILD_NUMBER}     # environment variable
)

See also the global HERMETIC_FETCHCONTENT_ADDITIONAL_TOOLCHAIN_FINGERPRINT_VARIABLES variable which applies the same extension to all dependencies in the build.

The HERMETIC_CREATE_TARGET_ALIASES options allows defining aliases for target during the configure phase. The CMake code provided will be invoked for every CMake target library that is exported by the content. The scope in which this code will be run contains the variable TARGET_NAME which will be set to the exported target name. The provided code fragment must set the list TARGET_ALIASES to contain all the names under which the exported target will be imported when it is consumed.

It is recommended to test for specific values of TARGET_NAME instead of doing blanket declarations of aliases "no matter the input" to avoid issues in cases where a library might start to provide multiple exported libraries depending on the build configuration or just over time.

In the example below, we will be renaming the target ZLIB::zlib that is exported by the project to ZLIB::ZLIB (notice the difference in casing). This can be required if for example another project is unable to find the library in it's original casing.

FetchContent_Declare(
  ZLIB
  GIT_REPOSITORY https://github.com/cpp-pm/zlib.git
  GIT_TAG 57af136e436c5596e4f1c63fd5bdd2ce988777d1
)

FetchContent_MakeHermetic(
  ZLIB
  HERMETIC_BUILD_SYSTEM cmake
  HERMETIC_CREATE_TARGET_ALIASES [=[
    if("${TARGET_NAME}" STREQUAL "ZLIB::zlib")
      set(TARGET_ALIASES "ZLIB::ZLIB")
    endif()
  ]=]
)

HermeticFetchContent_MakeAvailableAtConfigureTime(ZLIB)

The option HERMETIC_PREPATCHED_RESOLVER enables defining a lookup method for source population. This is particularly useful when used in combination with the PATCH_COMMAND setting in FetchContent_Declare() because it allows the developper to store a pre-patched version of the source code of a library in a central location to avoid the burden of running potentially lengthy patch operations on every build while still retaining the capability of quickly trying out other versions of a dependency during upgrade trials.

As for the previous option Hermetic FetchContent expects HERMETIC_PREPATCHED_RESOLVER to contain an executable fragment of CMake code (as a string) that is called in a isolated scope that contains the following variables with the values defined in FetchContent_Declare:

GIT_REPOSITORY
URL of the git repository. Any URL understood by the git command may be used.

GIT_TAG
Git branch name, tag or commit hash to be checked out

URL
List of paths and/or URL(s) of the external project's source.

URL_HASH
Hash of the archive file to be downloaded. Should be of the form <algo>=<hashValue> where algo can be any of the hashing algorithms supported by the file() command.

The CMake code can redefine any of these variables to change the source of the content. The CMake code must set RESOLVED_PATCH to a truthy value if it is able to set an alternate source for the content.

If RESOLVED_PATCH is set to a truthy value the source populate operation will rely on the new information provided an skip the execution of the PATCH_COMMAND. Otherwise the normal expected behavior is executed (e.g. downloading from the original source and running any specified PATCH_COMMAND)

In the following example we will assume that the project is storing a patched copy of the specified repository and revision of the original sources in it own github organization under a given commit ID:

FetchContent_Declare(
  boost
  GIT_REPOSITORY https://github.com/boostorg/boost.git
  # that's v1.84
  GIT_TAG        ad09f667e61e18f5c31590941e748ac38e5a81bf
  # apply the patch "some.patch"
  PATCH_COMMAND cmake -E patch < ${CMAKE_CURRENT_LIST_DIR}/patchtest/some.patch
)

FetchContent_MakeHermetic(
  boost
  HERMETIC_BUILD_SYSTEM cmake
  HERMETIC_PREPATCHED_RESOLVER [=[
    if(${GIT_TAG} STREQUAL "ad09f667e61e18f5c31590941e748ac38e5a81bf")
      # hypothetical <my-org>/boost-prepatched.git repo
      set(GIT_REPOSITORY "https://github.com/<my-org>/boost-prepatched.git")
      set(GIT_TAG "9c83f4a27e4227dbe02e4a47ede372ac2a4a043e")
      set(RESOLVED_PATCH TRUE)
    endif()
  ]=]
)

HermeticFetchContent_MakeAvailableAtBuildTime(boost)

The developer can use the options HERMETIC_CMAKE_EXPORT_LIBRARY_DECLARATION to provide a a library export declaration that will enable Hermetic FetchContent to consume the targets as-if the project had provided a proper package configuration. This is useful in cases in which no such information is available (for example in the case of HERMETIC_BUILD_SYSTEM == autotools or HERMETIC_BUILD_SYSTEM == openssl or if the project's CMake buildsystem does not install and export targets)

The following example shows how to define the Pcap::Pcap target properties required for Hermetic FetchContent to be able to construct a so-called "targets cache":

FetchContent_MakeHermetic(
  Pcap
  HERMETIC_BUILD_SYSTEM autotools
  HERMETIC_CMAKE_EXPORT_LIBRARY_DECLARATION
    [=[
      add_library(Pcap::Pcap STATIC IMPORTED)
      set_property(
        TARGET Pcap::Pcap
        PROPERTY IMPORTED_LOCATION
        "@HFC_PREFIX_PLACEHOLDER@/lib/libpcap.a"
      )
      set_property(
        TARGET Pcap::Pcap
        PROPERTY INTERFACE_INCLUDE_DIRECTORIES
        "@HFC_PREFIX_PLACEHOLDER@/include"
      )
    ]=]
)

The following template variables are available to define the targets:

@HFC_SOURCE_DIR_PLACEHOLDER@ which will be replaced with the source directory location when the library is consumed later on.
@HFC_BINARY_DIR_PLACEHOLDER@ which will be replaced with the binaries directory location when the library is consumed later on.
@HFC_PREFIX_PLACEHOLDER@ which will be replaced with the final install prefix location when the library is consumed later on.

In cases in which the dependencie's CMake build system does provide target exports files but is not complying to the common file nameming scheme for those exports (hermetic fetchContent uses the following by default ([Tt]argets|[Ee]xport(s?))\.cmake), another pattern can be supplied using the HERMETIC_DISCOVER_TARGETS_FILE_PATTERN option.

HermeticFetchContent_MakeAvailableAtBuildTime¶

HermeticFetchContent_MakeAvailableAtBuildTime(
  <name>
)

The HermeticFetchContent_MakeAvailableAtBuildTime() function makes the content <name> available at build time (basically only running the content's configure step) an exporting the required targets.

After this call the content's targets are available for target linking and dependency management. Build graph matters (e.g. when and what to build) are left to the project's build system.

HermeticFetchContent_MakeAvailableAtConfigureTime¶

HermeticFetchContent_MakeAvailableAtConfigureTime(
  <name>
)

The HermeticFetchContent_MakeAvailableAtConfigureTime() function makes the content <name> available to the project an other Hermetic FetchContent contents by running the complete build when this statement gets executed.

This is particularly useful when integrating with other build systems that cannot be integrated in the project's build graph seamlessly and that - thus - require headers and libraries to be available in an installed location during their configure step.

HermeticFetchContent_SetBaseDir¶

HermeticFetchContent_SetBaseDir(
  <directory>
)

Set the base directory for all the hermetic dependency build directory and related folders

Build introspection ¶

HermeticFetchContent adds a number of cmake build targets to the build system that can be executed after the project configuration to enable some build introspection:

build target "hfc_list_dependencies_build_dirs"¶

Prints all project's dependencies build directories to the console. Warning! This list might be incomplete if the project build at hand did not actually build the whole list of dependencies

build target "hfc_list_dependencies_install_dirs"¶

Prints all project's dependencies install directories to the console. Warning! Depending on the build state, the listed install trees might not be populated (yet).

build target "hfc_echo_${content_name}_install_dir"¶

Prints the install directory for ${content_name} on the console. Warning! Depending on the build state, the install tree might not be populated (yet).

build target "hfc_list_${content_name}_STATIC_LIBRARIES_locations"¶

Prints the list of static libraries of ${content_name} on the console.

build target "hfc_list_${content_name}_SHARED_LIBRARIES_locations"¶

Prints the list of shared libraries of ${content_name} on the console.

build target "hfc_list_${content_name}_MODULE_LIBRARIES_locations"¶

Prints the list of module libraries of ${content_name} on the console.

build target "hfc_list_${content_name}_UNKNOWN_LIBRARIES_locations"¶

Prints the list of 'unknown type' libraries of ${content_name} on the console.

build target "hfc_list_${content_name}_EXECUTABLES_locations"¶

Prints the list of executables of ${content_name} on the console.

Toolchain fingerprinting ¶

HermeticFetchContent computes a fingerprint of the active toolchain for each dependency. The fingerprint is embedded in the proxy toolchain file that governs the dependency's isolated build; if the fingerprint changes the proxy toolchain hash changes, which causes HFC to invalidate the dependency and re-configure / re-build it.

What is captured

The fingerprint is assembled from:

All CMAKE_* cache variables whose names match patterns known to influence ABI or compilation (compiler flags, compiler identity, build type, sysroot, OS and CPU settings, etc.). Variables like CMAKE_MAKE_PROGRAM or install-prefix variables that do not affect the compiled output are explicitly excluded.
Top-level directory properties: COMPILE_DEFINITIONS, COMPILE_OPTIONS, LINK_OPTIONS, INCLUDE_DIRECTORIES, and LINK_DIRECTORIES as set at the root CMakeLists.txt level (e.g. via add_compile_options() or add_compile_definitions()).
Generator expressions found in any of the above are evaluated in an isolated cmake mini-project so that conditional expressions such as $<$<CONFIG:Release>:-O3> or $<$<COMPILE_LANGUAGE:CXX>:-std=c++17> are resolved to their concrete values before hashing. This prevents genex tokens from leaking into the hash verbatim and causing false cache misses or cache hits.
Any extra variables listed in HERMETIC_ADDITIONAL_TOOLCHAIN_FINGERPRINT_VARIABLES (per-dependency) or HERMETIC_FETCHCONTENT_ADDITIONAL_TOOLCHAIN_FINGERPRINT_VARIABLES (global).

Controlling the fingerprinting mini-project

The isolated cmake process that evaluates generator expressions enables compiler-detection variables (CMAKE_<LANG>_*) that require an enabled language. The HFC_TOOLCHAIN_FINGERPRINT_LANGUAGES cache variable controls which languages the mini-project enables:

root_project (default): Mirror the languages enabled in the calling project() call.
none: Use LANGUAGES NONE — fast but compiler-derived variables are not captured.
explicit list: A semicolon-separated list such as C or C;CXX.

# Force fingerprinting to use C and C++ regardless of the root project's languages
set(HFC_TOOLCHAIN_FINGERPRINT_LANGUAGES "C;CXX" CACHE STRING "" FORCE)

Setting HERMETIC_FETCHCONTENT_TOOLCHAIN_FINGERPRINT_DISABLE_CAPTURE_TOP_LEVEL_DIR_PROPERTIES to ON disables the capture of top-level directory properties, which can be useful in projects where those properties contain rapidly-changing values that would cause unnecessary rebuilds.

Rationale ¶

Ressembling add_subdirectory through external cmake execution is key to providing reusable install trees :

One project depending on a library can reuse the library built by another project
The only common element has to be the common CMAKE_TOOLCHAIN_FILE settings

Finally it allows a build graph optimization to be at maximum parallelity. Indeed usual CMake practices for externally built libraries found via find_package allows reusing the same libraries across project, but requires building them in advance before the configure of the dependent project might even be started. However not all target depend directly of the said dependencies.

With this techniques it can integrate the clean build of the dependencies in the graph, effectively building at the finest granularity of dependencies everything that isn't dependent of the dependencies. Possibly allowing building earlier parts of the project that do not depend on the dependency itself.

Plain add_subdirectory allows this but lacks the isolation from the parent CMake project, hindering the reuse of the dependent library install tree without the presence of the sources, which is a big speedup in CI workflows, and new developer computer setup, optimizing avoid the need of cloning the actual dependencies source files with all the git history and every private implementation files.

FetchContent_MakeHermetic ¶

Overview ¶

Commands ¶

Build introspection ¶

Toolchain fingerprinting ¶

Rationale ¶

Table of Contents

Previous topic

Next topic

This Page

FetchContent_MakeHermetic¶

Overview¶

Commands¶

Build introspection¶

Toolchain fingerprinting¶

Rationale¶

FetchContent_MakeHermetic ¶

Overview ¶

Commands ¶

Build introspection ¶

Toolchain fingerprinting ¶

Rationale ¶