2.1   TriBITS Developer and User Roles

There are approximately five different types roles related to TriBITS. These different roles require different levels of expertise and knowledge of CMake and knowledge of the TriBITS system. The primary roles are 1) TriBITS Project User, 2) TriBITS Project Developer, 3) TriBITS Project Architect, 4) TriBITS System Developer, and 5) TriBITS System Architect. Each of these roles builds on the necessary knowledge of the lower-level roles.

The first role is that of a TriBITS Project User who only needs to be able to configure, build, and test a project that uses TriBITS as its build system. A person acting in this role needs to know little about CMake other than basics about how to run the cmake and ctest executables, how to set CMake cache variables, and the basics of building software by typing make and running tests with ctest. The proper reference for a TriBITS Project User is the Project-Specific Build Reference. The TriBITS Overview document may also be of some help. A TriBITS project user may also need to consult Package Dependencies and Enable/Disable Logic.

A TriBITS Project Developer is someone who contributes to a software project that uses TriBITS. They will add source files, libraries and executables , test executables and define tests run with ctest. They have to configure and build the project code in order to be able to develop and run tests and therefore this role includes all of the necessary knowledge and functions of a TriBITS Project User. A casual TriBITS Project Developer typically does not need to know a lot about CMake and really only needs to know a subset of the TriBITS Macros and Functions defined in the document TriBITS Users Guide and Reference in addition to the genetic TriBITS Build Reference document. A slightly more sophisticated TriBITS Project Developer will also add new packages, add new package dependencies, and define new external packages/TPLs. The TriBITS Users Guide and Reference should supply everything such a developer needs to know and more. However, only a smaller part of that document needs to be understood and accessed by people assuming this role.

The next level of roles is a TriBITS Project Architect. This is someone (perhaps only one person on a project development team) that knows the usage and functioning of TriBITS in great detail. They understand how to set up a TriBITS project from scratch, how to set up automated testing using the TriBITS system, and know how to use TriBITS to implement the overall software development process. A person in this role is also likely to be the one who makes the initial technical decision for their project to adopt TriBITS for its native build and test system. The document TriBITS Users Guide and Reference, detailed CMake/CTest/CDash documentation provided by Kitware, and great books like Professional CMake should provide most of what a person in this role needs to know. A person assuming this role is the primary audience for a lot of the more advanced material in that document.

The last two roles TriBITS System Developer and TriBITS System Architect are for those individuals that actually extend and modify the TriBITS system itself. A TriBITS System Developer needs to know how to add new TriBITS functionality while maintaining backward compatibility, know how to add new unit tests for the TriBITS system (see The TriBITS Test Package), and perform other related tasks. Such a developer needs to be very knowledgeable of the basic functioning of CMake and know how TriBITS is implemented in the CMake language. A TriBITS System Architect is someone who must be consulted on almost all non-trivial changes or additions to the TriBITS system. A TriBITS System Architect in addition needs to know the entire TriBITS system, the design philosophy that provides the foundation for TriBITS and be an expert in CMake, CTest, and CDash. Much of what needs to be known by a TriBITS System Developer and a TriBITS System Architect is contained in the document TriBITS Maintainers Guide and Reference. The rest of the primary documentation for these roles will be in the TriBITS CMake source code and various unit tests itself defined in The TriBITS Test Package. At the time of this writing, there is currently there is only one TriBITS System Architect (who also happens to be the primary author of this document).

An explicit goal of the document TriBITS Users Guide and Reference is to foster the creation of new TriBITS Project Architects (i.e. those who would make the decision to adopt TriBITS for their projects). An explicit goal of the document TriBITS Maintainers Guide and Reference is to foster the creation of new TriBITS System Developers to help extend and maintain the TriBITS package itself.

Depending on the particular role that a reader falls into, this document may not be necessary and the TriBITS Overview or the <Project>BuildReference documents may be more appropriate. Hopefully the above roles and discussion help the reader select the right document to start with.

NOTE: Before getting started with TriBITS, if a reader is unfamiliar with CMake, please review the CMake Language Overview and Gotchas. Once those CMake basics and common gotchas have been reviewed, we now get into the meat of TriBITS starting with software engineering principles that lie at the foundation of TriBITS. That is followed by with and overall of the structure of a TriBITS project.

2.2   Software Engineering Packaging Principles

The design of TriBITS takes into account standard software engineering packaging principles. In his book [Agile Software Development, 2003], Robert Martin defines several software engineering principles related to packaging software which are listed below:

• Package Cohesion OO Principles:
  1. REP (Release-Reuse Equivalency Principle): The granule of reuse is the granule of release.
  2. CCP (Common Closure Principle): The classes in a package should be closed together against the same kinds of changes. A change that affects a closed package affects all the classes in that package and no other packages.
  3. CRP (Common Reuse Principle): The classes in a package are used together. If you reuse one of the classes in a package, you reuse them all.
• Package Coupling OO Principles:
  4. ADP (Acyclic Dependencies Principle): Allow no cycles in the package dependency graph.
  5. SDP (Stable Dependencies Principle): Depend in the direction of stability.
  6. SAP (Stable Abstractions Principle): A package should be as abstract as it is stable.

Any of these six OO packaging principles (and other issues) may be considered when deciding how to partition software into different TriBITS Packages.

NOTE: The purpose of this TriBITS Developers Guide document is not teach basic software engineering so these various principles will not be expanded on further. However, interested readers are strongly encouraged to read [Agile Software Development, 2003] as one of the better software engineering books out there (see https://bartlettroscoe.github.io/reading-list/#most_recommended_se_books).

3.1   TriBITS Structural Units

A CMake project that uses TriBITS as its build and test system is composed of a single TriBITS Project, one or more TriBITS Repositories and one or more TriBITS Packages. In addition, a TriBITS Package can be broken up into TriBITS Subpackages. Together, the collection of TriBITS Packages and TriBITS Subpackages are called TriBITS Software Engineering Packages, or TriBITS Packages for short.

First, to better establish the basic nomenclature, the key structural TriBITS units are:

• TriBITS Top-Level Package: A collection of related software that typically includes one or more source files built into one or more libraries and has associated tests to help define and protect the functionality provided by the software. A top-level package (or just "package" if no ambiguity) also typically defines a single integrated unit of documentation and testing (see TriBITS Automated Testing). A top-level TriBITS package may or may not be broken down into multiple subpackages. Examples of TriBITS packages in TribitsExampleProject include SimpleCXX, MixedLang and WithSubpackages. (Don't confuse a TriBITS "Package" with a raw CMake "Package". A raw CMake "Package" actually maps to a TriBITS External Package/TPL; see Why a TriBITS Package is not a CMake Package.)

• TriBITS Subpackage: A part of a parent TriBITS Package that also typically has source files built into libraries and tests but is documented and tested along with the other subpackages in the parent package. The primary purpose for supporting subpackages is to provide finer-grained control of software dependencies. In TribitsExampleProject, WithSubpackages is an example of a package with subpackages 'A', 'B', and 'C'. The full subpackage name is prefixed by the parent package name (e.g. the full name for subpackage 'A' is WithSubpackagesA). The parent package is always implicitly dependent on its subpackages (e.g. parent package WithSubpackages depends on its subpackages WithSubpackagesA, WithSubpackagesB, and WithSubpackagesC).

• TriBITS External Package/TPL: The specification for a particular external dependency that is required or can be used in one or more TriBITS Packages. A modern TriBITS external package/TPL (Third Party Library) is typically just a small file FindTPL<tplName>.cmake that calls find_package(<externalPkg>) and defines the <tplName>::all_libs target. More generally, an external package/TPL can be specified as a list list of libraries and/or include directories for header files. Examples of basic external packages/TPLs include BLAS, LAPACK, and Boost.

• TriBITS Package: The combined set of TriBITS Top-Level Packages, TriBITS Subpackages, and TriBITS External Packages/TPLs that constitute the basic Software Engineering packages of a TriBITS project (see Software Engineering Packaging Principles): Packages are the basis for setting dependencies in the TriBITS system. For example, the Packages provided by the example top-level package WithSubpackages (in order of increasing dependencies) are WithSubpackagesA, WithSubpackagesB, WithSubpackagesC, and WithSubpackages (see TribitsExampleProject).

• TriBITS Repository: A collection of one or more TriBITS Packages specified in a <repoDir>/PackagesList.cmake file and zero or more external package/TPL declarations specified in a <repoDir>/TPLsList.cmake file. As discussed below, Repositories can include Native Repositories and Version Control (VC) Repositories.

• TriBITS Project: A collection of TriBITS Repositories and TriBITS Packages that defines a complete CMake PROJECT defining software which can be directly configured with cmake and then be built, tested, installed, etc.

• TriBITS Meta-Project: A TriBITS Project that contains no native TriBITS packages or TriBITS external packages/TPLs but is composed out packages from other TriBITS Repositories.

In this document, dependencies are described as either being upstream or downstream/forward defined as:

• If unit "B" requires (or can use, or comes before) unit "A", then "A" is an upstream dependency of "B".

• If unit "B" requires (or can use, or comes before) unit "A", then "B" is a downstream dependency or a forward dependency of "A".

The following subsections define the major structural units of a TriBITS project in more detail. Each structural unit is described along with the files and directories associated with each. In addition, a key set of TriBITS CMake variables for each are defined as well.

In the next major section following this one, some Example TriBITS Projects are described. For those who just want to jump in and learn best by example, these example projects are a good way to start. These example projects will be referenced in the more detailed descriptions given in this document.

The last issue to touch on before getting into the detailed descriptions of the different TriBITS structural units is the issue of how CMake variables are defined and used by TriBITS. The CMake variables described in the TriBITS structural units below fall into one of two major types:

• Local Fixed-Name Variables are used temporarily in the processing of a TriBITS unit. These include variables such as PROJECT_NAME, REPOSITORY_NAME, PACKAGE_NAME, and PARENT_PACKAGE_NAME. These are distinguished by having a non-namespaced fixed/constant name. They are typically part of TriBITS reflection system, allowing subordinate units to determine the encapsulating unit in which they are participating. For, example, a TriBITS subpackage can determine its name, its parent package's name and directories, its parent repository name and directories, and the enclosing project's name and directories without having to refer to any specific names.
• Globally Scoped Namespaced Variables are used to refer to properties of a named TriBITS unit that are seen globally by the entire TriBITS project. These include variables such as ${REPOSITORY_NAME}_SOURCE_DIR (e.g. TribitsExProj_SOURCE_DIR) and ${PACKAGE_NAME}_BINARY_DIR (e.g. SimpleCXX_BINARY_DIR). They are available after processing the unit, for use by downstream or subordinate units. They are part of the TriBITS dependency system, allowing downstream units to access properties of their known upstream dependencies.

More information about these various files is described in section Processing of TriBITS Files: Ordering and Details.

<projectDir>/
   ProjectName.cmake    # Defines PACKAGE_NAME
   CMakeLists.txt       # Base project CMakeLists.txt file
   CTestConfig.cmake    # [Optional] Needed for CDash submits
   Version.cmake        # [Optional] Dev mode, Project version, VC branch
   project-checkin-test-config.py # [Optional] checkin-test.py config
   cmake/
     NativeRepositoriesList.cmake    # [Optional] Rarely used
     ExtraRepositoriesList.cmake     # [Optional] Lists repos and VC URLs
     ProjectCiFileChangeLogic.py     # [Optional] CI global change/test logic
     ProjectCompilerPostConfig.cmake # [Optional] Override/tweak build flags
     ProjectDependenciesSetup.cmake  # [Optional] Project deps overrides
     CallbackDefineProjectPackaging.cmake  # [Optional] CPack settings
     tribits/    # [Optional] Or provide ${PROJECT_NAME}_TRIBITS_DIR
     ctest/
       CTestCustom.cmake.in  # [Optional] Custom ctest settings

# Must set the project name at very beginning before including anything else
set(PROJECT_NAME TribitsExProj)

# Turn on export dependency generation for WrapExteranl package
set(${PROJECT_NAME}_GENERATE_EXPORT_FILE_DEPENDENCIES_DEFAULT ON)

# Turn on by default the generation of the export files
set(${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES_DEFAULT ON)

################################################################################
#                                                                              #
#                        TribitsExampleProject                                 #
#                                                                              #
################################################################################

# To be safe, define your minimum CMake version.  This may be newer than the
# min required by TriBITS.
cmake_minimum_required(VERSION 3.23.0 FATAL_ERROR)

# Make CMake set WIN32 with CYGWIN for older CMake versions.  CMake requires
# this to be in the top-level CMakeLists.txt file and not an include file :-(
set(CMAKE_LEGACY_CYGWIN_WIN32 1 CACHE BOOL "" FORCE)

#
# A) Define your project name and set up major project options
#
# NOTE: Don't set options that would impact what packages get defined or
# enabled/disabled in this file as that would not impact other tools that
# don't process this file.
#

# Get PROJECT_NAME (must be in a file for other parts of system to read)
include("${CMAKE_CURRENT_SOURCE_DIR}/ProjectName.cmake")

# CMake requires that you declare the CMake project in the top-level file and
# not in an include file :-(
project(${PROJECT_NAME} NONE)

set(TRIBITS_HIDE_DEPRECATED_INCLUDE_DIRECTORIES_OVERRIDE  TRUE)

#
# B) Pull in the TriBITS system and execute
#

set(${PROJECT_NAME}_TRIBITS_DIR
   "${CMAKE_CURRENT_LIST_DIR}/../.."  CACHE  STRING
  "TriBITS base directory (default assumes in TriBITS source tree)")
include("${${PROJECT_NAME}_TRIBITS_DIR}/TriBITS.cmake")

# Set default location for header-only TPL to make easy to configure out of
# the TriBITS source tree.
set(HeaderOnlyTpl_INCLUDE_DIRS
  "${${PROJECT_NAME}_TRIBITS_DIR}/examples/tpls/HeaderOnlyTpl" 
  CACHE PATH "Default set by TriBITS/CMakeLists.txt" )

# Do all of the processing for this Tribits project
tribits_project()

include("${${PROJECT_NAME}_TRIBITS_DIR}/TriBITS.cmake")

include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/tribits/TriBITS.cmake)

set(VERA_TRIBITS_CMAKE_MINIMUM_REQUIRED 3.23.0)
cmake_minimum_required(VERSION ${VERA_TRIBITS_CMAKE_MINIMUM_REQUIRED}
  FATAL_ERROR)

include(SetDefaultAndFromEnv)

set(CTEST_NIGHTLY_START_TIME "04:00:00 UTC") # 10 PM MDT or 9 PM MST

if (NOT DEFINED CTEST_DROP_METHOD)
  set_default_and_from_env(CTEST_DROP_METHOD "https")
endif()

if (CTEST_DROP_METHOD STREQUAL "http" OR CTEST_DROP_METHOD STREQUAL "https")
  set_default_and_from_env(CTEST_DROP_SITE "my.cdash.org")
  set_default_and_from_env(CTEST_PROJECT_NAME "TribitsExampleProject")
  set_default_and_from_env(CTEST_DROP_LOCATION "/submit.php?project=TribitsExampleProject")
  set_default_and_from_env(CTEST_TRIGGER_SITE "")
  set_default_and_from_env(CTEST_DROP_SITE_CDASH TRUE)
endif()

set_default_and_from_env(TRIBITS_2ND_CTEST_DROP_SITE ...)
set_default_and_from_env(TRIBITS_2ND_CTEST_DROP_LOCATION ...)

set(${REPOSITORY_NAME}_VERSION 1.1)
set(${REPOSITORY_NAME}_MAJOR_VERSION 01)
set(${REPOSITORY_NAME}_MAJOR_MINOR_VERSION 010100)
set(${REPOSITORY_NAME}_VERSION_STRING "1.1 (Dev)")
set(${REPOSITORY_NAME}_ENABLE_DEVELOPMENT_MODE_DEFAULT ON) # Change to 'OFF' for a release

lib<libname>.so.01
lib<libname>.so.1.1

#
# Define project-specific options for the checkin-test script for
# TribitsExampleProject.
#

configuration = {

    # Default command line arguments
    'defaults': {
        '--send-email-to-on-push': 'trilinos-checkin-tests@software.sandia.gov',
        },

    # CMake options (-DVAR:TYPE=VAL) cache variables.
    'cmake': {
        
        # Options that are common to all builds.
        'common': [],

        # Defines --default-builds, in order.
        'default-builds': [
            # Options for the MPI_DEBUG build.
            ('MPI_DEBUG', [
                '-DTPL_ENABLE_MPI:BOOL=ON',
                '-DCMAKE_BUILD_TYPE:STRING=RELEASE',
                '-DTribitsExProj_ENABLE_DEBUG:BOOL=ON',
                '-DTribitsExProj_ENABLE_CHECKED_STL:BOOL=ON',
                '-DTribitsExProj_ENABLE_DEBUG_SYMBOLS:BOOL=ON',
                ]),
            # Options for the SERIAL_RELEASE build.
            ('SERIAL_RELEASE', [
                '-DTPL_ENABLE_MPI:BOOL=OFF',
                '-DCMAKE_BUILD_TYPE:STRING=RELEASE',
                '-DTribitsExProj_ENABLE_DEBUG:BOOL=OFF',
                '-DTribitsExProj_ENABLE_CHECKED_STL:BOOL=OFF',
                ]),
            ], # default-builds

        }, # cmake

    } # configuration

set(${PROJECT_NAME}_NATIVE_REPOSITORIES Repo0 Repo1)

tribits_project_define_extra_repositories(
  ExtraRepo1  ""  GIT  someurl.com:/ExtraRepo1   ""          Continuous
  ExtraRepo2  packages/SomePackage/Blah  GIT  someurl2.com:/ExtraRepo2
                                                 NOPACKAGES  Nightly
  ExtraRepo3  ""  HG  someurl3.com:/ExtraRepo3   ""          Continuous
  ExtraRepo4  ""  SVN  someurl4.com:/ExtraRepo4  ""          Nightly
  )

<projectDir>/packages/SomePackage/Blah

#
# Specialized logic for what file changes should trigger a global build in CI
# testing where testing should only occur package impacted by the change.
#

class ProjectCiFileChangeLogic:

  def isGlobalBuildFileRequiringGlobalRebuild(self, modifiedFileFullPath):
    modifiedFileFullPathArray = modifiedFileFullPath.split('/')
    lenPathArray = len(modifiedFileFullPathArray)
    if lenPathArray==1:
      # Files sitting directly under <projectDir>/
      if modifiedFileFullPathArray[0] == "CMakeLists.txt":
        return True
      if modifiedFileFullPathArray[0].rfind(".cmake") != -1:
        return True
    elif modifiedFileFullPathArray[0] == 'cmake':
      # Files under <projectDir>/cmake/
      if modifiedFileFullPathArray[1]=='ExtraRepositoriesList.cmake':
        return False
      elif modifiedFileFullPathArray[1] == 'ctest' and lenPathArray >= 3:
        if lenPathArray > 3:
          # This is a file
          # <projectDir>/cmake/ctest/<something>/[...something...]  so this is
          # for a specific machine and should not trigger a global build.
          return False
        else:
          # Any other file directly under cmake/ctest/ should trigger a global
          # build.
          return True
      else:
        # All other files under cmake/
        if modifiedFileFullPath.rfind(".cmake") != -1:
          # All other *.cmake files under cmake/ trigger a global build.
          return True
    # Any other files should not trigger a global build
    return False

if (${Trilinos_ENABLE_Kokkos})

  ...

  include(${Kokkos_GEN_DIR}/kokkos_generated_settings.cmake)

  if (NOT KOKKOS_ARCH STREQUAL "None")

    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${KOKKOS_CXX_FLAGS}")

    message("-- " "Skip adding flags for OpenMP because Kokkos flags does that ...")
    set(OpenMP_CXX_FLAGS_OVERRIDE " ")

  endif()

endif()

set_default(${PROJECT_NAME}_PROJECT_MASTER_EMAIL_ADDRESS
    projectx-regressions@somemailserver.org)

macro(TRIBITS_PROJECT_DEFINE_PACKAGING)

    tribits_copy_installer_resource(TribitsExProj_README
      "${TribitsExProj_SOURCE_DIR}/README.md"
      "${TribitsExProj_BINARY_DIR}/README.md")
    tribits_copy_installer_resource(TribitsExProj_LICENSE
      "${TribitsExProj_SOURCE_DIR}/LICENSE"
      "${TribitsExProj_BINARY_DIR}/LICENSE.txt")

    set(CPACK_PACKAGE_DESCRIPTION "TribitsExampleProject just shows you how to use TriBITS.")
    set(CPACK_PACKAGE_FILE_NAME "tribitsexproj-setup-${TribitsExProj_VERSION}")
    set(CPACK_PACKAGE_INSTALL_DIRECTORY "TribitsExProj ${TribitsExProj_VERSION}")
    set(CPACK_PACKAGE_REGISTRY_KEY "TribitsExProj ${TribitsExProj_VERSION}")
    set(CPACK_PACKAGE_NAME "tribitsexproj")
    set(CPACK_PACKAGE_VENDOR "Sandia National Laboratories")
    set(CPACK_PACKAGE_VERSION "${TribitsExProj_VERSION}")
    set(CPACK_RESOURCE_FILE_README "${TribitsExProj_README}")
    set(CPACK_RESOURCE_FILE_LICENSE "${TribitsExProj_LICENSE}")
    set(${PROJECT_NAME}_CPACK_SOURCE_GENERATOR_DEFAULT "TGZ;TBZ2")
    set(CPACK_SOURCE_FILE_NAME "tribitsexproj-source-${TribitsExProj_VERSION}")
    set(CPACK_COMPONENTS_ALL ${TribitsExProj_PACKAGES} Unspecified)

endmacro()

# Increase the amount of output being produced
set(CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE 54321)
set(CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE 123456)
# NOTE: Above, we use these numbers to make it easy to determine that these
# values are getting read correctly in various modes of running
# tribits_ctest_driver().  In a real project, you could set something like:
#
#   set(CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE 50000)
#   set(CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE 5000000)

set(CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE 50000)
set(CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE 5000000)

<repoDir>/
  PackagesList.cmake
  TPLsList.cmake
  Copyright.txt  # [Optional] Only needed if creating version header file
  Version.cmake  # [Optional] Info inserted into ${REPO_NAME}_version.h
  cmake/
     RepositoryDependenciesSetup.cmake # [Optional] CDash email addresses?
     CallbackSetupExtraOptions.cmake # [Optional] Called after main options
     CallbackDefineRepositoryPackaging.cmake # [Optional] CPack packaging

tribits_repository_define_packages(
  SimpleCxx          packages/simple_cxx         PT
  MixedLang          packages/mixed_lang         PT
  InsertedPkg        InsertedPkg                 ST
  WithSubpackages    packages/with_subpackages   PT
  WrapExternal       packages/wrap_external      ST
  )

tribits_disable_package_on_platforms(WrapExternal Windows)
tribits_allow_missing_external_packages(InsertedPkg)

tribits_repository_define_packages()

tribits_repository_define_tpls(
  MPI            "${${PROJECT_NAME}_TRIBITS_DIR}/core/std_tpls/"  PT
  BLAS           "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/"    PT
  LAPACK         "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/"    PT
  Boost          cmake/TPLs/    ST
  UMFPACK        cmake/TPLs/    ST
  AMD            cmake/TPLs/    EX
  PETSC          "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/"    ST
  )

tribits_repository_define_tpls()

            TriBITS: Tribal Build, Integrate, and Test System
                        Copyright (c) 2013 NTESS

Copyright 2013 National Technology & Engineering Solutions of Sandia,
LLC (NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the
U.S. Government retains certain rights in this software.

Copyright the TriBITS contributors.

set(${REPOSITORY_NAME}_VERSION 1.1)
set(${REPOSITORY_NAME}_MAJOR_VERSION 01)
set(${REPOSITORY_NAME}_MAJOR_MINOR_VERSION 010100)
set(${REPOSITORY_NAME}_VERSION_STRING "1.1 (Dev)")
set(${REPOSITORY_NAME}_ENABLE_DEVELOPMENT_MODE_DEFAULT ON) # Change to 'OFF' for a release

set_default(${REPOSITORY_NAME}_REPOSITORY_MASTER_EMAIL_ADDRESS
    repox-regressions@somemailserver.org)

macro(TRIBITS_REPOSITORY_SETUP_EXTRA_OPTIONS)

  assert_defined(${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES)

  if (${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES)
    message(
      "\n***"
      "\n*** NOTE: Setting ${PROJECT_NAME}_ENABLE_WrapExternal=OFF"
      " because ${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES='${${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES}'!"
      "\n***\n"
      )
    set(${PROJECT_NAME}_ENABLE_WrapExternal OFF)
  endif()

  if ("${Python3_EXECUTABLE}" STREQUAL "")
    message(
      "\n***"
      "\n*** NOTE: Setting ${PROJECT_NAME}_ENABLE_WrapExternal=OFF"
      " because Python3_EXECUTABLE=''!"
      "\n***\n"
      )
    set(${PROJECT_NAME}_ENABLE_WrapExternal OFF)
  endif()

  assert_defined(${PROJECT_NAME}_ENABLE_Fortran)

  if (NOT ${PROJECT_NAME}_ENABLE_Fortran)
    message(
      "\n***"
      "\n*** NOTE: Setting ${PROJECT_NAME}_ENABLE_MixedLang=OFF"
      " because ${PROJECT_NAME}_ENABLE_Fortran='${${PROJECT_NAME}_ENABLE_Fortran}'!"
      "\n***\n"
      )
    set(${PROJECT_NAME}_ENABLE_MixedLang OFF)
  endif()

endmacro()

macro(TRIBITS_REPOSITORY_DEFINE_PACKAGING)

  assert_defined(${REPOSITORY_NAME}_SOURCE_DIR)
  append_set(CPACK_SOURCE_IGNORE_FILES
    "${${REPOSITORY_NAME}_SOURCE_DIR}/cmake/ctest/"
    )

endmacro()

<packageDir>/
  CMakeLists.txt  # Only processed if the package is enabled
  cmake/
    Dependencies.cmake  # Always processed if its repo is processed
    <packageName>_config.h.in  # [Optional], name is not fixed

tribits_package_define_dependencies(
  LIB_REQUIRED_PACKAGES Teuchos Sacado Phalanx Intrepid Thyra
    Tpetra Epetra EpetraExt
  LIB_OPTIONAL_PACKAGES Stokhos
  TEST_OPTIONAL_PACKAGES Stratimikos
  LIB_REQUIRED_TPLS MPI Boost
  )

tribits_package_define_dependencies(
  SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS
    A   a   PT  REQUIRED
    B   b   ST  OPTIONAL
    C   c   ST  OPTIONAL
  REGRESSION_EMAIL_LIST with_packages-regressions@someurl.none
  )

tribits_package_define_dependencies()

#ifndef SIMPLECXX_CONFIG_H
#define SIMPLECXX_CONFIG_H

#cmakedefine HAVE_SIMPLECXX___INT64

#cmakedefine HAVE_SIMPLECXX_DEBUG
#cmakedefine HAVE_SIMPLECXX_SIMPLETPL

@SIMPLECXX_DEPRECATED_DECLARATIONS@

#endif /** SIMPLECXX_CONFIG_H **/

#
# A) Define the package
#
tribits_package( SimpleCxx  ENABLE_SHADOWING_WARNINGS  CLEANED )

#
# B) Platform-specific checks
#
include(CheckFor__int64)
check_for___int64(HAVE_SIMPLECXX___INT64)
tribits_pkg_export_cache_var(HAVE_SIMPLECXX___INT64)

#
# C) Set up package-specific options
#
tribits_add_debug_option()
tribits_add_show_deprecated_warnings_option()

#
# D) Add the libraries, tests, and examples
#
add_subdirectory(src)
tribits_add_test_directories(test)

# Set a variable that will be used in downstream packages

if (SimpleCxx_ENABLE_SimpleTpl)
  set(simpletplText "simpletpl ")
else()
  set(simpletplText)
endif()

global_set(EXPECTED_SIMPLECXX_AND_DEPS
  "SimpleCxx ${simpletplText}headeronlytpl")
tribits_pkg_export_cache_var(EXPECTED_SIMPLECXX_AND_DEPS)

#
# E) Do standard post processing
#
tribits_package_postprocess()

tribits_package_decl(WithSubpackages)

tribits_add_debug_option()

tribits_process_subpackages()

tribits_exclude_files(
  b/ExcludeFromRelease.txt
  b/src/AlsoExcludeFromTarball.txt
  )

tribits_package_def()

tribits_package_postprocess()

<packageDir>/<spkgDir>/
  CMakeLists.txt  # Only processed if this subpackage is enabled
  cmake/
    Dependencies.cmake  # Always processed if the parent package
                          # is listed in the enclosing Repository

tribits_package_define_dependencies(
  LIB_REQUIRED_PACKAGES  SimpleCxx
  LIB_OPTIONAL_PACKAGES  WithSubpackagesA  InsertedPkg 
  TEST_OPTIONAL_PACKAGES  MixedLang
  )

'WithSubpackagesA' = 'WithSubpackages' + 'A'

tribits_package_define_dependencies(
  LIB_REQUIRED_PACKAGES SimpleCxx
  )

tribits_package_define_dependencies(
  LIB_REQUIRED_TPLS Boost
  SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS
    A   A   PT  REQUIRED
    ...
  )

#
# A) Define the subpackage
#
tribits_subpackage(A)

#
# B) Set up subpackage-specific options
#

set(${PACKAGE_NAME}_SPECIAL_VALUE  3  CACHE STRING "Integer special value")
tribits_pkg_export_cache_var(${PACKAGE_NAME}_SPECIAL_VALUE)

#
# C) Add the libraries, tests, and examples
#

tribits_configure_file(${PACKAGE_NAME}_config.h)

tribits_include_directories(${CMAKE_CURRENT_BINARY_DIR})

tribits_include_directories(${CMAKE_CURRENT_SOURCE_DIR})
tribits_add_library(pws_a
  SOURCES A.cpp
  HEADERS A.hpp ${CMAKE_CURRENT_BINARY_DIR}/${PACKAGE_NAME}_config.h
)

tribits_add_test_directories(tests)

#
# D) Do standard post processing
#
tribits_subpackage_postprocess()

<tplDefsDir>/
  FindTPL<tplName>.cmake   # The name is not fixed (see <tplName>_FINDMOD)
  FindTPL<tplName>Dependencies.cmake   # [Optional], defines upstream dependencies

find_package(<externalPkg> REQUIRED)
tribits_extpkg_create_imported_all_libs_target_and_config_file(
  <tplName>
  INNER_FIND_PACKAGE_NAME <externalPkg>
  IMPORTED_TARGETS_FOR_ALL_LIBS <importedTarget0> <importedTarget1> ... )

tribits_tpl_find_include_dirs_and_libraries( <tplName>
  REQUIRED_HEADERS <header0> <header1> ...
  REQUIRED_LIBS_NAMES <libname0> <libname1> ...
  MUST_FIND_ALL_LIBS
  )

tribits_tpl_find_include_dirs_and_libraries( PETSC
  REQUIRED_HEADERS petsc.h
  REQUIRED_LIBS_NAMES petsc
  )

tribits_extpkg_define_dependencies( <tplName>
  DEPENDENCIES <upstreamTpl_0> <upstreamTpl_1> ... )

3.2   Processing of TriBITS Files: Ordering and Details

One of the most important things to know about TriBITS is what files it processes, in what order, and in what context. This is critical to being able to understand what impact (if any) setting a variable or otherwise changing the CMake run-time state will have on configuring a CMake project which uses TriBITS. While the different files that make up a TriBITS Project, TriBITS Repository, TriBITS Package, TriBITS Subpackage, and TriBITS TPL were defined in the section TriBITS Structural Units, that material did not fully describe the context and in what order these files are processed by the TriBITS framework.

The TriBITS system processes the project's files in one of two general use cases. The first use case is in the basic configuration of the project with a standard cmake command invocation in order to set up the build files in the binary directory (see Full TriBITS Project Configuration). The second use case is in reading the project's dependency-related files in order to build the package dependency data-structure (e.g. the <Project>PackageDependencies.xml file, see Reduced Package Dependency Processing). The second use case of reading the project's dependency files is largely a subset of the first.

Another factor that is important to understand is the scoping in which the various files are processed (with include() or add_subdirectory()). This scoping has a large impact on the configuration of the project and what effect the processing of files and setting variables have on the project as a whole. Some of the strange scoping rules for CMake are discussed in CMake Language Overview and Gotchas and should be understood before trying to debug issues with processing. Many of the basic files are processed (included) in the base project <projectDir>/CMakeLists.txt scope and therefore any local variables set in these files are accessible to the entire CMake project (after the file is processed, of course). Other files get processed inside of functions which have their own local scope and therefore only impact the rest of the project in more purposeful ways. And of course all of the package <packageDir>/CMakeLists.txt files that are processed using add_subdirectory() create a new local scope for that given package.

$ cmake [options] <projectDir>

$ cmake \
  -DTribitsExProj_TRIBITS_DIR=<tribitsDir> \
  -DTribitsExProj_ENABLE_MPI=ON \
  -DTribitsExProj_ENABLE_ALL_PACKAGES=ON \
  -DTribitsExProj_ENABLE_TESTS=ON \
  -DTribitsExProj_TRACE_FILE_PROCESSING=ON \
  -DTribitsExProj_ENABLE_CPACK_PACKAGING=ON \
  -DTribitsExProj_DUMP_CPACK_SOURCE_IGNORE_FILES=ON \
  <tribitsDir>/doc/TribitsExampleProject \
  | grep "^-- File Trace:"

-- File Trace: PROJECT    INCLUDE    [...]/Version.cmake
-- File Trace: REPOSITORY INCLUDE    [...]/cmake/CallbackSetupExtraOptions.cmake
-- File Trace: REPOSITORY INCLUDE    [...]/PackagesList.cmake
-- File Trace: REPOSITORY INCLUDE    [...]/TPLsList.cmake
-- File Trace: PACKAGE    INCLUDE    [...]/packages/simple_cxx/cmake/Dependencies.cmake
-- File Trace: PACKAGE    INCLUDE    [...]/packages/mixed_lang/cmake/Dependencies.cmake
-- File Trace: PACKAGE    INCLUDE    [...]/packages/with_subpackages/cmake/Dependencies.cmake
-- File Trace: PACKAGE    INCLUDE    [...]/packages/with_subpackages/a/cmake/Dependencies.cmake
-- File Trace: PACKAGE    INCLUDE    [...]/packages/with_subpackages/b/cmake/Dependencies.cmake
-- File Trace: PACKAGE    INCLUDE    [...]/packages/with_subpackages/c/cmake/Dependencies.cmake
-- File Trace: PACKAGE    INCLUDE    [...]/packages/wrap_external/cmake/Dependencies.cmake
-- File Trace: PROJECT    CONFIGURE  [...]/cmake/ctest/CTestCustom.cmake.in
-- File Trace: REPOSITORY READ       [...]/Copyright.txt
-- File Trace: REPOSITORY INCLUDE    [...]/Version.cmake
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/simple_cxx/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/simple_cxx/test/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/mixed_lang/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/mixed_lang/test/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/with_subpackages/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/with_subpackages/a/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/with_subpackages/a/tests/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/with_subpackages/b/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/with_subpackages/b/tests/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/with_subpackages/c/CMakeLists.txt
-- File Trace: PACKAGE    ADD_SUBDIR [...]/packages/with_subpackages/c/tests/CMakeLists.txt
-- File Trace: PROJECT    INCLUDE    [...]/cmake/CallbackDefineProjectPackaging.cmake
-- File Trace: REPOSITORY INCLUDE    [...]/cmake/CallbackDefineRepositoryPackaging.cmake

3.3   Coexisting Projects, Repositories, and Packages

Certain simplifications are allowed when defining TriBITS projects, repositories and packages. The known allowed simplifications are described below.

TriBITS Repository Dir == TriBITS Project Dir: It is allowed for a TriBITS Project and a TriBITS Repository to be the same source directory and in fact this is the default for every TriBITS project (unless the <projectDir>/cmake/NativeRepositoriesList.cmake is defined). In this case, the repository name REPOSITORY_NAME and the project name PROJECT_NAME are the same as well. This is quite common and is in fact the default that every TriBITS Project is also a TriBITS repository (and therefore must contain <repoDir>/PackagesList.cmake and <repoDir>/TPLsList.cmake files). This is the case, for example, with the Trilinos and the TribitsExampleProject projects and repositories. In this case, the Project's and the Repository's Version.cmake and Copyright.txt files are also one and the same, as they should be (see Project and Repository Versioning and Release Mode).

TriBITS Package Dir == TriBITS Repository Dir: It is also allowed for a TriBITS Repository to have only one package and to have that package be the base repository directory. The TriBITS Repository and the single TriBITS Package would typically have the same name in this case (but that is actually not required but it is confusing if they are not the same). For example, in the TriBITS test project MockTrilinos, the repository and package extraRepoOnePackage are the same directory. In this case, the file extraRepoOnePackage/PackagesList.cmake looks like:

tribits_repository_define_packages(
  extraRepoOnePackage   .     ST
  )

(Note the dot '.' for the package directory.)

This is also how the real TriBITS repository and package DataTransferKit is set up (at least that is the way it was when this document was first written).

However, to maximize flexibility, it is recommended that a TriBITS package and its TriBITS repository not share the same directory or the same name. This allows a TriBITS repository to define more packages later.

TriBITS Package Dir == TriBITS Repository Dir == TriBITS Project Dir: In the extreme, it is possible to collapse a single TriBITS package, repository, and project into the same base source directory. They can also share the same name for the package, repository and package. One example of this is the TriBITS project and The TriBITS Test Package themselves, which are both rooted in the base TriBITS/ source directory of the stand-alone TriBITS repository. There are a few restrictions and modifications needed to get this to work:

• The base CMakeLists.txt file must be modified to allow it to be processed both as the base project CMakeLists.txt file and as the package's base CMakeLists.txt file. In the case of TriBITS/CMakeLists.txt, a big if statement is used.

Other than that simple modification to the top-level CMakeLists.txt file, a TriBITS project, repository, and package can all be rooted in the same source directory.

The primary use case for collapsing a project, repository, and package into a single base source directory would be to support the stand-alone build of a TriBITS package as its own entity that uses an independent installation of the TriBITS (or a minimal snapshot of TriBITS). If a given TriBITS package has no required upstream TriBITS package dependencies and minimal external package/TPL dependencies (or only uses Standard TriBITS TPLs or Common TriBITS TPLs already defined in the tribits/core/std_tpls/ or tribits/common_tpls/ directories), then creating a stand-alone project build of a single TriBITS package requires fairly little extra overhead or duplication.

3.4   Standard and Common TPLs

While a TriBITS Repository can define their own external packages/TPLs and their own TPL find modules (see TriBITS External Package/TPL), the TriBITS source tree contains TriBITS find modules for a few different standard TPLs and common TPLs. Standard TriBITS TPLs are integral to the TriBITS system itself while Common TriBITS TPLs are TPL that are used in several different TriBITS Repositories and are contained in TriBITS for convenience and uniformity.

tribits/core/std_tpls/

FindTPLCUDA.cmake
FindTPLMPI.cmake

tribits_repository_define_tpls(
  MPI   "${${PROJECT_NAME}_TRIBITS_DIR}/core/std_tpls/"  PT
  CUDA  "${${PROJECT_NAME}_TRIBITS_DIR}/core/std_tpls/"  ST
  ...
  )

tribits/common_tpls/

find_modules
FindTPLBinUtils.cmake
FindTPLBLAS.cmake
FindTPLBoost.cmake
FindTPLCGNS.cmake
FindTPLCGNSDependencies.cmake
FindTPLHDF5.cmake
FindTPLLAPACK.cmake
FindTPLLAPACKDependencies.cmake
FindTPLNetcdf.cmake
FindTPLNetcdfDependencies.cmake
FindTPLPETSC.cmake
FindTPLPnetcdf.cmake
FindTPLProjectLastLib.cmake
utils

tribits_repository_define_tpls(
  BLAS   "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/"  PT
  LAPACK  "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/"  PT
  ...
  )

4.1   TriBITS-Compliant Internal Packages

For TriBITS packages that are defined, built, and installed from a TriBITS CMake project, there are an additional set of requirements for them to behavior correctly with respect to other TriBITS packages.

The requirements for TriBITS-compliant internal packages are:

• All of the requirements for a TriBITS-Compliant Package.
• At the end of configuration and generation, writes out a TriBITS-Compliant External Package file <Package>Config.cmake and supporting files under the build directory <buildDir>/cmake_packages/<Package>/ allowing the built (but not installed) package to be used by downstream CMake packages/projects.
• Provides an install target to create a TriBITS-Compliant External Package file <Package>Config.cmake and supporting files under the install directory <installDir>/lib/cmake/<Package>/ allowing the installed package to be used by downstream CMake packages/projects.
• [Optional] All of the upstream dependencies (recursively) are also TriBITS-compliant packages.

If a TriBITS package provides any CTest tests/examples, then it must also satisfy the following requirements:

• Test names must be prefixed with the package name <Package>_.
• The package name <Package> must be appended to the LABELS test property list.
• Tests should only be added if the variable <Package>_ENABLE_TESTS is true.
• Examples (that run as CTest tests) should only be added if the variable <Package>_ENABLE_EXAMPLES is true.
• The PROCESSORS test property and other test properties must be set in a way consistent with tribits_add_test() so as to run in parallel with other tests and not overwhelm the computing resources on the machine.
• The test <fullTestName> must not be added if the cache variable <fullTestName>_DISABLE is set to TRUE or if the cache variable <fullTestName>_SET_DISABLED_AND_MSG is set to non-empty (and the message string should be printed to STDOUT).

TriBITS internal packages that are defined using the TriBITS framework using the TriBITS-provided macros and functions such as tribits_add_library() and have tests defined using the functions tribits_add_test() and tribits_add_advanced_test() are automatically TriBITS-compliant internal packages. And when these TriBITS-implemented internal packages are installed, they automatically provide TriBITS-compliant external packages. But it is possible for a CMake package to write its own raw CMake code to satisfy these basic requirements for both internal and (installed) external packages.

4.2   TriBITS-Compliant External Packages

For packages that are installed on the system and not built in the current CMake project, a streamlined type of TriBITS External Package/TPL is a TriBITS-compliant external package. These special types of external package's don't need to provide a FindTPL<tplName>.cmake find module. Instead, they are fully defined by calling find_package(<Package>) or include(<someBaseDir>/<Package>Config.cmake) to load their <Package>Config.cmake package config file.

The requirements for TriBITS-compliant external packages are:

• All of the requirements for a TriBITS-Compliant Package.
• Defined by an installed <Package>Config.cmake file that provides IMPORTED targets and set() statements for all of the needed variables.
• Provides CMake variables:
  • <Package>_CONFIG or <Package>_TRIBITS_COMPLIANT_PACKAGE_CONFIG_FILE: Points to the file <Package>Config.cmake (i.e. ${CMAKE_CURRENT_LIST_FILE})
• [Optional] All of the upstream dependencies (recursively) are also provided as TriBITS-compliant external packages with <UpstreamPackage>Config.cmake files (see above) and all of the targets and variables for a TriBITS-compliant external package are defined when the <Package>Config.cmake file is included (or pulled in with find_package() or find_dependency()).

NOTE: TriBITS-compliant external packages that provide TriBITS-compliant external packages for all of their upstream dependencies are said to be fully TriBITS-compliant external packages while those that support the minimal requirements are said to be minimally TriBITS-compliant external packages. The TriBITS external package/TPL system is robust enough to deal with minimally TriBITS-compliant external packages. Any TriBITS external packages/TPLs upstream from a minimally TriBITS-compliant external package will be found again in the current TriBITS project. (In these cases, it is up to the user to make sure that the same upstream packages are found.)

5.1   TribitsHelloWorld

TribitsHelloWorld is about the simplest possible TriBITS project that you can imagine and is contained under the directory:

tribits/examples/TribitsHelloWorld/

This example project contains only a single TriBITS package and no frills at all (does not support MPI or Fortran). However, it does show how minimal a TriBITS Project (which is also a TriBITS Repository) and a TriBITS Package can be and still demonstrates some of the value of TriBITS over raw CMake. The simple HelloWorld package is used to compare with the raw CMakeLists.txt file in the RawHeloWorld example project in the TriBITS Overview document.

The directory structure for this example his shown below:

  TribitsHelloWorld/
    CMakeLists.txt
    PackagesList.cmake
    ProjectName.cmake
    README
    TPLsList.cmake
    hello_world/
      CMakeLists.txt
      cmake/
        Dependencies.cmake
      hello_world_lib.cpp
      hello_world_lib.hpp
      hello_world_main.cpp
      hello_world_unit_tests.cpp

This has all of the required TriBITS Project Core Files, TriBITS Repository Core Files, and TriBITS Package Core Files. It just builds a simple library, a simple executable, a test executable, and the tests them as shown by the file TribitsHelloWorld/hello_world/CMakeLists.txt which is:

tribits_package(HelloWorld)
tribits_add_library(hello_world_lib
  HEADERS hello_world_lib.hpp SOURCES hello_world_lib.cpp)
tribits_add_executable(hello_world NOEXEPREFIX SOURCES hello_world_main.cpp
  INSTALLABLE)
tribits_add_test(hello_world NOEXEPREFIX PASS_REGULAR_EXPRESSION "Hello World")
tribits_add_executable_and_test(unit_tests SOURCES hello_world_unit_tests.cpp
  PASS_REGULAR_EXPRESSION "All unit tests passed")
tribits_package_postprocess()

The build and test of this simple project is tested in the The TriBITS Test Package file:

TriBITS/test/core/ExamplesUnitTests/CMakeLists.txt

Note that this little example is a fully functional TriBITS Repository and can be embedded in to a larger TriBITS meta-project and be seamlessly built along with any other such TriBITS-based software.

5.2   TribitsExampleProject

TribitsExampleProject in an example TriBITS Project and TriBITS Repository contained in the TriBITS source tree under:

tribits/examples/TribitsExampleProject/

When this is used as the base TriBITS project, this is the directory corresponds to <projectDir> and <repoDir> referenced in TriBITS Project Core Files and TriBITS Repository Core Files, respectively.

Several files from this project are used as examples in the section TriBITS Project Structure. Here, a fuller description is given of this project and a demonstration of how TriBITS works. From this simple example project, one can quickly see how the basic structural elements of a TriBITS project, repository, and package (and subpackage) are pulled together.

This simple project shows how what is listed in files:

• <repoDir>/PackagesList.cmake,
• <packageDir>/cmake/Dependencies.cmake, and
• <packageDir>/<spkgDir>/cmake/Dependencies.cmake

are used to specify the packages and packages in a TriBITS project and repository. More details about the contents of the Dependencies.cmake files is described in the section Package Dependencies and Enable/Disable Logic.

The name of this project PROJECT_NAME is given in its TribitsExampleProject/ProjectName.cmake file:

# Must set the project name at very beginning before including anything else
set(PROJECT_NAME TribitsExProj)

# Turn on export dependency generation for WrapExteranl package
set(${PROJECT_NAME}_GENERATE_EXPORT_FILE_DEPENDENCIES_DEFAULT ON)

# Turn on by default the generation of the export files
set(${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES_DEFAULT ON)

The variable PROJECT_NAME=TribitsExProj is used to prefix (using "${PROJECT_NAME}_") all of the project's global TriBITS variables like TribitsExProj_ENABLE_TESTS, TribitsExProj_ENABLE_ALL_PACKAGES, etc.

The directory structure and key files for this example project are shown in the partial list of TribitsExampleProject Files and Directories below:

TribitsExampleProject/
  CMakeLists.txt
  Copyright.txt
  PackagesList.cmake
  ProjectName.cmake
  project-checkin-test-config.py
  TPLsList.cmake
  Version.cmake
  ...
  cmake/
    CallbackDefineProjectPackaging.cmake
    CallbackDefineRepositoryPackaging.cmake
    CallbackSetupExtraOptions.cmake
  packages/
    simple_cxx/
      CMakeLists.txt
      cmake/
        CheckFor__int64.cmake
        Dependencies.cmake
        SimpleCxx_config.h.in
      src/
        CMakeLists.txt
        SimpleCxx_HelloWorld.cpp
        SimpleCxx_HelloWorld.hpp
      test/
        CMakeLists.txt
        SimpleCxx_HelloWorld_Tests.cpp
    mixed_lang/ ...
    with_subpackages/
      CMakeLists.txt
      cmake/
        Dependencies.cmake
      A/
        CMakeLists.txt
        cmake/
          Dependencies.cmake
        ...
      B/ ...
      C/ ...
    wrap_external/ ...

Above, the sub-directories under packages/ are sorted according to the order listed in the TribitsExampleProject/PackagesList.cmake file:

tribits_repository_define_packages(
  SimpleCxx          packages/simple_cxx         PT
  MixedLang          packages/mixed_lang         PT
  InsertedPkg        InsertedPkg                 ST
  WithSubpackages    packages/with_subpackages   PT
  WrapExternal       packages/wrap_external      ST
  )

tribits_disable_package_on_platforms(WrapExternal Windows)
tribits_allow_missing_external_packages(InsertedPkg)

From this file, we get the list of top-level packages SimpleCxx, MixedLang, WithSubpackages, and WrapExternal (and their base package directories and testing group, see <repoDir>/PackagesList.cmake). (NOTE: By default the package InsertedPkg is not defined because its directory is missing, see How to insert a package into an upstream repo.)

A full listing of package files in TribitsExampleProject Files and Directories is only shown for the SimpleCxx package directory packages/simple_cxx/. For this package, <packageDir> = <repoDir>/packages/simple_cxx and PACKAGE_NAME = SimpleCxx. As explained in TriBITS Package Core Files, the files <packageDir>/cmake/Dependencies.cmake and <packageDir>/CMakeLists.txt must exist for every package directory listed in <repoDir>/PackagesList.cmake and we see these files under in the directory packages/simple_cxx/. The package SimpleCxx does not have any upstream package dependencies.

Now consider the example top-level package WithSubpackages which, as the name suggests, is broken down into subpackages. The WithSubpackages dependencies file:

TribitsExampleProject/packages/with_subpackages/cmake/Dependencies.cmake

with contents:

tribits_package_define_dependencies(
  SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS
    A   a   PT  REQUIRED
    B   b   ST  OPTIONAL
    C   c   ST  OPTIONAL
  REGRESSION_EMAIL_LIST with_packages-regressions@someurl.none
  )

references the three subpackages with sub-directories <spkgDir> = A, B, and C under the parent package directory packages/package_with_packages/ which are shown in TribitsExampleProject Files and Directories. This gives another set of three packages WithSubpackagesA, WithSubpackagesB, and WithSubpackagesC. Combining <packageDir> = packages/package_with_packages and <spkgDir> for each subpackage gives the subpackage directories:

TribitsExampleProject/packages/with_subpackages/a/
TribitsExampleProject/packages/with_subpackages/b/
TribitsExampleProject/packages/with_subpackages/c/

Together with the top-level parent package WithSubpackages itself, this top-level package provides four TriBITS Packages giving the final list of packages provided by this TriBITS repo as:

SimpleCxx MixedLang WithSubpackagesA WithSubpackagesB WithSubpackagesC \
  WithSubpackages WrapExternal 7

The above list of packages is printed (with the number of packages printed at the end) by TriBITS to the cmake stdout on the line starting with "Final set of non-enabled packages:" when no packages are enabled (see Selecting the list of packages to enable). (Note that TriBITS does not put in line-brakes with continuation characters "\" as shown above.) TriBITS defines enable/disable cache variables for each of these defined packages like TribitsExProj_ENABLE_SimpleCxx and TribitsExProj_ENABLE_WithSubpackagesA, and defines all the variables listed in TriBITS Package Cache Variables that are settable by the users or by the dependency logic described in section Package Dependencies and Enable/Disable Logic.

When starting a new TriBITS project, repository, or package, one should consider basing these on the examples in TribitsExampleProject. In fact, the skeletons for any of the

• TriBITS Project Core Files,
• TriBITS Repository Core Files,
• TriBITS Package Core Files, or
• TriBITS Subpackage Core Files

should be copied from this example project as they represent best practice when using TriBITS for the typical use cases.

5.3   TribitsExampleProject2

TribitsExampleProject2 in an example TriBITS Project and TriBITS Repository contained in the TriBITS source tree under:

tribits/examples/TribitsExampleProject2/

This example TriBITS project provides some examples for a few other features and testing scenarios. It contains three internal packages Package1, Package2, and Package3 as shown in its PackagesList.cmake file:

tribits_repository_define_packages(
  Package1          packages/package1         PT
  Package2          packages/package2         PT
  Package3          packages/package3         PT
  )

and supports four external packages/TPLs Tpl1, Tpl2, Tpl3, and Tpl4 as shown in its TPLsList.cmake file:

tribits_repository_define_tpls(
  Tpl1      "cmake/tpls/"      PT
  Tpl2      "cmake/tpls/"      PT
  Tpl3      "${CMAKE_CURRENT_LIST_DIR}/cmake/tpls/"      PT
  Tpl4      "cmake/tpls/"      PT
  )

# NOTE: Above we are setting the findmod path to an absolute path just to test
# that case with TPL dependencies (see trilinos/Trilinos#10774).

5.4   MockTrilinos

The TriBITS project MockTrilinos is contained under the directory:

tribits/examples/MockTrilinos/

This TriBITS project is not a full TriBITS project (i.e. it does not build anything). Instead, it is primarily used to test the TriBITS system using tests defined in the The TriBITS Test Package. The MockTrilinos project is actually given the name PROJECT_NAME = Trilinos and contains a subset of Trilinos packages with slightly modified dependencies from a real version of the Trilinos project from May 2009. The list of packages in:

tribits/examples/MockTrilinos/PackagesList.cmake

is:

tribits_repository_define_packages(
  TrilinosFramework   cmake                           PT
  Teuchos             packages/teuchos                PT
  RTOp                packages/rtop                   PT
  Epetra              packages/epetra                 PT
  Zoltan              packages/zoltan                 PT
  Shards              packages/shards                 PT
  Triutils            packages/triutils               PT
  Tpetra              packages/tpetra                 PT
  EpetraExt           packages/epetraext              PT
  Stokhos             packages/stokhos                EX
  Sacado              packages/sacado                 ST
  Thyra               packages/thyra                  PT
  Isorropia           packages/isorropia              PT
  AztecOO             packages/aztecoo                PT
  Galeri              packages/galeri                 PT
  Amesos              packages/amesos                 PT
  Intrepid            packages/intrepid               PT
  Ifpack              packages/ifpack                 PT
  ML                  packages/ml                     PT
  Belos               packages/belos                  ST
  Stratimikos         packages/stratimikos            PT
  RBGen               packages/rbgen                  PT
  Phalanx             packages/phalanx                ST
  Panzer              packages/panzer                 ST
  AlwaysMissing       AlwaysMissing                   PT
  )

# NOTE: Sacado was really PT but for testing purpose it is made ST
# NOTE: Belos was really PT but for testing purpose it is made ST

tribits_allow_missing_external_packages(AlwaysMissing)
tribits_disable_package_on_platforms(ML BadSystem1)
tribits_disable_package_on_platforms(Ifpack BadSystem1 BadSystem2)

All of the package directories listed above have <packageDir>/cmake/Dependencies.cmake files but generally do not have <packageDir>/CMakeLists.txt files since most of usage of MockTrilinos just involves the testing of the algorithms and behaviors described in the section Package Dependencies and Enable/Disable Logic.

MockTrilinos also contains a number of extra TriBITS repositories used in various tests. These extra repositories offer examples of different types of TriBITS repositories like:

• extraRepoOnePackage: Contains just the single package extraRepoOnePackage which is defined in the base repository directory.
• extraRepoOnePackageThreeSubpackages: Contains just the single package extraRepoOnePackageThreeSubpackages which is defined in the base repository directory but is broken up into subpackages.
• extraRepoTwoPackages: Contains just two packages but provides an example of defining multiple repositories with possible missing required and optional upstream packages (see Multi-Repository Support).
• extraTrilinosRepo: Just a typical extra repo with add-on packages and external packages/TPLs that depends on a few upstream MockTrilinos packages.

New test extra repositories are added when new types of tests are needed that would require new package and TPL dependency structures since existing dependency tests based on MockTrilinos are expensive to change by their very nature.

The primary reason that the MockTrilinos test project is mentioned in this developers guide is because it contains a variety of packages, subpackages, and TPLs with a variety of different types of dependencies. This variety is is needed to more fully test the TriBITS system but this project and the tests also serve as examples and extra documentation for the behavior of the TriBITS system. Several of the dependency-related examples referenced in this document come from MockTrilinos.

Most of the dependency tests involving MockTrilinos are specified in:

TriBITS/test/core/DependencyUnitTests/CMakeLists.txt

A great deal about the current behavior of TriBITS Package Dependencies and Enable/Disable Logic can be learned from inspecting these tests. There are also some faster-running unit tests involving MockTrilinos defined in the file:

TriBITS/test/core/TribitsAdjustPackageEnables_UnitTests.cmake

5.5   ReducedMockTrilinos

The TriBITS project ReducedMockTrilinos is contained under the directory:

tribits/examples/ReducedMockTrilinos/

It is a scaled-down version of the MockTrilinos test project with just a handful of packages and some modified dependencies. Its primary purpose for this example project is to be used for examples in the section Package Dependencies and Enable/Disable Logic and to test a few features of the TriBITS system not covered in other tests.

The list of packages in:

tribits/examples/ReducedMockTrilinos/PackagesList.cmake

is:

tribits_repository_define_packages(
  Teuchos             packages/teuchos                PT
  RTOp                packages/rtop                   PT
  Epetra              packages/epetra                 PT
  Triutils            packages/triutils               ST
  EpetraExt           packages/epetraext              ST
  Thyra               packages/thyra                  PT
  )

All of the listed packages are standard TriBITS packages except for the mock Thyra package which is broken down into subpackages. More details of this example project are described in Package Dependencies and Enable/Disable Logic.

5.6   Trilinos

The real Trilinos project and repository itself is an advanced example for the usage of TriBITS. Almost every single-repository use case for TriBITS is demonstrated somewhere in Trilinos. While some of the usage of TriBITS in Trilinos may not be not exemplary (e.g., because it represents old usage, or was written by CMake/TriBITS beginners) it does represent real working usage. Given that Trilinos is a widely available software repository, anyone should be able to access a newer version of Trilinos and mine it for CMake and TriBITS examples.

5.7   The TriBITS Test Package

The last TriBITS example mentioned here is the TriBITS test package named (appropriately) TriBITS itself defined in the TriBITS repository. The directory for the TriBITS test package is the base TriBITS source directory tribits. This allows any TriBITS project to add testing for the TriBITS system by just listing the TriBITS repository in its <projectDir>/cmake/ExtraRepositoriesList.cmake file. Trilinos lists the TriBITS repository in its ExtraRepositoriesList.cmake file as:

tribits_project_define_extra_repositories(

TriBITS "" GIT https://github.com/TriBITSPub/TriBITS "" Continuous ... )

No downstream TriBITS packages list a dependency on TriBITS in their <packageDir>/cmake/Dependencies.cmake files. Defining the TriBITS TriBITS package in only done for running the TriBITS tests.

Once the TriBITS test package is added to the list of project/repository packages, it can be enabled just like any other package by adding the following to the cmake command-line options:

-D <Project>_ENABLE_TriBITS=ON \
-D <Project>_ENABLE_TESTS=ON

One can then inspect the added tests prefixed by "TriBITS_" to see what tests are defined and how they are run. There is a wealth of information about the TriBITS system embedded in these tests and where documentation and these tests disagreed, believe the tests!

6.1   Example ReducedMockTrilinos Project Dependency Structure

To demonstrate the TriBITS package dependency handling system, the small simple ReducedMockTrilinos project is used. The list of packages for this project is defined in the file ReducedMockTrilinos/PackagesList.cmake (see <repoDir>/PackagesList.cmake) which contents:

tribits_repository_define_packages(
  Teuchos             packages/teuchos                PT
  RTOp                packages/rtop                   PT
  Epetra              packages/epetra                 PT
  Triutils            packages/triutils               ST
  EpetraExt           packages/epetraext              ST
  Thyra               packages/thyra                  PT
  )

All of the listed packages are standard TriBITS packages except for the mock Thyra package which is broken down into subpackages as shown in packages/thyra/cmake/Dependnecies.cmake (see <packageDir>/cmake/Dependencies.cmake) which is:

tribits_package_define_dependencies(
  SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS
    CoreLibs      src                  PT  REQUIRED
    GoodStuff     good_stuff           ST  OPTIONAL
    CrazyStuff    crazy_stuff          EX  OPTIONAL
    Epetra        adapters/epetra      PT  OPTIONAL
    EpetraExt     adapters/epetraext   ST  OPTIONAL
  )

This gives the full list of top-level TriBITS packages:

Teuchos RTOp Epetra Triutils EpetraExt Thyra

Adding in the subpackages defined in the top-level Thyra package, the full set of TriBITS Packages for this project is:

Teuchos RTOp Epetra Triutils EpetraExt ThyraCoreLibs ThyraGoodStuff \
  ThyraCrazyStuff ThyraEpetra ThyraEpetraExt Thyra

Note that one can see this full list of top-level packages and packages in the cmake configure output lines starting with:

Final set of non-enabled top-level packages:
Final set of non-enabled packages:

respectively, when configuring with no package enables as shown in the example Default configure with no packages enabled on input.

The list of TriBITS External Packages/TPLs for this example project given in the file ReducedMockTrilinos/TPLsList.cmake (see <repoDir>/TPLsList.cmake) which is:

tribits_repository_define_tpls(
  MPI            "${${PROJECT_NAME}_TRIBITS_DIR}/core/std_tpls/"  PT
  BLAS           "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/"    PT
  LAPACK         "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/"    PT
  Boost          cmake/TPLs/    ST
  UMFPACK        cmake/TPLs/    ST
  AMD            cmake/TPLs/    EX
  PETSC          "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/"    ST
  )

Take note of the Package Test Group (i.e. PT, ST, or EX) assigned to each package as it plays a significant role in how the TriBITS dependency system handles enables and disables.

The dependency structure of this simple TriBITS project is shown below in ReducedMockTrilinos Dependencies.

ReducedMockTrilinos Dependencies:

Package dependencies information:

-- Trilinos_DEFINED_TPLS: MPI BLAS LAPACK Boost UMFPACK AMD PETSC
-- Trilinos_NUM_DEFINED_TPLS='7'

-- Trilinos_DEFINED_INTERNAL_TOPLEVEL_PACKAGES: Teuchos RTOp Epetra Triutils EpetraExt Thyra
-- Trilinos_NUM_DEFINED_INTERNAL_TOPLEVEL_PACKAGES='6'

-- Trilinos_DEFINED_TOPLEVEL_PACKAGES: MPI BLAS LAPACK Boost UMFPACK AMD PETSC Teuchos RTOp Epetra Triutils EpetraExt Thyra
-- Trilinos_NUM_DEFINED_TOPLEVEL_PACKAGES='13'

-- Trilinos_DEFINED_INTERNAL_PACKAGES: Teuchos RTOp Epetra Triutils EpetraExt ThyraCoreLibs ThyraGoodStuff ThyraCrazyStuff ThyraEpetra ThyraEpetraExt Thyra
-- Trilinos_NUM_DEFINED_INTERNAL_PACKAGES='11'

-- Trilinos_DEFINED_PACKAGES: MPI BLAS LAPACK Boost UMFPACK AMD PETSC Teuchos RTOp Epetra Triutils EpetraExt ThyraCoreLibs ThyraGoodStuff ThyraCrazyStuff ThyraEpetra ThyraEpetraExt Thyra
-- Trilinos_NUM_DEFINED_PACKAGES='18'

-- MPI_FORWARD_LIB_DEFINED_DEPENDENCIES: Teuchos[O] Epetra[O]

-- BLAS_FORWARD_LIB_DEFINED_DEPENDENCIES: LAPACK[O] Teuchos[R] Epetra[R]

-- LAPACK_LIB_DEFINED_DEPENDENCIES: BLAS[O]
-- LAPACK_FORWARD_LIB_DEFINED_DEPENDENCIES: Teuchos[R] Epetra[R]

-- Boost_FORWARD_LIB_DEFINED_DEPENDENCIES: Teuchos[O]

-- UMFPACK_FORWARD_LIB_DEFINED_DEPENDENCIES: EpetraExt[O]

-- AMD_FORWARD_LIB_DEFINED_DEPENDENCIES: EpetraExt[O]

-- PETSC_FORWARD_LIB_DEFINED_DEPENDENCIES: EpetraExt[O]

-- Teuchos_LIB_DEFINED_DEPENDENCIES: BLAS[R] LAPACK[R] Boost[O] MPI[O]
-- Teuchos_FORWARD_LIB_DEFINED_DEPENDENCIES: RTOp[R] EpetraExt[R] ThyraCoreLibs[R]

-- RTOp_LIB_DEFINED_DEPENDENCIES: Teuchos[R]
-- RTOp_FORWARD_LIB_DEFINED_DEPENDENCIES: ThyraCoreLibs[R]

-- Epetra_LIB_DEFINED_DEPENDENCIES: BLAS[R] LAPACK[R] MPI[O]
-- Epetra_FORWARD_LIB_DEFINED_DEPENDENCIES: Triutils[R] EpetraExt[R] ThyraEpetra[R]

-- Triutils_LIB_DEFINED_DEPENDENCIES: Epetra[R]
-- Triutils_FORWARD_LIB_DEFINED_DEPENDENCIES: EpetraExt[O]

-- EpetraExt_LIB_DEFINED_DEPENDENCIES: Teuchos[R] Epetra[R] Triutils[O] UMFPACK[O] AMD[O] PETSC[O]
-- EpetraExt_FORWARD_LIB_DEFINED_DEPENDENCIES: ThyraEpetraExt[R]

-- ThyraCoreLibs_LIB_DEFINED_DEPENDENCIES: Teuchos[R] RTOp[R]
-- ThyraCoreLibs_FORWARD_LIB_DEFINED_DEPENDENCIES: ThyraGoodStuff[R] ThyraEpetra[R] Thyra[R]

-- ThyraGoodStuff_LIB_DEFINED_DEPENDENCIES: ThyraCoreLibs[R]
-- ThyraGoodStuff_FORWARD_LIB_DEFINED_DEPENDENCIES: ThyraCrazyStuff[R] Thyra[O]

-- ThyraCrazyStuff_LIB_DEFINED_DEPENDENCIES: ThyraGoodStuff[R]
-- ThyraCrazyStuff_FORWARD_LIB_DEFINED_DEPENDENCIES: Thyra[O]

-- ThyraEpetra_LIB_DEFINED_DEPENDENCIES: Epetra[R] ThyraCoreLibs[R]
-- ThyraEpetra_FORWARD_LIB_DEFINED_DEPENDENCIES: ThyraEpetraExt[R] Thyra[O]

-- ThyraEpetraExt_LIB_DEFINED_DEPENDENCIES: ThyraEpetra[R] EpetraExt[R]
-- ThyraEpetraExt_FORWARD_LIB_DEFINED_DEPENDENCIES: Thyra[O]

-- Thyra_LIB_DEFINED_DEPENDENCIES: ThyraCoreLibs[R] ThyraGoodStuff[O] ThyraCrazyStuff[O] ThyraEpetra[O] ThyraEpetraExt[O]

Dumping direct enabled dependencies for each package ...

-- MPI: No enabled dependencies!

-- BLAS: No enabled dependencies!

-- LAPACK_LIB_ENABLED_DEPENDENCIES: BLAS[O]

-- Boost: No enabled dependencies!

-- UMFPACK: No enabled dependencies!

-- AMD: No enabled dependencies!

-- PETSC: No enabled dependencies!

-- Teuchos_LIB_ENABLED_DEPENDENCIES: BLAS[R] LAPACK[R]

-- RTOp_LIB_ENABLED_DEPENDENCIES: Teuchos[R]

-- Epetra_LIB_ENABLED_DEPENDENCIES: BLAS[R] LAPACK[R]

-- Triutils: No enabled dependencies!

-- EpetraExt: No enabled dependencies!

-- ThyraCoreLibs_LIB_ENABLED_DEPENDENCIES: Teuchos[R] RTOp[R]

-- ThyraGoodStuff: No enabled dependencies!

-- ThyraCrazyStuff: No enabled dependencies!

-- ThyraEpetra_LIB_ENABLED_DEPENDENCIES: Epetra[R] ThyraCoreLibs[R]

-- ThyraEpetraExt: No enabled dependencies!

-- Thyra_LIB_ENABLED_DEPENDENCIES: ThyraCoreLibs[R] ThyraEpetra[O]

Setting up export dependencies for all enabled packages ...

-- Teuchos: No library dependencies!
-- RTOp_FULL_ENABLED_DEP_PACKAGES: Teuchos
-- Epetra: No library dependencies!
-- ThyraCoreLibs_FULL_ENABLED_DEP_PACKAGES: RTOp Teuchos
-- ThyraEpetra_FULL_ENABLED_DEP_PACKAGES: ThyraCoreLibs Epetra RTOp Teuchos
-- Thyra_FULL_ENABLED_DEP_PACKAGES: ThyraEpetra ThyraCoreLibs Epetra RTOp Teuchos

The above dependency structure printout is produced by configuring with ${PROJECT_NAME}_DUMP_PACKAGE_DEPENDENCIES=ON (which also results in more dependency information than what is shown above, e.g. like computed forward package dependencies). Note that the top-level package Thyra is shown to depend on its subpackages (not the other way around). (Many people are confused about the nature of the dependencies between packages and subpackages. See <packageDir>/<spkgDir>/cmake/Dependencies.cmake for more discussion.)

A number of user-settable CMake cache variables determine what packages and what tests and examples get enabled. These cache variables are described in Selecting the list of packages to enable and are described below. Also, the assigned Package Test Group (i.e. PT, ST, and EX) also affects what packages get enabled or disabled.

Any of these packages can be enabled or disabled with ${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=(ON|OFF) (the default enable is typically empty "", see PT/ST packages given default unset enable/disable state). For ReducedMockTrilinos, this gives the enable/disable cache variables (with the initial default values):

Trilinos_ENABLE_Teuchos=""
Trilinos_ENABLE_RTOp""
Trilinos_ENABLE_Epetra=""
Trilinos_ENABLE_Triutils=""
Trilinos_ENABLE_EpetraExt=""
Trilinos_ENABLE_ThyraCore=""
Trilinos_ENABLE_ThyraGoodStuff=""
Trilinos_ENABLE_ThyraCrazyStuff="OFF"  # Because it is 'EX'
Trilinos_ENABLE_ThyraEpetra=""
Trilinos_ENABLE_ThyraEpetraExt=""

Every TriBITS package is assumed to have tests and/or examples so TriBITS defines the following cache variables as well (with the initial default values):

Teuchos_ENABLE_TESTS=""
RTOp_ENABLE_TESTS=""
Epetra_ENABLE_TESTS=""
Triutils_ENABLE_TESTS=""
EpetraExt_ENABLE_TESTS=""
ThyraCoreLibs_ENABLE_TESTS=""
ThyraGoodStuff_ENABLE_TESTS=""
ThyraEpetra_ENABLE_TESTS=""
ThyraEpetraExt_ENABLE_TESTS=""
Thyra_ENABLE_TESTS=""

NOTE: TriBITS only sets the variables <TRIBITS_PACKAGE>_ENABLE_TESTS into the cache if the package <TRIBITS_PACKAGE> becomes enabled at some point. This cuts down the clutter in the CMake cache for large projects with lots of packages where the user only enables a subset of the packages.

NOTE: TriBITS also defines the cache variables <TRIBITS_PACKAGE>_ENABLE_EXAMPLES for each enabled TriBITS package which is handled the same way as the <TRIBITS_PACKAGE>_ENABLE_TEST variables.

Also, every defined external package/TPL is given its own TPL_ENABLE_<TRIBITS_TPL> enable/disable cache variable. For the TPLs in ReducedMockTrilinos, this gives the enable/disable cache variables (with default values):

TPL_ENABLE_MPI=""
TPL_ENABLE_BLAS=""
TPL_ENABLE_LAPACK=""
TPL_ENABLE_Boost=""
TPL_ENABLE_UMFPACK=""
TPL_ENABLE_AMD=""
TPL_ENABLE_PETSC=""

In addition, for every optional package dependency, TriBITS defines a cache variable <TRIBITS_PACKAGE>_ENABLE_<OPTIONAL_DEP>. For the optional dependencies shown in ReducedMockTrilinos Dependencies, that gives the additional cache variables (with default values):

Teuchos_ENABLE_Boost=""
Teuchos_ENABLE_MPI=""
Teuchos_ENABLE_Boost=""
Epetra_ENABLE_MPI=""
EpetraExt_ENABLE_Triutils=""
EpetraExt_ENABLE_UMFPACK=""
EpetraExt_ENABLE_AMD=""
EpetraExt_ENABLE_PETSC=""
Thyra_ENABLE_ThyraGoodStuff=""
Thyra_ENABLE_ThyraCrazyStuff=""
Thyra_ENABLE_ThyraEpetra=""
Thyra_ENABLE_ThyraEpetraExt=""

The above optional package-specific cache variables allow one to control whether or not support for upstream dependency X is turned on in package Y independent of whether or not X and Y are themselves both enabled. For example, if the packages Triutils and EpetraExt are both enabled, one can explicitly disable support for the optional dependency Triutils in EpetraExt by setting EpetraExt_ENABLE_Triutils=OFF. One may want to do this for several reasons but the bottom line is that this gives the user more detailed control over package dependencies. See the TriBITS Dependency Handling Behaviors and Explicit disable of an optional package dependency for more discussion and examples.

Before getting into specific Example Enable/Disable Use Cases, some of the TriBITS Dependency Handling Behaviors are first defined below.

6.2   TriBITS Dependency Handling Behaviors

Below, some of the rules and behaviors of the TriBITS dependency management system are described. Examples refer to the Example ReducedMockTrilinos Project Dependency Structure. More detailed examples of these behaviors are given in the section Example Enable/Disable Use Cases.

In brief, the rules/behaviors of the TriBITS package dependency management system are:

0. No circular dependencies of any kind are allowed
1. PT/ST packages given default unset enable/disable state
2. EX packages disabled by default
3. Package enable triggers auto-enables of upstream dependencies
4. Package disable triggers auto-disables of downstream dependencies
5. PT/ST TPLs given default unset enable/disable state
6. EX TPLs given default unset enable/disable state
7. Required TPLs are auto-enabled for enabled packages
8. Optional TPLs only enabled explicitly by the user
9. Disables trump enables where there is a conflict
10. Enable/disable of parent package is enable/disable for subpackages
11. Enable/disable of parent package tests/examples is enable/disable for subpackages tests/examples
12. Subpackage enable does not auto-enable the parent package
13. Support for optional package/TPL is enabled by default
14. Support for optional package can be explicitly disabled
15. Explicit enable of optional package/TPL support auto-enables package/TPL
16. ST packages only auto-enabled if ST code is enabled
17. <Project>_ENABLE_ALL_FORWARD_DEP_PACKAGES enables downstream packages/tests
18. <Project>_ENABLE_ALL_PACKAGES enables all PT (cond. ST) packages
19. <Project>_ENABLE_TESTS only enables explicitly enabled package tests
20. If no packages are enabled, nothing will get built
21. TriBITS prints all enables and disables to stdout
22. TriBITS auto-enables/disables done using non-cache local variables

In more detail, these rules/behaviors are:

0. No circular dependencies of any kind are allowed: The zeroth rule of the TriBITS dependency management system is that no TriBITS package (or its tests) can declare a dependency on a downstream package, period! This is one of the most basic Software Engineering Packaging Principles stated as the ADP (Acyclic Dependencies Principle), i.e. "Allow no cycles in the package dependency graph". By default, the TriBITS system will check for circular dependencies and will fail the configure if any are found. For a more detailed discussion of circular package dependencies, see Design Considerations for TriBITS.

1. PT/ST packages given default unset enable/disable state: A package <TRIBITS_PACKAGE> with testing group PT or ST is given an unset enable/disable state by default (i.e. ${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=""). For example, the PT package Teuchos is not enabled or disabled by default and is given the initial value Trilinos_ENABLE_Teuchos="". For an example, see Default configure with no packages enabled on input. This allows PT, and ST packages to be enabled or disabled using other logic defined by TriBITS which is described below. TriBITS defines persistent cache variables for these with the default value of empty "". Therefore, if the user or other CMake code does not hard enable or disable one of these variables, then on future configures it will be defined but have the value of empty "".

2. EX packages disabled by default: An package <TRIBITS_PACKAGE> with testing group EX is disabled by default (i.e. ${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=OFF). For an example, see Default configure with no packages enabled on input. This results in all required downstream packages to be disabled by default. However, the user can explicitly set ${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=ON for an EX package and it will be enabled (unless one of its required dependencies are not enabled for some reason). In this case, the cache variable is given the cache value of OFF.

3. Package enable triggers auto-enables of upstream dependencies: Any package <TRIBITS_PACKAGE> can be explicitly enabled by the user by setting the cache variable ${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=ON (e.g. Trilinos_ENABLE_EpetraExt=ON). When an package is enabled in this way, the TriBITS system will try to enable all of the required upstream packages and TPLs defined by the package (specified in its Dependencies.cmake file). If an enabled package can't be enabled and has to be disabled, either a warning is printed or processing will stop with an error (depending on the value of ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES, see Disables trump enables where there is a conflict). In addition, if ${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON, then TriBITS will try to enable all of the specified optional PT packages as well (and also optional upstream ST packages if ${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON). For an example, see Explicit enable of a package, its tests, an optional TPL, with ST enabled.

4. Package disable triggers auto-disables of downstream dependencies: Any package <TRIBITS_PACKAGE> can be explicitly disabled by the user by setting the cache variable ${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=OFF (or TPL_ENABLE_<TRIBITS_PACKAGE>=OFF for an external package/TPL) (e.g. Trilinos_ENABLE_Teuchos=OFF, TPL_ENABLE_BLAS=OFF). When an package is explicitly disabled, it will result in the disable of all dependent downstream external packages/TPLs and internal packages that have required dependency on it. It will also disable optional support for the disabled packages in downstream packages that list it as an optional dependency. For an example, see Explicit disable of a package.

5. PT/ST TPLs given default unset enable/disable state: A TriBITS TPL <TRIBITS_TPL> with testing group PT or ST is given an unset enable/disable state by default (i.e. TPL_ENABLE_<TRIBITS_TPL>=""). For example, the PT TPL BLAS is not enabled or disabled by default (i.e. TPL_ENABLE_BLAS=""). For an example, see Default configure with no packages enabled on input. This allows PT, and ST TPLs to be enabled or disabled using other logic.

6. EX TPLs given default unset enable/disable state: A TriBITS TPL <TRIBITS_TPL> with testing group EX, is given an unset enable/disable state by default (i.e. TPL_ENABLE_<TRIBITS_TPL>="", same as for PT and EX TPLs). For an example, see Default configure with no packages enabled on input. This is different behavior than for EX packages described above which provides an initial hard disable. However, since TriBITS will never automatically enable an optional TPL (see Optional TPLs only enabled explicitly by the user) and since only downstream EX packages are allowed to have a required dependencies on an EX TPL, there is no need to set the default enable for an EX TPL to OFF. If an EX package has a required dependency on an EX TPL, just enabling the EX package should automatically enable the EX TPL as described in Required TPLs are auto-enabled for enabled packages.

7. Required TPLs are auto-enabled for enabled packages: All TPLs listed as required TPL dependencies for the final set of enabled packages are set to enabled (i.e. TPL_ENABLE_<TRIBITS_TPL>=ON), unless the listed TPLs are already explicit disabled (in which case the package would be disabled or an error would occur, see Disables trump enables where there is a conflict). For example, if the Teuchos package is enabled, then that will trigger the enable of its required TPLs BLAS and LAPACK. For an example, see Explicit enable of a package and its tests.

8. Optional TPLs only enabled explicitly by the user: Optional TPLs, regardless of their testing group PT, ST or EX, will only be enabled if they are explicitly enabled by the user. For example, just because the package Teuchos is enabled, the optional TPLs Boost and MPI will not be enabled by default, even if ${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON. To enable the optional TPL Boost, for example, and enable support for Boost in the Teuchos package, the user must explicitly set TPL_ENABLE_Boost=ON. For an example, see Explicit enable of a package, its tests, an optional TPL, with ST enabled.

9. Disables trump enables where there is a conflict and TriBITS will never override a disable in order to satisfy some dependency. For example, if the user sets Trilinos_ENABLE_Teuchos=OFF and Trilinos_ENABLE_RTOp=ON, then TriBITS will not override the disable of Teuchos in order to satisfy the required dependency of RTOp. In cases such as this, the behavior of the TriBITS dependency adjustment system will depend on the setting of the top-level user cache variable ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES:

   • If ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON, then TriBITS will disable the explicit enable and continue on. In the above example, TriBITS will override Trilinos_ENABLE_RTOp=ON and set Trilinos_ENABLE_RTOp=OFF and print a verbose warning to the cmake stdout.
   • If ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=OFF, then TriBITS will generate a detailed error message printed to cmake stdout and then abort processing. In the above example, TriBITS will report that RTOp is enabled but the required package Teuchos is disabled and therefore RTOp can't be enabled and processing must stop.

   For an example of both behaviors, see Conflicting explicit enable and disable.

10. Enable/disable of parent package is enable/disable for subpackages: An explicit enable/disable of a top-level parent package with subpackages with ${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=(ON|OFF) is equivalent to the explicit enable/disable of all of the parent package's subpackages. For example, explicitly setting Trilinos_ENABLE_Thyra=ON is equivalent to explicitly setting:

    Trilinos_ENABLE_ThyraCoreLibs=ON
    Trilinos_ENABLE_ThyraGoodStuff=ON   # Only if enabling ST code!
    Trilinos_ENABLE_ThyraEpetra=ON
    Trilinos_ENABLE_ThyraEpetraExt=ON   # Only if enabling ST code!
    

    (Note that Trilinos_ENABLE_ThyraCrazyStuff is not set to ON because it is already set to OFF by default, see EX packages disabled by default.) Likewise, explicitly setting Trilinos_ENABLE_Thyra=OFF is equivalent to explicitly setting all of the Thyra subpackages to OFF at the outset. For a PT example, see Explicit enable of a package and its tests. For a ST example, see Explicit enable of a package, its tests, an optional TPL, with ST enabled.

11. Enable/disable of parent package tests/examples is enable/disable for subpackages tests/examples: Setting <TRIBITS_PACKAGE>_ENABLE_TESTS=[ON|OFF] is equivalent to setting the default for <TRIBITS_PACKAGE><SP>_ENABLE_TESTS=[ON|OFF] for each subpackage <SP> of the parent package <TRIBITS_PACKAGE> (if <TRIBITS_PACKAGE> has subpackages). Same is true for <TRIBITS_PACKAGE>_ENABLE_EXAMPLES=[ON|OFF] setting the default for <TRIBITS_PACKAGE><SP>_ENABLE_EXAMPLES=[ON|OFF]. In addition, setting <TRIBITS_PACKAGE>_ENABLE_TESTS=[ON|OFF] will set <TRIBITS_PACKAGE>_ENABLE_EXAMPLES=[ON|OFF] by default as well (but not vice versa).

12. Subpackage enable does not auto-enable the parent package: Enabling an package that is a subpackage does not automatically enable the parent package (except for at the very end, mostly just for show). For example, enabling the package ThyraEpetra does not result in enable of the parent Thyra package, (except when ${PROJECT_NAME}_ENABLE_ALL_FORWARD_DEP_PACKAGES=ON for course). For an example, see Explicit enable of a subpackage. This means that if a downstream package declares a dependency on the package ThyraEpetra, but not the parent package Thyra, then the Thyra package (and its other subpackages and their dependencies) will not get auto-enabled. Also note that enabling a subset of the required subpackages for a parent package does not require the enable of the other required subpackages. For example, if both ThyraEpetra and ThyraCore were both required subpackages of the parent package Thyra, then enabling ThyraCore does not require the enabling of ThyraEpetra. In these cases where one of a parent package's subpackages is enabled for some reason, the final value of ${PROJECT_NAME}_ENABLE_${PACKAGE_NAME} will be set to ON (but this does imply that all of the required subpackages will be enabled, only that the parent package will be processed.)

13. Support for optional package/TPL is enabled by default: For an package <TRIBITS_PACKAGE> with an optional dependency on an upstream package or TPL <TRIBITS_DEP_PACKAGE_OR_TPL>, TriBITS will automatically set the intra-enable variable <TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE_OR_TPL>=ON if <TRIBITS_PACKAGE> and <TRIBITS_DEP_PACKAGE_OR_TPL> are both enabled. For example, if the packages Triutils and EpetraExt are both enabled, then TriBITS will automatically set EpetraExt_ENABLE_Triutils=ON by default and enables support for Triutils in EpetraExt. Likewise, if Teuchos and the optional TPL Boost are both enabled, then TriBITS will automatically set Teuchos_ENABLE_Boost=ON by default. This is obviously the logical behavior. See the full context for these examples in Explicit enable of a package, its tests, an optional TPL, with ST enabled.

14. Support for optional package can be explicitly disabled: Even though TriBITS will automatically set <TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE_OR_TPL>=ON by default if <TRIBITS_PACKAGE> and <TRIBITS_DEP_PACKAGE_OR_TPL> are both enabled at the project level (as described above), the user can explicitly set <TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE_OR_TPL>=OFF which will turn off optional package-level support for the upstream package or TPL <TRIBITS_DEP_PACKAGE_OR_TPL> in the downstream package <TRIBITS_PACKAGE>. For example, the user can enable EpetraExt and Triutils at the project level, but set EpetraExt_ENABLE_Triutils=OFF which will turn off package-level support for Triutils in the EpetraExt package. Likewise, the user can enable Teuchos, Epetra and the optional TPL Boost, but set Epetra_ENABLE_Boost=ON. This would provide support for Boost in Teuchos but not in Epetra. For examples, see Explicit disable of an optional package dependency and Explicit disable of an optional TPL dependency.

15. Explicit enable of optional package/TPL support auto-enables package/TPL: If the user explicitly enables the TriBITS package <TRIBITS_PACKAGE> and explicitly sets <TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE_OR_TPL>=ON on input, then that will automatically enable the package or TPL <TRIBITS_DEP_PACKAGE_OR_TPL> (and all of its upstream dependencies accordingly). For example, if the user sets Trilinos_ENABLE_EpetraExt=ON and EpetraExt_ENABLE_Triutils=ON, then that will result in the auto-enable of Trilinos_ENABLE_Triutils=ON regardless of the value of ${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES or ${PROJECT_NAME}_SECONDARY_TESTED_CODE. This true even if the optional package or TPL is EX. For example, setting Thyra_ENABLE_ThyraCrazyStuff=ON will result in the enabling of the EX package ThyraCrazyStuff. However, always remember that Disables trump enables where there is a conflict. For example, see Explicit enable of an optional package dependency.

16. ST packages only auto-enabled if ST code is enabled: TriBITS will only enable an optional ST package when ${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON if ${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON is also set. Otherwise, when an optional ST upstream dependent package is not enabled due to ${PROJECT_NAME}_SECONDARY_TESTED_CODE=OFF, then a one-line warning is printed to stdout. For example, if the EpetraExt package is enabled but ST code is not enabled, then the optional package Triutils will not be enabled (for an example, see Explicit enable of a package and its tests). However, when ${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON and EpetraExt is enabled, then Triutils will be enabled too (for an example, see Explicit enable of a package, its tests, an optional TPL, with ST enabled). The TriBITS default is ${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON. This helps to avoid problems when users try to set a permutation of enables/disables which is not regularly tested.

17. <Project>_ENABLE_ALL_FORWARD_DEP_PACKAGES enables downstream packages/tests: Setting the user cache-variable ${PROJECT_NAME}_ENABLE_ALL_FORWARD_PACKAGES=ON will result in the downstream PT internal packages and tests to be enabled (and all PT and ST packages and tests when ${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON) for all explicitly enabled internal packages. For example, in the mock Trilinos project, configuring with Trilinos_ENABLE_Epetra=ON, Trilinos_ENABLE_TESTS=ON, and Trilinos_ENABLE_ALL_FORWARD_PACKAGES=ON will result the package enables (and test and example enables) for the downstream packages Triutils, EpetraExt, ThyraCoreLibs, ThyraEpetra and Thyra. For an example, see Explicit enable of a package and downstream packages and tests. Note that when setting this option, the enable of an external package/TPL will not result in the auto-enable of downstream internal packages. For example, setting Trilinos_ENABLE_BLAS=ON will not result in the auto-enable of any internal packages that depend on BLAS like Teuchos (in the mock Trilinos project).

18. <Project>_ENABLE_ALL_PACKAGES enables all PT (cond. ST) packages: Setting the user cache-variable ${PROJECT_NAME}_ENABLE_ALL_PACKAGES=ON will result in the enable of all PT packages when ${PROJECT_NAME}_SECONDARY_TESTED_CODE=OFF and all PT and ST packages when ${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON. For an example, see Enable all packages. When the project is a meta-project, this will only enable the project's primary meta-project packages (PMPP). That is, a package will only be enabled due to ${PROJECT_NAME}_ENABLE_ALL_PACKAGES=ON when its parent repository does not have ${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES set to TRUE. However, if:

    ${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES=TRUE
    

    then the package may be enabled if it (or its parent package) is listed in ${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES_EXCEPT.

19. <Project>_ENABLE_TESTS only enables explicitly enabled package tests: Setting ${PROJECT_NAME}_ENABLE_TESTS=ON will only enable tests for explicitly enabled packages on input. For example, configuring with Trilinos_ENABLE_RTOp=ON and Trilinos_ENABLE_TESTS=ON will only result in the enable of tests for RTOp, not Teuchos (even through TriBITS will enable Teuchos because it is a required dependency of RTOp). See an example, see Explicit enable of a package and its tests. When the project is a meta-project, this will only enable tests for the project's primary meta-project packages (PMPP). The uses the same logic as for ${PROJECT_NAME}_ENABLE_ALL_PACKAGES, see <Project>_ENABLE_ALL_PACKAGES enables all PT (cond. ST) packages.

20. If no packages are enabled, nothing will get built: Most TriBITS projects are set up such that if the user does not explicitly enable at least one package in some way, then nothing will be enabled or built. In this case, when ${PROJECT_NAME}_ALLOW_NO_PACKAGES=TRUE a warning will be printed and configuration will complete. However, if ${PROJECT_NAME}_ALLOW_NO_PACKAGES=FALSE, then the configure will die with an error message. For example, the checkin-test.py tool sets ${PROJECT_NAME}_ALLOW_NO_PACKAGES=OFF to make sure that something gets enabled and tested in order to accept the results of the test and allow a push. For an example, see Default configure with no packages enabled on input.

21. TriBITS prints all enables and disables to stdout: TriBITS prints out (to cmake stdout) the initial set of enables/disables on input, prints a line whenever it sets (or overrides) an enable or disable, and prints out the final set of enables/disables. Therefore, the user just needs to grep the cmake stdout to find out why any particular package or TPL is enabled or disabled in the end. In addition, will print out when tests/examples for a given package gets enabled and when support for optional packages and TPLs is enabled or not. Examples of this output is given in all of the below examples but a detailed description of this output is given in Explicit enable of a package and its tests.

22. TriBITS auto-enables/disables done using non-cache local variables: TriBITS setting (or overrides) of enable/disable cache variables are done by setting local non-cache variables at the top project-level scope (i.e. the <projectDir>/CMakeLists.txt file scope) and does not touch the value of the cache variables that may be set by the user on input or the cache variables with documentation set by TriBITS. This is done so they don't get set in the cache and so that the same dependency enable/disable logic is redone, from scratch, with each re-configure. This results in the same enable/disable logic output as for the initial configure. This is to avoid confusion by the user about why some packages and TPLs are enabled and some are not on subsequent reconfigures. This is also desirable behavior as it preserves the user's input values for these variables to document what was set by the user. However, this implementation choice (and the tricky relationship between cache and non-cache CMake variables) must be clearly understood when one wants to go about tweaking these TriBITS enable/disable variables as described in How to check for and tweak TriBITS "ENABLE" cache variables and How to tweak downstream TriBITS "ENABLE" variables during package configuration.

TriBITS prints out a lot of information about the enable/disable logic as it applies the above rules/behaviors. For a large TriBITS project with lots of packages, this can produce a lot of output to stdout. One just needs to understand what TriBITS is printing out and where to look in the output for different information. The examples in the section Example Enable/Disable Use Cases show what this output looks like for the various enable/disable scenarios and tries to explain in more detail the reasons for why the given behavior is implemented the way that it is. Given this output, the rule definitions given above, and the detailed Example Enable/Disable Use Cases, one should always be able to figure out exactly why the final set of enables/disables is the way it is, even in the largest and most complex of TriBITS projects. (NOTE: The same can not be said for many other large software configuration and deployment systems where basic decisions about what to enable and disable are hidden from the user and can be very difficult to track down and debug.)

The above behaviors address the majority of the functionality of the TriBITS dependency management system. However, when dealing with TriBITS projects with multiple repositories, some other behaviors are supported through the definition of a few more variables. The following TriBITS repository-related variables alter what packages in a given TriBITS repository get enabled implicitly or not by TriBITS:

${REPOSITORY_NAME}_NO_IMPLICIT_PACKAGE_ENABLE

If set to ON, then the packages in Repository ${REPOSITORY_NAME} will not be implicitly enabled in any of the package adjustment logic.

${REPOSITORY_NAME}_NO_IMPLICIT_PACKAGE_ENABLE_EXCEPT

List of packages in the Repository ${REPOSITORY_NAME} that will be allowed to be implicitly enabled. Only checked if ${REPOSITORY_NAME}_NO_IMPLICIT_PACKAGE_ENABLE is true.

The above variables typically are defined in the outer TriBITS Project's CTest driver scripts or even in top-level project files in order to adjust how packages in its listed repositories are handled. What these variable do is to allow a large project to turn off the auto-enable of optional packages in a given TriBITS repository to provide more detailed control of what gets used from a given TriBITS repository. This, for example, is used in the CASL VERA project to manage some of its extra repositories and packages to further reduce the number of packages that get auto-enabled.

6.3   Example Enable/Disable Use Cases

Below, a few of the standard enable/disable use cases for a TriBITS project are given using the Example ReducedMockTrilinos Project Dependency Structure that demonstrate the TriBITS Dependency Handling Behaviors.

The use cases covered are:

• Default configure with no packages enabled on input
• Explicit enable of a package and its tests
• Explicit enable of a package, its tests, an optional TPL, with ST enabled
• Explicit disable of a package
• Conflicting explicit enable and disable
• Explicit enable of an optional TPL
• Explicit disable of an optional TPL
• Explicit disable of a required TPL
• Explicit enable of a subpackage
• Explicit enable of an optional package dependency
• Explicit disable of an optional package dependency
• Explicit enable of an optional TPL dependency
• Explicit disable of an optional TPL dependency
• Explicit enable of a package and downstream packages and tests
• Enable all packages

All of these use cases and more can be easily run from the command-line by first setting:

$ export REDUCED_MOCK_TRILINOS=<base-dir>/tribits/examples/ReducedMockTrilinos

and then copy and pasting the cmake commands shown below. Just make sure to run these in a temp directory because this actually configures a CMake project in the local directory. Just make sure and run:

$ rm -r CMake*

before each run to clear the CMake cache.

These use cases are now described in detail below.

Default configure with no packages enabled on input

The first use-case to consider is the configure of a TriBITS project without enabling any packages. For the ReducedMockTrilinos project, this is done with:

$ cmake ${REDUCED_MOCK_TRILINOS}

which produces the relevant dependency-related output:

Explicitly enabled top-level packages on input (by user):  0
Explicitly enabled packages on input (by user):  0
Explicitly disabled top-level packages on input (by user or by default):  0
Explicitly disabled packages on input (by user or by default):  ThyraCrazyStuff 1
Explicitly enabled external packages/TPLs on input (by user):  0
Explicitly disabled external packages/TPLs on input (by user or by default):  0

...

Final set of enabled top-level packages:  0
Final set of enabled packages:  0
Final set of non-enabled top-level packages:  Teuchos RTOp Epetra Triutils EpetraExt Thyra 6
Final set of non-enabled packages:  Teuchos RTOp Epetra Triutils EpetraExt \
 ThyraCoreLibs ThyraGoodStuff ThyraCrazyStuff ThyraEpetra ThyraEpetraExt Thyra 11
Final set of enabled external packages/TPLs:  0
Final set of non-enabled external packages/TPLs:  MPI BLAS LAPACK Boost UMFPACK AMD PETSC 7

...

***
*** WARNING:  There were no packages configured so no libraries or tests/examples will be built!
***

The above example demonstrates the following behaviors of the TriBITS dependency handling system:

• PT/ST packages given default unset enable/disable state
• EX packages disabled by default (i.e., the EX package ThyraCrazyStuff is set to OFF by default at the very beginning).
• PT/ST TPLs given default unset enable/disable state
• EX TPLs given default unset enable/disable state
• If no packages are enabled, nothing will get built

Explicit enable of a package and its tests

One of the most typical use cases is for the user to explicitly enable one or more top-level TriBITS package and enable its tests. This configuration would be used to drive local development on a specific set of packages (i.e. tests do not need to be enabled for packages not being changed).

Consider the configure of the ReducedMockTrilinos project enabling the top-level Thyra package and its tests with:

$ cmake -DTrilinos_ENABLE_Thyra:BOOL=ON \
   -DTrilinos_ENABLE_TESTS:BOOL=ON \
   ${REDUCED_MOCK_TRILINOS}

which produces the relevant dependency-related output:

Explicitly enabled top-level packages on input (by user):  Thyra 1
Explicitly enabled packages on input (by user):  Thyra 1
Explicitly disabled top-level packages on input (by user or by default):  0
Explicitly disabled packages on input (by user or by default):  ThyraCrazyStuff 1
Explicitly enabled external packages/TPLs on input (by user):  0
Explicitly disabled external packages/TPLs on input (by user or by default):  0

Enabling subpackages for hard enables of parent packages due to \
 Trilinos_ENABLE_<PARENT_PACKAGE>=ON ...

-- Setting subpackage enable Trilinos_ENABLE_ThyraCoreLibs=ON because parent \
 package Trilinos_ENABLE_Thyra=ON
-- Setting subpackage enable Trilinos_ENABLE_ThyraEpetra=ON because parent package \
 Trilinos_ENABLE_Thyra=ON

Disabling forward required packages and optional intra-package support that have a \
 dependency on disabled packages Trilinos_ENABLE_<TRIBITS_PACKAGE>=OFF ...

-- Setting Thyra_ENABLE_ThyraCrazyStuff=OFF because Thyra has an optional library \
 dependence on disabled package ThyraCrazyStuff

Enabling all tests and/or examples that have not been explicitly disabled because \
 Trilinos_ENABLE_[TESTS,EXAMPLES]=ON ...

-- Setting ThyraCoreLibs_ENABLE_TESTS=ON
-- Setting ThyraCoreLibs_ENABLE_EXAMPLES=ON
-- Setting ThyraEpetra_ENABLE_TESTS=ON
-- Setting ThyraEpetra_ENABLE_EXAMPLES=ON
-- Setting Thyra_ENABLE_TESTS=ON
-- Setting Thyra_ENABLE_EXAMPLES=ON

Enabling all required (and optional since Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES=ON) \
 upstream packages for current set of enabled packages \
 (Trilinos_ENABLE_SECONDARY_TESTED_CODE=OFF) ...

-- NOTE: Not Setting Trilinos_ENABLE_ThyraGoodStuff=ON even though Thyra \
 has an optional dependence on ThyraGoodStuff because Trilinos_ENABLE_SECONDARY_TESTED_CODE=OFF
-- NOTE: Not Setting Trilinos_ENABLE_ThyraEpetraExt=ON even though Thyra \
 has an optional dependence on ThyraEpetraExt because Trilinos_ENABLE_SECONDARY_TESTED_CODE=OFF
-- Setting Trilinos_ENABLE_Epetra=ON because ThyraEpetra has a required dependence on Epetra
-- Setting Trilinos_ENABLE_Teuchos=ON because ThyraCoreLibs has a required dependence on Teuchos
-- Setting Trilinos_ENABLE_RTOp=ON because ThyraCoreLibs has a required dependence on RTOp
-- Setting TPL_ENABLE_BLAS=ON because Epetra has a required dependence on BLAS
-- Setting TPL_ENABLE_LAPACK=ON because Epetra has a required dependence on LAPACK

Enabling all optional intra-package enables <TRIBITS_PACKAGE>_ENABLE_<DEPPACKAGE> that are 
 not currently disabled if both sets of packages are enabled ...

-- Setting Thyra_ENABLE_ThyraEpetra=ON since Trilinos_ENABLE_Thyra=ON AND Trilinos_ENABLE_ThyraEpetra=ON

Final set of enabled top-level packages:  Teuchos RTOp Epetra Thyra 4
Final set of enabled packages:  Teuchos RTOp Epetra ThyraCoreLibs ThyraEpetra Thyra 6
Final set of non-enabled top-level packages:  Triutils EpetraExt 2
Final set of non-enabled packages:  Triutils EpetraExt ThyraGoodStuff ThyraCrazyStuff ThyraEpetraExt 5
Final set of enabled external packages/TPLs:  BLAS LAPACK 2
Final set of non-enabled external packages/TPLs:  MPI Boost UMFPACK AMD PETSC 5

Getting information for all enabled external packages/TPLs ...

Processing enabled external package/TPL: BLAS
Processing enabled external package/TPL: LAPACK

Configuring individual enabled Trilinos packages ...

Processing enabled top-level package: Teuchos (Libs)
Processing enabled top-level package: RTOp (Libs)
Processing enabled top-level package: Epetra (Libs)
Processing enabled top-level package: Thyra (CoreLibs, Epetra, Tests, Examples)

This is a configuration that a developer would use to develop on the Thyra package and its subpackages for example. There is no need to be enabling the tests and examples for upstream packages unless those packages are going to be changed as well.

This case demonstrates a number of TriBITS dependency handling behaviors that are worth some discussion.

First, note that enabling the parent package Thyra with Trilinos_ENABLE_Thyra=ON right away results in the auto-enable of its PT subpackages ThyraCoreLibs and ThyraEpetra which demonstrates the behavior Enable/disable of parent package is enable/disable for subpackages. Note that the ST subpackages ThyraGoodStuff and ThyraEpetraExt where not enabled because ${PROJECT_NAME}_SECONDARY_TESTED_CODE=OFF (which is off by default) which demonstrates the behavior ST packages only auto-enabled if ST code is enabled.

Second, note the auto-enable of required upstream packages Epetra, RTOp and Teuchos shown in lines like:

-- Setting Trilinos_ENABLE_Teuchos=ON because ThyraCoreLibs has a required dependence on Teuchos

Lastly, note that the final set of enabled packages, packages, tests/examples and external packages/TPLs can be clearly seen when processing the external packages/TPLs and top-level packages in the lines:

Getting information for all enabled external packages/TPLs ...

-- Processing enabled external package/TPL: BLAS
-- Processing enabled external package/TPL: LAPACK

Configuring individual enabled Trilinos packages ...

Processing enabled top-level package: Teuchos (Libs)
Processing enabled top-level package: RTOp (Libs)
Processing enabled top-level package: Epetra (Libs)
Processing enabled top-level package: Thyra (CoreLibs, Epetra, Tests, Examples)

Note that subpackage enables are listed with their parent packages along with if the tests and/or examples are enabled. Top-level packages that don't have subpackages just show Libs and Tests and Examples if they have been enabled as well.

Explicit enable of a package, its tests, an optional TPL, with ST enabled

An extended use case shown here is for the explicit enable of a package and its tests along with the enable of an optional TPL and with ST code enabled. This is a configuration that would be used to support the local development of a TriBITS package that involves modifying ST software.

Consider the configure of the ReducedMockTrilinos project with:

$ cmake -DTPL_ENABLE_Boost:BOOL=ON \
   -DTrilinos_ENABLE_Thyra:BOOL=ON \
   -DTrilinos_ENABLE_TESTS:BOOL=ON \
   -DTrilinos_ENABLE_SECONDARY_TESTED_CODE:BOOL=ON \
   -DTrilinos_ENABLE_TESTS:BOOL=ON \
   ${REDUCED_MOCK_TRILINOS}

which produces the relevant dependency-related output:

Explicitly enabled top-level packages on input (by user):  Thyra 1
Explicitly enabled packages on input (by user):  Thyra 1
Explicitly disabled top-level packages on input (by user or by default):  0
Explicitly disabled packages on input (by user or by default):  ThyraCrazyStuff 1
Explicitly enabled external packages/TPLs on input (by user):  Boost 1
Explicitly disabled external packages/TPLs on input (by user or by default):  0

Enabling subpackages for hard enables of parent packages due to \
 Trilinos_ENABLE_<PARENT_PACKAGE>=ON ...

-- Setting subpackage enable Trilinos_ENABLE_ThyraCoreLibs=ON because parent package \
 Trilinos_ENABLE_Thyra=ON
-- Setting subpackage enable Trilinos_ENABLE_ThyraGoodStuff=ON because parent package \
 Trilinos_ENABLE_Thyra=ON
-- Setting subpackage enable Trilinos_ENABLE_ThyraEpetra=ON because parent package \
 Trilinos_ENABLE_Thyra=ON
-- Setting subpackage enable Trilinos_ENABLE_ThyraEpetraExt=ON because parent package \
 Trilinos_ENABLE_Thyra=ON

Disabling forward required packages and optional intra-package support that have a \
 dependency on disabled packages Trilinos_ENABLE_<TRIBITS_PACKAGE>=OFF ...

-- Setting Thyra_ENABLE_ThyraCrazyStuff=OFF because Thyra has an optional library \
 dependence on disabled package ThyraCrazyStuff

Enabling all tests and/or examples that have not been explicitly disabled because \
 Trilinos_ENABLE_[TESTS,EXAMPLES]=ON ...

-- Setting ThyraCoreLibs_ENABLE_TESTS=ON
-- Setting ThyraGoodStuff_ENABLE_TESTS=ON
-- Setting ThyraEpetra_ENABLE_TESTS=ON
-- Setting ThyraEpetraExt_ENABLE_TESTS=ON
-- Setting Thyra_ENABLE_TESTS=ON

Enabling all required (and optional since Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES=ON) \
 upstream packages for current set of enabled packages ...

-- Setting Trilinos_ENABLE_EpetraExt=ON because ThyraEpetraExt has a required dependence on EpetraExt
-- Setting Trilinos_ENABLE_Epetra=ON because ThyraEpetra has a required dependence on Epetra
-- Setting Trilinos_ENABLE_Teuchos=ON because ThyraCoreLibs has a required dependence on Teuchos
-- Setting Trilinos_ENABLE_RTOp=ON because ThyraCoreLibs has a required dependence on RTOp
-- Setting Trilinos_ENABLE_Triutils=ON because EpetraExt has an optional dependence on Triutils
-- Setting TPL_ENABLE_BLAS=ON because Epetra has a required dependence on BLAS
-- Setting TPL_ENABLE_LAPACK=ON because Epetra has a required dependence on LAPACK

Enabling all optional intra-package enables <TRIBITS_PACKAGE>_ENABLE_<DEPPACKAGE> that are \
 not currently disabled if both sets of packages are enabled ...

-- Setting Teuchos_ENABLE_BLAS=ON since Trilinos_ENABLE_Teuchos=ON AND TPL_ENABLE_BLAS=ON
-- Setting Teuchos_ENABLE_LAPACK=ON since Trilinos_ENABLE_Teuchos=ON AND TPL_ENABLE_LAPACK=ON
-- Setting Teuchos_ENABLE_Boost=ON since Trilinos_ENABLE_Teuchos=ON AND TPL_ENABLE_Boost=ON
-- NOT setting Teuchos_ENABLE_MPI=ON since MPI is NOT enabled at this point!
-- Setting RTOp_ENABLE_Teuchos=ON since Trilinos_ENABLE_RTOp=ON AND Trilinos_ENABLE_Teuchos=ON
-- Setting Epetra_ENABLE_BLAS=ON since Trilinos_ENABLE_Epetra=ON AND TPL_ENABLE_BLAS=ON
-- Setting Epetra_ENABLE_LAPACK=ON since Trilinos_ENABLE_Epetra=ON AND TPL_ENABLE_LAPACK=ON
-- NOT setting Epetra_ENABLE_MPI=ON since MPI is NOT enabled at this point!
-- Setting Triutils_ENABLE_Epetra=ON since Trilinos_ENABLE_Triutils=ON AND Trilinos_ENABLE_Epetra=ON
-- Setting EpetraExt_ENABLE_Teuchos=ON since Trilinos_ENABLE_EpetraExt=ON AND Trilinos_ENABLE_Teuchos=ON
-- Setting EpetraExt_ENABLE_Epetra=ON since Trilinos_ENABLE_EpetraExt=ON AND Trilinos_ENABLE_Epetra=ON
-- Setting EpetraExt_ENABLE_Triutils=ON since Trilinos_ENABLE_EpetraExt=ON AND Trilinos_ENABLE_Triutils=ON
-- NOT setting EpetraExt_ENABLE_UMFPACK=ON since UMFPACK is NOT enabled at this point!
-- NOT setting EpetraExt_ENABLE_AMD=ON since AMD is NOT enabled at this point!
-- NOT setting EpetraExt_ENABLE_PETSC=ON since PETSC is NOT enabled at this point!
-- Setting ThyraCoreLibs_ENABLE_Teuchos=ON since Trilinos_ENABLE_ThyraCoreLibs=ON AND Trilinos_ENABLE_Teuchos=ON
-- Setting ThyraCoreLibs_ENABLE_RTOp=ON since Trilinos_ENABLE_ThyraCoreLibs=ON AND Trilinos_ENABLE_RTOp=ON
-- Setting ThyraGoodStuff_ENABLE_ThyraCoreLibs=ON since Trilinos_ENABLE_ThyraGoodStuff=ON AND Trilinos_ENABLE_ThyraCoreLibs=ON
-- Setting ThyraEpetra_ENABLE_Epetra=ON since Trilinos_ENABLE_ThyraEpetra=ON AND Trilinos_ENABLE_Epetra=ON
-- Setting ThyraEpetra_ENABLE_ThyraCoreLibs=ON since Trilinos_ENABLE_ThyraEpetra=ON AND Trilinos_ENABLE_ThyraCoreLibs=ON
-- Setting ThyraEpetraExt_ENABLE_ThyraEpetra=ON since Trilinos_ENABLE_ThyraEpetraExt=ON AND Trilinos_ENABLE_ThyraEpetra=ON
-- Setting ThyraEpetraExt_ENABLE_EpetraExt=ON since Trilinos_ENABLE_ThyraEpetraExt=ON AND Trilinos_ENABLE_EpetraExt=ON
-- Setting Thyra_ENABLE_ThyraCoreLibs=ON since Trilinos_ENABLE_Thyra=ON AND Trilinos_ENABLE_ThyraCoreLibs=ON
-- Setting Thyra_ENABLE_ThyraGoodStuff=ON since Trilinos_ENABLE_Thyra=ON AND Trilinos_ENABLE_ThyraGoodStuff=ON
-- Setting Thyra_ENABLE_ThyraEpetra=ON since Trilinos_ENABLE_Thyra=ON AND Trilinos_ENABLE_ThyraEpetra=ON
-- Setting Thyra_ENABLE_ThyraEpetraExt=ON since Trilinos_ENABLE_Thyra=ON AND Trilinos_ENABLE_ThyraEpetraExt=ON

Final set of enabled top-level packages:  Teuchos RTOp Epetra Triutils EpetraExt Thyra 6
Final set of enabled packages:  Teuchos RTOp Epetra Triutils EpetraExt ThyraCoreLibs \
 ThyraGoodStuff ThyraEpetra ThyraEpetraExt Thyra 10
Final set of non-enabled top-level packages:  0
Final set of non-enabled packages:  ThyraCrazyStuff 1
Final set of enabled external packages/TPLs:  BLAS LAPACK Boost 3
Final set of non-enabled external packages/TPLs:  MPI UMFPACK AMD PETSC 4

Getting information for all enabled external packages/TPLs ...

Processing enabled external package/TPL: BLAS
Processing enabled external package/TPL: LAPACK
Processing enabled external package/TPL: Boost

Configuring individual enabled Trilinos packages ...

Processing enabled top-level package: Teuchos (Libs)
Processing enabled top-level package: RTOp (Libs)
Processing enabled top-level package: Epetra (Libs)
Processing enabled top-level package: Triutils (Libs)
Processing enabled top-level package: EpetraExt (Libs)
Processing enabled top-level package: Thyra (CoreLibs, GoodStuff, Epetra, EpetraExt, Tests, Examples)

A few more behaviors of the TriBITS system that this particular configuration use-case shows are described below.

First, note the enable of the ST Thyra subpackages in lines like:

-- Setting subpackage enable Trilinos_ENABLE_ThyraGoodStuff=ON because parent package \
 Trilinos_ENABLE_Thyra=ON

Second, note the auto-enable of support for optional packages in lines like:

-- Setting EpetraExt_ENABLE_Triutils=ON since Trilinos_ENABLE_EpetraExt=ON \
 AND Trilinos_ENABLE_Triutils=ON

Third, note the auto-enable of support for the optional TPL Boost in the line:

-- Setting Teuchos_ENABLE_Boost=ON since TPL_ENABLE_Boost=ON

Explicit disable of a package

Another common use case is to enable a package but to disable an optional upstream package. This type of configuration would be used as part of a "black list" approach to enabling only a subset of packages and optional support. The "black list" approach is to enable a package with ${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON (the TriBITS default) but then to turn off a specific set of packages that you don't want. This is contrasted with a "white list" approach where you would configure with ${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=OFF and then have to manually enable all of the optional packages you want. Experience with projects like Trilinos show that the "black list" approach is generally to be preferred for a few reasons.

Consider the configure of the ReducedMockTrilinos project enabling Thyra but disabling Epetra with:

$ cmake  -DTrilinos_ENABLE_Thyra:BOOL=ON \
   -DTrilinos_ENABLE_Epetra:BOOL=OFF \
   -DTrilinos_ENABLE_TESTS:BOOL=ON \
   ${REDUCED_MOCK_TRILINOS}

which produces the relevant dependency-related output:

Disabling forward required packages and optional intra-package support that have a \
 dependency on disabled packages Trilinos_ENABLE_<TRIBITS_PACKAGE>=OFF ...

-- Setting Trilinos_ENABLE_Triutils=OFF because Triutils has a required library dependence \
 on disabled package Epetra
-- Setting Trilinos_ENABLE_EpetraExt=OFF because EpetraExt has a required library \
 disabled package Epetra
-- Setting Trilinos_ENABLE_ThyraEpetra=OFF because ThyraEpetra has a required library \
 disabled package Epetra
-- Setting Trilinos_ENABLE_ThyraEpetraExt=OFF because ThyraEpetraExt has a required library \
 disabled package EpetraExt
-- Setting Thyra_ENABLE_ThyraCrazyStuff=OFF because Thyra has an optional library \
 disabled package ThyraCrazyStuff
-- Setting Thyra_ENABLE_ThyraEpetra=OFF because Thyra has an optional library \
 disabled package ThyraEpetra
-- Setting Thyra_ENABLE_ThyraEpetraExt=OFF because Thyra has an optional library \
 disabled package ThyraEpetraExt

Final set of enabled top-level packages:  Teuchos RTOp Thyra 3
Final set of enabled packages:  Teuchos RTOp ThyraCoreLibs Thyra 4
Final set of non-enabled top-level packages:  Epetra Triutils EpetraExt 3
Final set of non-enabled packages:  Epetra Triutils EpetraExt ThyraGoodStuff \
 ThyraCrazyStuff ThyraEpetra ThyraEpetraExt 7
Final set of enabled external packages/TPLs:  BLAS LAPACK 2
Final set of non-enabled external packages/TPLs:  MPI Boost UMFPACK AMD PETSC 5

Configuring individual enabled Trilinos packages ...

Processing enabled top-level package: Teuchos (Libs)
Processing enabled top-level package: RTOp (Libs)
Processing enabled top-level package: Thyra (CoreLibs, Tests, Examples)

Note how the disable of Epetra wipes out all of the required and optional packages and intra-package dependencies that depend on Epetra. What is left is only the ThyraCoreLibs and its upstream dependencies that don't depend on Epetra (which is only RTOp and Teuchos).

Conflicting explicit enable and disable

One use case that occasionally comes up is when a set of inconsistent enables and disables are set. While this seems illogical that anyone would ever do this, when it comes to larger more complex projects with lots of packages and lots of dependencies, this can happen very easily. In some cases, someone is enabling a set of packages they want and is trying to weed out as many of the (what they think) are optional dependencies that they don't need and accidentally disables a package that is an indirect required dependency of one of the packages they want (in which case the configure should likely fail and provide a good error message). The other use case where conflicting enables/disables can occur is in CTest drivers using tribits_ctest_driver() where an upstream package has failed and is explicitly disabled (in which case it should gracefully disable downstream dependent packages). TriBITS can either be set up to have the disable override the explicit enable or stop the configure in error depending on the value of the cache variable ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES (see Disables trump enables where there is a conflict).

For example, consider what happens with the ReducedMockTrilinos project if someone tries to enable the RTOp package and disable the Teuchos package. This is not consistent because RTOp has a required dependency on Teuchos. The default behavior of TriBITS is this case is shown in the below configure:

$ cmake -DTrilinos_ENABLE_Epetra:BOOL=ON \
   -DTrilinos_ENABLE_RTOp:BOOL=ON \
   -DTrilinos_ENABLE_Teuchos:BOOL=OFF \
   ${REDUCED_MOCK_TRILINOS}

which produces the relevant dependency-related output:

Explicitly enabled top-level packages on input (by user):  RTOp Epetra 2
Explicitly disabled top-level packages on input (by user or by default):  Teuchos 1

Disabling forward required packages and optional intra-package support that have \
 a dependency on disabled packages Trilinos_ENABLE_<TRIBITS_PACKAGE>=OFF ...

   ***
   *** ERROR: Setting Trilinos_ENABLE_RTOp=OFF which was 'ON' because RTOp has a required \
 library dependence on disabled package Teuchos!
   ***

As shown above, the TriBITS default (which is ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=OFF) results in a configure-time error with a good error message.

However, if one sets ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON and configures with:

$ cmake -DTrilinos_ENABLE_Epetra:BOOL=ON \
   -DTrilinos_ENABLE_RTOp:BOOL=ON \
   -DTrilinos_ENABLE_Teuchos:BOOL=OFF \
   -DTrilinos_DISABLE_ENABLED_FORWARD_DEP_PACKAGES:BOOL=ON \
   ${REDUCED_MOCK_TRILINOS}

then the disable trumps the enable and results in a successful configure as shown in the following relevant dependency-related output:

Explicitly enabled top-level packages on input (by user):  RTOp Epetra 2
Explicitly disabled top-level packages on input (by user or by default):  Teuchos 1

Disabling forward required packages and optional intra-package support that \
 have a dependency on disabled packages Trilinos_ENABLE_<TRIBITS_PACKAGE>=OFF ...

 ***
 *** NOTE: Setting Trilinos_ENABLE_RTOp=OFF which was 'ON' because RTOp has \
 a required library dependence on disabled package Teuchos but \
 Trilinos_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON!
 ***

Final set of enabled top-level packages:  Epetra 1
Final set of non-enabled top-level packages:  Teuchos RTOp Triutils EpetraExt Thyra 5

As shown above, what you end up with is just the enabled package Epetra which does not have a required dependency on the disabled package Teuchos. Developers of large complex TriBITS projects would be wise to set the default for ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES to ON, especially in automated builds and testing.

Explicit enable of an optional TPL:

ToDo: Set Trilinos_ENABLE_Thyra=ON and TPL_ENABLE_MPI=ON

Explicit disable of an optional TPL:

ToDo: Set Trilinos_ENABLE_Thyra=ON and TPL_ENABLE_MPI=OFF

Explicit disable of a required TPL

ToDo: Set Trilinos_ENABLE_Epetra=ON and Trilinos_ENABLE_BLAS=OFF

Explicit enable of a subpackage

ToDo: Enable ThyraEpetra and show how it enables other packages and at the end, enables the Thyra package (just for show).

Explicit enable of an optional package dependency

ToDo: Set Trilinos_ENABLE_EpetraExt=ON and EpetraExt_ENABLE_Triutils=ON and shows how it enables Trilinos_ENABLE_Triutils=ON even through ST code is not enabled.

Explicit disable of an optional package dependency

ToDo: Set Trilinos_ENABLE_EpetraExt=ON, Trilinos_ENABLE_Triutils=ON, and EpetraExt_ENABLE_Triutils=OFF. Discuss how EpetraExt's and ThyraEpetraExt's CMakeLists.txt files might turn off some features if they detects that EpetraExt/Triutils support is turned off.

Explicit enable of an optional TPL dependency

ToDo: The current ReducedMockTrilinos is not set up to give a good example of this. We should add an optional Boost dependency to say, Epetra. Then we could show the enable of Teuchos and Epetra and Epetra_ENABLE_Boost=ON. That would enable Boost and enable support for Boost in Epetra but would not provide support for Boost in Teuchos.

Explicit disable of an optional TPL dependency

ToDo: The current ReducedMockTrilinos is not set up to give a good example of this. We should add an optional Boost dependency to say, Epetra. Then we could show the enable of Teuchos and Epetra and TPL_ENABLE_Boost=ON but set Epetra_ENABLE_Boost=OFF. That would provide support for Boost in Teuchos but not in Epetra.

Explicit enable of a package and downstream packages and tests

ToDo: Set Trilinos_ENABLE_RTOp=ON, Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES=ON, and Trilinos_ENABLE_TESTS=ON and show what packages and tests/examples get enabled. This is the use case for the checkin-test.py tool for PT enabled code.

Enable all packages

The last use case to consider is enabling all defined packages. This configuration would be used for either doing a full test of all of the packages defined or to create a distribution of the project.

Enabling all PT packages the with:

$ cmake -DTrilinos_ENABLE_ALL_PACKAGES:BOOL=ON \
   -DTrilinos_DUMP_PACKAGE_DEPENDENCIES:BOOL=ON \
   ${REDUCED_MOCK_TRILINOS}

produces the relevant dependency-related output:

Enabling all packages that are not currently disabled because of \
 Trilinos_ENABLE_ALL_PACKAGES=ON (Trilinos_ENABLE_SECONDARY_TESTED_CODE=OFF) ...

-- Setting Trilinos_ENABLE_Teuchos=ON
-- Setting Trilinos_ENABLE_RTOp=ON
-- Setting Trilinos_ENABLE_Epetra=ON
-- Setting Trilinos_ENABLE_ThyraCoreLibs=ON
-- Setting Trilinos_ENABLE_ThyraEpetra=ON
-- Setting Trilinos_ENABLE_Thyra=ON

Enabling all required (and optional since Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES=ON) upstream packages for current set of enabled packages
(Trilinos_ENABLE_SECONDARY_TESTED_CODE=OFF) ...

-- NOTE: Not Setting Trilinos_ENABLE_ThyraGoodStuff=ON even though Thyra has an optional dependence on ThyraGoodStuff because Trilinos_ENABLE_SECONDARY_TESTED_CODE=OFF
-- NOTE: Not Setting Trilinos_ENABLE_ThyraEpetraExt=ON even though Thyra has an optional dependence on ThyraEpetraExt because Trilinos_ENABLE_SECONDARY_TESTED_CODE=OFF
-- Setting TPL_ENABLE_BLAS=ON because Epetra has a required dependence on BLAS
-- Setting TPL_ENABLE_LAPACK=ON because Epetra has a required dependence on LAPACK

Final set of enabled top-level packages:  Teuchos RTOp Epetra Thyra 4
Final set of enabled packages:  Teuchos RTOp Epetra ThyraCoreLibs ThyraEpetra Thyra 6
Final set of non-enabled top-level packages:  Triutils EpetraExt 2
Final set of non-enabled packages:  Triutils EpetraExt ThyraGoodStuff ThyraCrazyStuff ThyraEpetraExt 5
Final set of enabled external packages/TPLs:  BLAS LAPACK 2
Final set of non-enabled external packages/TPLs:  MPI Boost UMFPACK AMD PETSC 5

Getting information for all enabled external packages/TPLs ...

Processing enabled external package/TPL: BLAS
Processing enabled external package/TPL: LAPACK

Configuring individual enabled Trilinos packages ...

Processing enabled top-level package: Teuchos (Libs)
Processing enabled top-level package: RTOp (Libs)
Processing enabled top-level package: Epetra (Libs)
Processing enabled top-level package: Thyra (CoreLibs, Epetra)

As shown above, only the PT packages get enabled. To also enable the ST packages as well, one additionally set ${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON at configure time.

6.4   TriBITS Project Dependencies XML file and tools

The TriBITS CMake configure system can write out the project's package dependencies into a file <Project>Dependencies.xml (or any name one wants to give it). This file is used by a number of the TriBITS SE-related tools. The structure of this XML file, showing one of the more interesting mock packages from the MockTrilinos project is shown below:

<PackageDependencies project="Trilinos">
  ...
  <Package name="Amesos" dir="packages/amesos" type="PT">
    <LIB_REQUIRED_DEP_PACKAGES value="Teuchos,Epetra"/>
    <LIB_OPTIONAL_DEP_PACKAGES value="EpetraExt"/>
    <TEST_REQUIRED_DEP_PACKAGES/>
    <TEST_OPTIONAL_DEP_PACKAGES value="Triutils,Galeri"/>
    <LIB_REQUIRED_DEP_TPLS/>
    <LIB_OPTIONAL_DEP_TPLS value="SuperLUDist,ParMETIS,UMFPACK,SuperLU,MUMPS"/>
    <TEST_REQUIRED_DEP_TPLS/>
    <TEST_OPTIONAL_DEP_TPLS/>
    <EmailAddresses>
      <Regression address="amesos-regression@repo.site.gov"/>
    </EmailAddresses>
    <ParentPackage value=""/>
  </Package>
  ...
</PackageDependencies>

This XML file contains the names, directories, Test Test Category (i.e. type), CDash email address, and all of the package and TPL dependencies for every package in the TriBITS project (including add-on repositories if specified). There are several python tools under tribits/ci_support/ that read in this file and use the created data-structure for various tasks. This file and these tools are used by checkin-test.py and tribits_ctest_driver(). But these tools can also be used to construct other workflows and tools. These tools require a Python3 installation and for the python3 executable to be installed.

A TriBITS project configure can create this file as a byproduct of configuration by setting the configure option (see Outputting package dependency information), or the CMake -P script TribitsDumpDepsXmlScript.cmake can be used to create this file on the fly without having to configure a TriBITS project. To create this file outside of configuration, one can run:

cmake \
  [-D PROJECT_SOURCE_DIR=<projectSourceDir>] \
  [-D <Project>_PRE_REPOSITORIES=<prepo0>,<prepo1>,...] \
  [-D <Project>_EXTRA_REPOSITORIES=<erepo0>,<erepo1>,...] \
  -D <Project>_DEPS_XML_OUTPUT_FILE=<projectDepsFileOut> \
  -P <tribitsDir>/ci_support/TribitsDumpDepsXmlScript.cmake

If TriBITS is snashotted into the project in the standard location <projectDir>/cmake/tribits or the entire TriBITS repo is cloned under <projectDir>/TriBITS (so that the tribits dir is <projectDir>/TriBITS/tribits) then one can leave off -DPROJECT_SOURCE_DIR=<projectSourceDir> and (if not wanting to include extra repos) just run:

cmake \
  -D <Project>_DEPS_XML_OUTPUT_FILE=<projectDepsFileOut> \
  -P <projectSourceDir>/cmake/tribits/ci_support/TribitsDumpDepsXmlScript.cmake

Once the XML file <projectDepsFileOut> is created, it can be used in various types of analysis and used with different tools and commands.

The tool get-tribits-packages-from-files-list.py can be used to determine the list of TriBITS packages that need to be tested given a list of changed files (e.g. as returned from git diff --name-only <from>..<to> > changed-files.txt). This is used in the checkin-test.py tool and the tribits_ctest_driver() function to determine what TriBITS packages need to be tested based on what files have been changed.

The tool get-tribits-packages-from-last-tests-failed.py can be used to extract the list of TriBITS packages that correspond to the failings tests listed in the CTest-generated <build-dir>/Testing/Temporary/LastTestsFailed*.log file. This tool is used in the tribits_ctest_driver() function in CI-testing mode to determine what packages must be re-tested if they failed in the last CI iteration.

The tool filter-packages-list.py takes in a list of TriBITS package names and then filters the list according the Test Test Category of the packages. This is used in testing workflows that only test a subset of packages according to the Test Test Category at different stages in the workflow. For example, the checkin-test.py tool and the tribits_ctest_driver() function use this filtering to only test Primary Tested (PT) or Secondary Tested (ST) packages for a given set of changed files in a continuous integration workflow (see Nested Layers of TriBITS Project Testing).

7.1   Test Classifications for Repositories, Packages, and Tests

TriBITS defines a few different testing-related classifications for a TriBITS project. These different classifications are used to select subsets of the project's repositories, packages (and code within these packages), and tests to be included in a given project build and test definition. These different classification are:

• Repository Test Classification
• Package Test Group
• Test Test Category

These different test-related classifications are used to defined several different Nested Layers of TriBITS Project Testing. First, the Repository Test Classification determines what repositories are even processed in order for their packages to even consider being enabled. Second, if a repository is selected, then the Package Test Group determines what packages (and optional code in those packages) are even enabled such that their <packageDir>/CMakeLists.txt files are even processed (i.e. according to TriBITS Dependency Handling Behaviors). Lastly, if an package gets enabled, then the Test Test Category determines what test executables and test cases get defined using the functions tribits_add_executable(), tribits_add_test() and tribits_add_advanced_test().

More detailed descriptions of Repository Test Classifications, Package Test Groups, and Test Test Categories are given in the following subsections.

Repository Test Classification

The first type of test-related classification is for extra repositories defined in the file <projectDir>/cmake/ExtraRepositoriesList.cmake (pulled in through the ${PROJECT_NAME}_EXTRAREPOS_FILE cache variable) using the REPO_CLASSIFICATION field in the macro call tribits_project_define_extra_repositories(). These classifications map to the standard CTest dashboard types Continuous, Nightly, and Experimental (see CTest documentation and TriBITS CTest/CDash Driver for details).

• Repositories marked Continuous match the standard CTest dashboard type Continuous. These repositories are pulled in when ${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE=Continuous, or Nightly. Repositories marked as Continuous are cloned, updated, and processed by default in all project automated testing described in Nested Layers of TriBITS Project Testing. NOTE: One should not confuse this with the Test Test Category CONTINUOUS.

• Repositories marked Nightly match the standard CTest dashboard type Nightly. These repositories are pulled in by default when ${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE=Nightly. Repositories marked as Nightly are not processed by default as part of either Pre-Push CI Testing or Post-Push CI Testing. One would mark a repository as Nightly for few reasons. First, an extra repo may be marked as Nightly if it may not be available to all developers to clone and only the nightly testing processes and machines may have access. Also, an extra repo may also be marked as Nightly if it does not contain any packages that the project wants to pay the cost to include in even Post-Push CI Testing. NOTE: One should not confuse this with the Test Test Category NIGHTLY.

• Repositories marked Experimental match the standard CTest dashboard type Experimental. Repositories marked as Experimental are not processed by default as part of any automated testing described in the subsection Nested Layers of TriBITS Project Testing (except for perhaps some experimental builds). The main reason that an extra repo may be marked as Experimental is that it may only contain EX packages and therefore none of these packages would be enabled by default anyway. Also, a repo may be marked as Experimental if it is developed in a very sloppy way such that one cannot even assume that the repository's <repoDir>/PackagesList.cmake, <repoDir>/TPlsList.cmake, and <packageDir>/cmake/Dependencies.cmake files are not without errors. Since these files are always processed if the repository is included, then any errors in these files will cause the entire configure to fail!

Package Test Group

Once a set of TriBITS repositories are selected in accordance with their Repository Test Classification, that determines the set of packages defined for the TriBITS project. Given the set of defined packages, the set of packages that get enabled is determined by the Package Test Group which is defined and described here.

Every TriBITS Package is assigned a test group. These test groups are for Primary Tested (PT) code, Secondary Tested (ST) code, and Experimental (EX) code. The test group defines what package get selected (or are excluded from being selected) to include in a given build for testing-related purposes. packages may also conditionally build in additional code based on the testing group. The detailed rules for when an package is selected or excluded from the build based on the test group is given in TriBITS Dependency Handling Behaviors. We only summarize those rules here.

More detailed descriptions of the test groups are given below.

• Primary Tested (PT) Code is of the highest priority to keep working for the current development effort. Packages may be selected to be PT for a number of reasons. First, if the capability provided by the code is mature and if a regression would cause major harm to a customer, the code should likely be marked as PT. Also, if the build and correct functioning of the code is needed by other development team members to support their day-to-day development activities, then the code should be marked as PT as well. An external package/TPL, on the other hand, is marked as PT if it is required by a downstream PT package. Every project developer is expected to have every PT TPL installed on every machine where they do development on and from which they push to the global repo (see checkin-test.py tool). PT packages are the foundation for Pre-Push CI Testing.

• Secondary Tested (ST) Code is still very important code for the project and represents important capability to maintain but is excluded from the PT set of code for one of a few different reasons. First, code may be marked as ST if is not critical to drive most day-to-day development activities. If ST code breaks, it usually will not cause immediate and major harm to most developers. Also, code may be marked as ST if it has required dependencies on ST external packages/TPLs which are either hard to install or may not be available on all platforms where developers do their development and from where they push changes to the global repo. In addition, code may be marked as ST if the project is just too big and developers can't be expected to build and test all of this code with every push (so a decision is made to only make some code as PT so that pushes don't take too long). ST code can be included in the TriBITS auto-enable algorithms by setting the variable ${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON (see TriBITS Dependency Handling Behaviors). Otherwise, ST code is not enabled by auto-enable algorithms. Typically, ST code is excluded from the default builds in Pre-Push CI Testing but ST code is typically tested in Post-Push CI Testing, Nightly Testing as well as in other testing cases.

• Experimental (EX) Code is usually too unstable, buggy, or non-portable to be maintained as part of the project's automated testing processes. Or the code may not be important enough to the project to bother paying the cost to test it in the project's automated testing processes. The ability to mark some code as EX allows the developers of that code to include their code in the project's VC repos along with the rest of the project's code and be able to take advantage of all of the development tools provided by the project (using TriBITS) but at the same time not having to "sign up" for all of the responsibilities of maintaining working software that every developer has to take a part in helping to keep working.

The test group for each type of entity is assigned in the following places:

• The top-level TriBITS Package's test group is assigned using the CLASSIFICATION field in the macro call tribits_repository_define_packages() in its parent repository's <repoDir>/PackagesList.cmake file.
• A TriBITS Subpackage's test group is assigned using the CLASSIFICATIONS field of the SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS argument in the macro call tribits_package_define_dependencies() in its parent package's <packageDir>/cmake/Dependencies.cmake file.
• A TriBITS External Package/TPL's test group is assigned using the CLASSIFICATION field in the macro call tribits_repository_define_tpls() in its parent repository's <repoDir>/TPLsList.cmake file.

After these files are processed, the variable ${PACKAGE_NAME}_TESTGROUP gives the test group for each defined Package while the variable ${TPL_NAME}_TESTGROUP gives the test group for each defined TPL.

Note that the test group classification PT/ST/EX is not to be confused with the maturity level of the package as discussed in the TriBITS Lifecycle Model. The test group classification in no way implies the maturity of the given TriBITS Package or piece of code. Instead, the test group is just used to sub-select packages (and pieces of code within those packages) that are the most important to sustain for the various current development group's activities. While more-mature code would typically never be classified as EX (Experimental), there are cases were immature packages may be classified as ST or even PT. For example, a very important research project may be driving the development of a very new algorithm with the low maturity level of Research Stable (RS) or even Exploratory (EP) because keeping that code working may be critical to keeping the research project on track.

In addition to just selecting PT and ST packages as a whole, a TriBITS PT package can also contain conditional code and test directories that get enabled when ${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON and therefore represents more ST code. The package's <packageDir>/CMakeLists.txt files can contain simple if statements and can use the tribits_set_st_for_dev_mode() function to automatically select extra code to enable when ST is enabled or when the project is in release mode.

Test Test Category

Once a package is even defined (due to its parent repository's selection consistent with its Repository Test Classification) and is the package is enabled (consistent with its Package Test Group) then the set of individual test executables and test cases that are included or not in that package depends on the CATEGORIES argument in the functions tribits_add_executable(), tribits_add_test() and tribits_add_advanced_test(), and the ${PROJECT_NAME}_TEST_CATEGORIES variable. This Test Test Category defines the last "knob" that the development team has in controlling what tests get run in a particular test scenario as described in the section Nested Layers of TriBITS Project Testing.

The currently allowed values for the Test Test Category are BASIC, CONTINUOUS, NIGHTLY, HEAVY, and PERFORMANCE. Tests are enabled based on their assigned test test category matching the categories set in the CMake cache variable ${PROJECT_NAME}_TEST_CATEGORIES. The test test categories BASIC, CONTINUOUS, NIGHTLY, and HEAVY are subsets of each other. That is, a BASIC test is automatically included in the set of CONTINUOUS, NIGHTLY, and HEAVY tests (as set using ${PROJECT_NAME}_TEST_CATEGORIES).

The different test test categories are described below in more detail:

• Tests marked BASIC represent key functionality that is needed by nearly every developer that works on the project and so must be protected at all times and are therefore included in Pre-Push CI Testing. Tests marked as BASIC are enabled for the values of ${PROJECT_NAME}_TEST_CATEGORIES of BASIC, CONTINUOUS, NIGHT, and HEAVY. The category BASIC is the default test test category given to all test executables and tests that don't specify the CATEGORIES argument.

• Tests marked CONTINUOUS also represent importantly functionality but are typically not run in Pre-Push CI Testing testing but instead are run in Post-Push CI Testing, Nightly Testing, and other types of testing. Tests marked as CONTINUOUS are enabled for the values of ${PROJECT_NAME}_TEST_CATEGORIES equal to CONTINUOUS, NIGHT, and HEAVY. A test may be marked CONTINUOUS and not BASIC for a few different reasons. For example, the code needed to run the test may take too long to build or the test itself may take too long to run in order to afford including it in Pre-Push CI Testing.

• Tests marked NIGHTLY usually take even longer to build and/or run than CONTINUOUS tests and therefore are too expensive to include in Post-Push CI Testing. Tests may also be marked as NIGHTLY even if they might run relatively fast if there is a desire to not cause the CI server to fail if these tests fail. In this case, the decision is to take the testing and maintenance of these tests and the capabilities they represent "offline" so that they don't influence the daily development cycle for the project but instead are addressed in a "secondary feedback loop". Tests marked as NIGHTLY are enabled for the values of ${PROJECT_NAME}_TEST_CATEGORIES equal to NIGHT, and HEAVY.

• Tests marked HEAVY are usually reserved for very expensive tests that are too expensive to run nightly. HEAVY tests require more testing resources and therefore may only be run on a fully optimized build and/or run less frequently. Tests marked as HEAVY are enabled only for the value of ${PROJECT_NAME}_TEST_CATEGORIES equal to HEAVY.

• Tests marked PERFORMANCE are a special category of tests that are specially designed to measure the serial (non-parallel) run-time performance of parts of the software (see Performance Testing). Tests marked as PERFORMANCE are enabled only for the value of ${PROJECT_NAME}_TEST_CATEGORIES equal to PERFORMANCE.

Every TriBITS project has a default setting for ${PROJECT_NAME}_TEST_CATEGORIES that is set for a basic cmake configure of the project (see ${PROJECT_NAME}_TEST_CATEGORIES_DEFAULT for more details). In addition, the different testing processes described in the section Nested Layers of TriBITS Project Testing set this to different values.

7.2   Nested Layers of TriBITS Project Testing

Now that the different types of Test Classifications for Repositories, Packages, and Tests have been defined, this section describes how these different test-related classifications are used to select repositories, packages (and code) and tests to run in the standard project testing processes. More than any other section in this document, this section will describe and assume a certain class of software development processes (namely agile processes) where testing and continuous integration (CI) are critical components. However, detailed descriptions of these processes are deferred to the later sections Pre-push Testing using checkin-test.py and TriBITS CTest/CDash Driver.

The standard TriBITS-supported project testing processes are:

• Pre-Push CI Testing
• Post-Push CI Testing
• Nightly Testing
• Heavy Testing
• Performance Testing

These standard testing processes are outlined in more detail below and show how the different test-related categories are used to define each of these.

Pre-Push CI Testing

The first level of testing is Pre-Push CI Testing that is performed before changes to the project are pushed to the master branch(es) in the global repository(s). With TriBITS, this type of testing and the following push is typically done using the checkin-test.py tool. This category of testing is described in much more detail in Pre-push Testing using checkin-test.py. All of the "default builds" used with the checkin-test.py tool select repositories, packages and code, and individual tests using the following test-related classifications:

Typically a TriBITS project will define a "standard development environment" which is comprised of a standard compiler (e.g. GCC 8.3.0), external package/TPL versions (e.g. OpenMPI 4.0.5, Boost 4.9, etc.), and other tools (e.g. cmake 3.23.0, git 2.10.1, etc.). This standard development environment is expected to be used to test changes to the project's code before any push. By using a standard development environment, if the code builds and all the tests pass for the "default" pre-push builds for one developer, then that maximizes the probability that the code will also build and all tests will pass for every other developer using the same development environment. This is critical to keep the development team maximally productive. Portability is also important for most projects but portability testing is best done in a secondary feedback look using Nightly Testing builds. TriBITS has some support for helping to set up a standard software development environment as described in section TriBITS Development Toolset.

The basic assumption of all CI processes (including the one described here) is that if anyone pulls the project's development sources at any time, then all of the code will build and all of the tests will pass for the "default" build cases. For a TriBITS project, this means that the project's --default-builds (see above) will all pass for every PT package. All of these software development processes make this basic assumption and agile software development methods fall apart if this is not true.

Post-Push CI Testing

After changes are pushed to the master branch(es) in the global repository(s), Post-Push CI Testing is performed where a CI server detects the changes and immediately fires off a CI build using CTest to test the changes and the results are posted to a CDash server (in the "Continuous" section on the project's dashboard page). This process is driven by CTest driver code that calls tribits_ctest_driver() as described in the section TriBITS CTest/CDash Driver. Various types of specific CI builds can be constructed and run (see CTest/CDash CI Server) but these post-push CI builds typically select repositories, packages and code, and individual tests using the following test-related classifications:

Post-push CI testing would assume to use the same standard development environment as used for Pre-Push CI Testing. Also, the project may also choose to run additional automated post-push CI builds that exactly match the pre-push CI default builds to help check on the health of these builds continuously and not just rely on the development team to always perform the pre-push CI builds correctly before pushing.

Nightly Testing

In addition to pre-push and post-push CI testing, a typical TriBITS project will set up multiple Nightly Testing builds (or once-a-day builds, they don't need to only run at night). These builds are also driven by CTest driver scripts described in the section TriBITS CTest/CDash Driver and post results to the project's CDash server (in the "Nightly" section on the project's dashboard page). Nightly builds don't run in a continuous loop but instead are run once a day (e.g. driven by a cron job) and there tends to be many different nightly build cases that test the project using different compilers (e.g. GCC, Intel, Microsoft, etc., and different versions of each), different external package/TPL versions (e.g. different OpenMPI versions, different MPICH versions, etc.), different platforms (e.g. Linux, Windows, etc.), and varying many other options and settings on these different platforms. What all nightly builds have in common is that they tend to select repositories, packages and code, and individual tests using the following test-related classifications:

The nightly builds comprise the basic "heart beat" for the project.

Heavy Testing

Heavy Testing builds are just an extension to the Nightly Testing builds that add on more expensive tests marked using the Test Test Category HEAVY. For projects that define heavy tests and heavy builds, individual test cases may be allowed to take 24 hours or longer to run so they can't even be run every day in nightly testing. What standard heavy builds have in common is that they tend to select repositories, packages and code, and individual tests using the following test-related classifications:

Project developer teams should strive to limit the number of test cases that are marked as HEAVY since these tests will typically not get run in very may builds or may not be run every day and developers will tend to never enable them when doing more extensive testing using --st-extra-builds with the checkin-test.py tool in extended pre-push testing.

Performance Testing

Performance Testing builds are a special class of builds that have tests that are specifically designed to test the run-time performance of a particular piece of code or algorithm. These tests tend to be sensitive to loads on the machine and therefore typically need to be run on an unloaded machine for reliable results. Details on how to write good performance tests with hard pass/fail time limits is beyond the scope of this document. All TriBITS does is to define the special Test Test Category PERFORMANCE to allow TriBITS packages to declare these tests in a consistent way so that they can be run along with performance tests defined in other TriBITS packages. From a TriBITS standpoint, all performance testing builds would tend to select repositories, packages and code, and individual tests using the following test-related classifications:

7.3   Pre-push Testing using checkin-test.py

CMake provides the integrated tool CTest (executable ctest) which is used to define and run different tests. However, a lot more needs to be done to effectively test changes for a large project before pushing to the master branch(es) in the main repository(s). Things get especially complicated and tricky when multiple version-control (VC) repositories are involved. The TriBITS system provides the tool checkin-test.py for automating the process of:

1. Determining if the local VC repo(s) are ready to integrate with the remote master branch(es),
2. Pulling and integrating the most current changes from the remote VC repo(s),
3. Figuring out what TriBITS packages need to be enabled and testing (by examining VC file diffs),
4. Configuring only the necessary TriBITS packages and tests (and their downstream dependencies by default) and building, running tests, and reporting results (via email), and
5. Only if all specified builds and tests pass, amending the last commit message with the test results, then pushing local commits to the remove VC repo(s) and sending out summary emails.

There are several advantages to using a project's checkin-test.py tool for pushing changes to the main development branch which include:

1. provides a consistent definition for "okay to push" for all developers
2. protects other developers from pulling badly broken code
3. reduces the number of packages that need to be tested by automatically determining the set based on changed files and package dependencies
4. avoids developer mistakes in performing repetitive tasks and forgetting important steps in the process
5. marks a set of working commits that are safe to search with git bisect to find problems (see Using Git Bisect with checkin-test.py workflows)

When using the checkin-test.py tool, every TriBITS project defines one or more "default builds" (specified through the --default-builds argument) for pre-push CI testing that form the criteria for if it is okay to push code changes or not. The "default builds" select repositories, packages and code, and individual tests as described in Pre-Push CI Testing. A TriBITS project defines its default pre-push builds using the file <projectDir>/project-checkin-test-config.py. For an example, the file TribitsExampleProject/project-checkin-test-config.py is shown below:

#
# Define project-specific options for the checkin-test script for
# TribitsExampleProject.
#

configuration = {

    # Default command line arguments
    'defaults': {
        '--send-email-to-on-push': 'trilinos-checkin-tests@software.sandia.gov',
        },

    # CMake options (-DVAR:TYPE=VAL) cache variables.
    'cmake': {
        
        # Options that are common to all builds.
        'common': [],

        # Defines --default-builds, in order.
        'default-builds': [
            # Options for the MPI_DEBUG build.
            ('MPI_DEBUG', [
                '-DTPL_ENABLE_MPI:BOOL=ON',
                '-DCMAKE_BUILD_TYPE:STRING=RELEASE',
                '-DTribitsExProj_ENABLE_DEBUG:BOOL=ON',
                '-DTribitsExProj_ENABLE_CHECKED_STL:BOOL=ON',
                '-DTribitsExProj_ENABLE_DEBUG_SYMBOLS:BOOL=ON',
                ]),
            # Options for the SERIAL_RELEASE build.
            ('SERIAL_RELEASE', [
                '-DTPL_ENABLE_MPI:BOOL=OFF',
                '-DCMAKE_BUILD_TYPE:STRING=RELEASE',
                '-DTribitsExProj_ENABLE_DEBUG:BOOL=OFF',
                '-DTribitsExProj_ENABLE_CHECKED_STL:BOOL=OFF',
                ]),
            ], # default-builds

        }, # cmake

    } # configuration

This gives --default-builds=MPI_DEBUG,SERIAL_RELEASE. As shown, typically two default builds are defined so that various options can be toggled between the two builds. Typical options to toggle include enabling/disabling MPI and enabling/disabling run-time debug mode checking (i.e. toggle ${PROJECT_NAME}_ENABLE_DEBUG). Typically, other important options will also be toggled between these two builds.

Note that both of the default builds shown above, including the MPI_DEBUG build, actually set optimized compiler flags with -DCMAKE_BUILD_TYPE:STRING=RELEASE. What makes the MPI_DEBUG build a "debug" build is turning on optional run-time debug-mode checking, not disabling optimized code. This is important so that the defined tests run fast. For most projects, the default pre-push builds should not be used to debug-enabled code which is suitable to run through a debugger (e.g. gdb). Instead, these "debug" builds are designed to test changes to the project's code efficiently before pushing changes. Typically, a development team should not have to test the chosen compiler's ability to generate non-optimized debug code and suffer slower test times before pushing.

Note that turning on -DTribitsExProj_ENABLE_CHECKED_STL=ON as shown above can only be used when the external packages/TPLs have no C++ code using the C++ STL or if that particular build points to C++ TPLs also compiled with checked STL enabled. The TribitsExampleProject default builds do not depend on any C++ TPLs that might use the C++ STL so enabling this option adds additional positive debug-mode checking for C++ code.

The checkin-test.py tool is a fairly sophisticated piece of software that is well tested and very robust. The level of testing of this tool is likely greater than any of the software that it will be used to test (unless the project is a real-time flight control system or nuclear reactor control system or something). This is needed so as to provide confidence in the developers that the tool will only push their changes if everything checks out as it should. There are a lot of details and boundary cases that one has to consider and a number of use cases that need to be supported by such a tool. For more detailed documentation, see checkin-test.py --help.

Note that the checkin-test.py tool can also be used to implement "poor-man's" post-push testing processes as described in Post-Push CI and Nightly Testing using checkin-test.py. However, most software projects will want to go with the more elaborate and more feature-full CTest/CDash system described in TriBITS CTest/CDash Driver.

7.4   TriBITS CTest/CDash Driver

The TriBITS system uses a sophisticated and highly customized CTest -S driver script to test TriBITS projects and submit results to a CDash server. The primary code for driving this is contained in the CTest function tribits_ctest_driver() contained in the file TribitsCTestDriverCore.cmake. This script loops through all of the specified TriBITS packages for a given TriBITS project and does a configure, built, and test and then submits results to the specified CDash server incrementally. If the configure or library build of any upstream TriBITS package fails, then that TriBITS package is disabled in all downstream TriBITS package builds so as not to propagate already known failures. Each TriBITS top-level package is assigned its own CDash regression email address (see CDash regression email addresses) and each package configure/build/test is given its own row for the package build in the CDash server. A CTest script using tribits_ctest_driver() is run in one of three different modes. First, it can run standard once-a-day, from-scratch builds as described in CTest/CDash Nightly Testing. Second, it can run as a CI server as described in CTest/CDash CI Server. Third, it can run in experimental mode testing a local repository using the TriBITS-defined make dashboard target.

7.5   TriBITS CDash Customizations

CDash is not currently designed to accommodate multi-package, multi-repository VC projects in the way they are supported by TriBITS. However, CDash provides some ability to customize a CDash project and submits to address missing features. Each TriBITS package is given a CTest/CDash "Label" with the name of the TriBITS package. CDash will then aggregate the different package configure/build/test runs for each package into aggregated "builds". The commits pulled for each of the extra VC repos listed in the <projectDir>/cmake/ExtraRepositoriesList.cmake file are shown in an uploaded CDash "Notes" file for each TriBITS package configure/build/test submit. This uploaded "Notes" file also contains a cleaned up version of the CMakeCache.txt file as well as a copy of the ctest -s script that ran the case.

CDash offers numerous features such as the ability to construct a number of different types of queries and is extremely helpful in using past test data.

tribits_read_all_project_deps_files_create_deps_graph()

9.1   Basic Development Workflow

The basic development workflow of a TriBITS project is not much different than with any other CMake project that uses CTest to define and run tests. One pulls updates from the master VC repo then configures with cmake, and iteratively builds, runs tests, adds files, changes files, does a final test, then pushes updates. The major difference is that a well constructed development process will use the checkin-test.py tool to test and push all changes that affect the build or the tests. The basic steps in configuring, building, running tests, etc., are given in the project's <Project>BuildReference. file (see Project-Specific Build Reference).

9.2   Multi-Repository Development Workflow

The development workflow for a project with multiple VC repos is very similar to a project with just a single VC repo if the project provides a standard <projectDir>/cmake/ExtraRepositoriesList.cmake file. The major difference is in making changes, creating commits, etc. The gitdist tool makes these steps easier and has been shown to work fairly well for up to 20 extra VC repos (as used in the CASL VERA project). The checkin-test.py tool automatically handles all of the details of pulling, diffing, pushing etc. to all the VC repos.

10.1   How to add a new TriBITS Package

To add a new TriBITS package, it is recommended to take the template from one of the TribitsExampleProject packages that most closely fits the needs of the new package and modify it for the new package. For example, the files for the SimpleCxx package can be copied one at a time and modified for the new package.

To add a new TriBITS package (with no subpackages), do the following:

1. Chose a name <packageName> for the new package and which TriBITS repository (<repoDir>) to put the package into. WARNING! The chosen name <packageName> must be unique across all TriBITS repositories (see Globally unique TriBITS package names).
2. Create the directory <repoDir>/<packageDir>/ for the new package and put in skeleton files for <packageDir>/cmake/Dependencies.cmake and <packageDir>/CMakeLists.txt. Set the desired upstream TPL and package dependencies in the new Dependencies.cmake file but initially comment out everything in the CMakeLists.txt file except for the tribits_package() and tribits_package_postprocess() commands.
3. Add a row for the new package to the <repoDir>/PackagesList.cmake file after all of its upstream dependent packages. If a mistake is made and it is listed before one of its upstream dependent packages, the TriBITS CMake code will catch this and issue an error.
4. Configure the TriBITS project enabling the new empty package <packageName>. This will enable the listed dependencies.
5. Incrementally fill in the package's CMakeLists.txt files defining libraries, executables, tests and examples. The project should be built and tests run as new pieces are added.

Once the new package is defined, downstream packages can define dependencies on this new package.

10.2   How to add a new TriBITS Package with Subpackages

Adding a new package with subpackages is similar to adding a new regular package described in How to add a new TriBITS Package. Again, it is recommended that one copies an example package from TribitsExampleProject. For example, one could copy files and directories from the example package WithSubpackages.

To add a new TriBITS package with packages, do the following:

1. Chose a name <packageName> for the new package and which TriBITS repository (<repoDir>) to put the package into. WARNING! The chosen name <packageName> must be unique across all TriBITS repositories (see Globally unique TriBITS package names).
2. Create the directory <repoDir>/<packageDir>/ for the new package and put in skeleton files for <packageDir>/cmake/Dependencies.cmake and <packageDir>/CMakeLists.txt. Initially don't define any subpackages and comment out everything in the CMakeLists.txt file except for the tribits_package() and tribits_package_postprocess() commands.
3. Add a new row for the new package to the <repoDir>/PackagesList.cmake file after all of the upstream dependencies of its to-be-defined subpackages.
4. Configure the TriBITS project enabling the new empty package <packageName>.
5. Incrementally add the subpackages as described in How to add a new TriBITS Subpackage, filling out the various CMakeLists.txt files defining libraries, executables, tests and examples.

Once the new packages are defined, downstream packages can define dependencies on these.

10.3   How to add a new TriBITS Subpackage

Given an existing top-level TriBITS package that is already broken down into subpackages (see How to add a new TriBITS Package with Subpackages), adding a new subpackage does not require changing any project-level or repository-level files. One only needs to add the declaration for the new subpackages in its parent's <packageDir>/cmake/Dependencies.cmake file then fill out the pieces of the new subpackage defined in the section TriBITS Subpackage Core Files. It is recommended to copy files from one of the TribitsExampleProject subpackages in the WithSubpackages package.

To add a new TriBITS subpackage to a top-level package that already has subpackages, do the following:

1. Chose a name <spkgName> for the new subpackage which only has to be different than the other subpackages in the parent package. This name gets appended to the parent package's name <packageName> to form the package name <packageName><spkgName>.
2. Create the directory <packageDir><spkgDir>/ for the new package and put in skeleton files for <packageDir>/<spkgDir>/cmake/Dependencies.cmake and <packageDir>/<spkgDir>/CMakeLists.txt. Set the desired upstream TPL and package dependencies in the new Dependencies.cmake file but initially comment out everything in the CMakeLists.txt file except for the tribits_subpackage() and tribits_subpackage_postprocess() commands.
3. Add a row for the new subpackage to the argument SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS in the macro call tribits_package_define_dependencies() in the parent package's <packageDir>/cmake/Dependencies.cmake file after all of its upstream dependent subpackages. If a mistake is made and it is listed before one of its upstream dependent subpackages, the TriBITS CMake code will catch this and issue an error.
4. Configure the TriBITS project enabling the new empty package <packageName><spkgName>. This will enable the listed dependencies.
5. Incrementally fill in the subpackage's CMakeLists.txt files defining libraries, executables, tests and examples. The project should be built and tests run as new pieces are added.

10.4   How to add a new TriBITS external package/TPL

In order for a TriBITS package to define a dependency on a new TriBITS External Package/TPL (i.e. a TPL that has not already been declared in the current repo's or an upstream repo's <repoDir>/TPLsList.cmake file), one must add and modify a few repository-level files in addition to modifying files within the TriBITS packages that use the new external package/TPL.

To add a new TriBITS TPL, do the following:

1. Chose a name <tplName> for the new TPL (must be globally unique across all TriBITS repos. (See Globally unique TriBITS TPL names.)

2. Choose the subdirectory <tplDefsDir> for the new TPL files. These files are usually placed under the TriBITS repository directory <repoDir>/ (e.g. <repoDir>/cmake/tpls/) where the first downstream dependent package is defined or can be under a TriBITS package directory <packageDir>/ (e.g. <packageDir>/cmake/tpls/) if only one package is using that TPL.

3. Create the FindTPL<tplName>.cmake file (or some other name, see <tplName>_FINDMOD) under <tplDefsDir>/. (See Creating the FindTPL<tplName>.cmake file.) However, if the external package/TPL is a TriBITS-compliant external package this file is not needed and is ignored.

4. [Optional] Create the FindTPL<tplName>Dependencies.cmake file in the same directory as the FindTPL<tplName>.cmake file, <tplDefsDir>/. (See FindTPL<tplName>Dependencies.cmake.) NOTE: This file is need for a TriBITS-compliant external package if it has upstream dependent external packages/TPLs (where the file FindTPL<tplName>.cmake is not needed).

5. Add a row to the <repoDir>/TPLsList.cmake file for the new external package/TPL after any upstream TPLs that this new TPL may depend on. NOTE: For a TriBITS-compliant external package, the special value TRIBITS_PKG is used for the TPL (See <repoDir>/TPLsList.cmake.)

6. Configure the TriBITS project enabling the new TPL with TPL_ENABLE_<tplName>=ON and see that the TPL is found correctly at configure time.

7. Add <tplName> to the package Dependencies.cmake files of downstream dependent packages that will use this TPL (see <packageDir>/cmake/Dependencies.cmake or <packageDir>/<spkgDir>/cmake/Dependencies.cmake for a subpackage).

8. [Optional] Add #cmakedefine for an optional package LIB TPL dependency in the package's <packageName>_config.h.in file using:

   #cmakedefine HAVE_<PACKAGE_NAME_UC>_<TPL_NAME_UC>
   

   so that the package's LIB code build knows if the TPL is defined or not (see <packageName>_config.h.in and tribits_configure_file()). (NOTE: Do not add this define for a optional test-only TPL dependency. We don't want all of the libraries in a package to have to be rebuild when we enable or disable tests for the package.)

9. Use the TPL functionality in the packages that define the dependency on the new TPL, configure, test, etc.

Creating the FindTPL<tplName>.cmake file

The main axes of variation in defining FindTPL<tplName>.cmake modules are:

1. Use an inner find_package(<externalPkg>) call or just use a simple list of header file names and library names?
2. The call find_package(<externalPkg>) defines modern IMPORTED targets or only provides variables for the list of include directories and libraries?
3. Maintain backward compatibility with the legacy TriBITS TPL system that allows the user to set cache variables for the include directories and link libraries or force users to always call find_package(<externalPkg>)?

See the different variations in the below sections:

• Creating FindTPL<tplName>.cmake using find_package()
  • Creating FindTPL<tplName>.cmake using find_package() with IMPORTED targets
  • Creating FindTPL<tplName>.cmake using find_package() without IMPORTED targets
• Creating a FindTPL<tplName>.cmake module without find_package()
• Requirements for FindTPL<tplName>.cmake modules

find_package(<externalPkg> REQUIRED)
tribits_extpkg_create_imported_all_libs_target_and_config_file(
  <tplName>
  INNER_FIND_PACKAGE_NAME <externalPkg>
  IMPORTED_TARGETS_FOR_ALL_LIBS <importedTarget0> <importedTarget1> ... )

set(REQUIRED_HEADERS <header0> <header1> ...)
set(REQUIRED_LIBS_NAMES <libname0> <libname1> ...)
set(IMPORTED_TARGETS_FOR_ALL_LIBS <importedTarget0> <importedTarget1> ...)

tribits_tpl_allow_pre_find_package(<tplName>  <tplName>_ALLOW_PREFIND)

if (<tplName>_ALLOW_PREFIND)
  message("-- Using find_package(<externalPkg> ...) ...")
  find_package(<externalPkg>)
  if (<externalPkg>_FOUND)
    message("-- Found <externalPkg>_DIR='${<externalPkg>_DIR}'")
    message("-- Generating <tplName>::all_libs and <tplName>Config.cmake")
    tribits_extpkg_create_imported_all_libs_target_and_config_file(<tplName>
      INNER_FIND_PACKAGE_NAME  <externalPkg>
      IMPORTED_TARGETS_FOR_ALL_LIBS  ${IMPORTED_TARGETS_FOR_ALL_LIBS} )
  endif()
endif()

if (NOT TARGET <tplName>::all_libs)
  tribits_tpl_find_include_dirs_and_libraries( <tplName>
    REQUIRED_HEADERS ${REQUIRED_HEADERS}
    REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES} )
endif()

set(REQUIRED_HEADERS  Tpl2a.hpp) # Only look for one header file to find include dir
set(REQUIRED_LIBS_NAMES  tpl2b tpl2a)
set(IMPORTED_TARGETS_FOR_ALL_LIBS  tpl2::tpl2a tpl2::tpl2b)

tribits_tpl_allow_pre_find_package(Tpl2  Tpl2_ALLOW_PREFIND)

if (Tpl2_ALLOW_PREFIND)
  message("-- Using find_package(Tpl2 ...) ...")
  find_package(Tpl2)
  if (Tpl2_FOUND)
    message("-- Found Tpl2_DIR='${Tpl2_DIR}'")
    message("-- Generating Tpl2::all_libs and Tpl2Config.cmake")
    tribits_extpkg_create_imported_all_libs_target_and_config_file(Tpl2
      INNER_FIND_PACKAGE_NAME  Tpl2
      IMPORTED_TARGETS_FOR_ALL_LIBS  ${IMPORTED_TARGETS_FOR_ALL_LIBS} )
  endif()
endif()

if (NOT TARGET Tpl2::all_libs)
  tribits_tpl_find_include_dirs_and_libraries( Tpl2
    REQUIRED_HEADERS ${REQUIRED_HEADERS}
    REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES} )
endif()

message("-- Using find_package(<externalPkg> ...) ...")
find_package(<externalPkg> REQUIRED)

# Tell TriBITS that we found <tplName> and there no need to look any further
set(TPL_<tplName>_INCLUDE_DIRS ${<externalPkg>_INCLUDE_DIRS} CACHE PATH "...")
set(TPL_<tplName>_LIBRARIES ${<externalPkg>_LIBRARIES} CACHE FILEPATH "...")
set(TPL_<tplName>_LIBRARY_DIRS ${<externalPkg>_LIBRARY_DIRS} CACHE PATH "...")

tribits_tpl_find_include_dirs_and_libraries( <tplName>
  REQUIRED_HEADERS  neverFindThisHeader
  REQUIRED_LIBS_NAMES   neverFindThisLib
  )

set(REQUIRED_HEADERS <header0> <header1> ...)
set(REQUIRED_LIBS_NAMES <libname0> <libname1> ...)

message("-- Using find_package(<externalPkg> ...) ...")
find_package(<externalPkg>)
if (<externalPkg>_FOUND)
  # Tell TriBITS that we found <tplName> and there no need to look any further
  set(TPL_<tplName>_INCLUDE_DIRS ${<externalPkg>_INCLUDE_DIRS} CACHE PATH "...")
  set(TPL_<tplName>_LIBRARIES ${<externalPkg>_LIBRARIES} CACHE FILEPATH "...")
  set(TPL_<tplName>_LIBRARY_DIRS ${<externalPkg>_LIBRARY_DIRS} CACHE PATH "...")
endif()

tribits_tpl_find_include_dirs_and_libraries( <tplName>
  REQUIRED_HEADERS ${REQUIRED_HEADERS}
  REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES}
  MUST_FIND_ALL_LIBS )

set(REQUIRED_HEADERS <header0> <header1> ...)
set(REQUIRED_LIBS_NAMES <libname0> <libname1> ...)

tribits_tpl_allow_pre_find_package(<tplName>  <tplName>_ALLOW_PREFIND)

if (<tplName>_ALLOW_PREFIND)
  message("-- Using find_package<externalPkg> ...) ...")
  find_package(<externalPkg>)
  if (<externalPkg>_FOUND)
    # Tell TriBITS that we found <tplName> and there no need to look any further
    set(TPL_<tplName>_INCLUDE_DIRS ${<externalPkg>_INCLUDE_DIRS} CACHE PATH "...")
    set(TPL_<tplName>_LIBRARIES ${<externalPkg>_LIBRARIES} CACHE FILEPATH "...")
    set(TPL_<tplName>_LIBRARY_DIRS ${<externalPkg>_LIBRARY_DIRS} CACHE PATH "...")
  endif()
endif()

tribits_tpl_find_include_dirs_and_libraries( <tplName>
  REQUIRED_HEADERS ${REQUIRED_HEADERS}
  REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES}
  MUST_FIND_ALL_LIBS )

set(<tplName>_FORCE_PRE_FIND_PACKAGE  TRUE  CACHE  BOOL
  "Always first call find_package(<tplName> ...) unless explicit override")

########################################################################
# See associated tribits/Copyright.txt file for copyright and license! #
########################################################################

set(HDF5_INTERNAL_IS_MODERN FALSE)

if (Netcdf_ALLOW_MODERN)

  print_var(Netcdf_ALLOW_MODERN)
  message("-- Using find_package(HDF5 CONFIG) ...")
  find_package(HDF5 CONFIG)
  if (HDF5_FOUND)
    message("-- Found HDF5_CONFIG=${HDF5_CONFIG}")
    message("-- Generating Netcdf::all_libs and NetcdfConfig.cmake")
    message("-- HDF5_EXPORT_LIBRARIES=${HDF5_EXPORT_LIBRARIES}")
    tribits_extpkg_create_imported_all_libs_target_and_config_file(
      HDF5
      INNER_FIND_PACKAGE_NAME  HDF5
      IMPORTED_TARGETS_FOR_ALL_LIBS   ${HDF5_EXPORT_LIBRARIES})
    set(HDF5_INTERNAL_IS_MODERN TRUE)
  endif()

endif()

set(HDF5_FOUND_MODERN_CONFIG_FILE ${HDF5_INTERNAL_IS_MODERN} CACHE INTERNAL "True if HDF5 was found by the modern method")

if (NOT TARGET HDF5::all_libs)

  # First, set up the variables for the (backward-compatible) TriBITS way of
  # finding HDF5.  These are used in case find_package(HDF5 ...) is not called
  # or does not find HDF5.  Also, these variables need to be non-null in order
  # to trigger the right behavior in the function
  # tribits_tpl_find_include_dirs_and_libraries().

  set(REQUIRED_HEADERS hdf5.h)
  set(REQUIRED_LIBS_NAMES hdf5)

  if (HDF5_REQUIRE_FORTRAN)
    set(REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES} hdf5_fortran)
  endif()

  if (TPL_ENABLE_MPI)
    set(REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES} z)
  endif()

  if (TPL_ENABLE_Netcdf)
    set(REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES} hdf5_hl)
  endif()

  #
  # Second, search for HDF5 components (if allowed) using the standard
  # find_package(HDF5 ...).
  #
  tribits_tpl_allow_pre_find_package(HDF5  HDF5_ALLOW_PREFIND)
  if (HDF5_ALLOW_PREFIND)

    message("-- Using find_package(HDF5 ...) ...")

    set(HDF5_COMPONENTS C)
    if (HDF5_REQUIRE_FORTRAN)
      list(APPEND HDF5_COMPONENTS Fortran)
    endif()

    if (TPL_ENABLE_MPI)
      set(HDF5_PREFER_PARALLEL TRUE)
    endif()

    find_package(HDF5 COMPONENTS ${HDF5_COMPONENTS})

    # Make sure that HDF5 is parallel.
    if (TPL_ENABLE_MPI AND NOT HDF5_IS_PARALLEL)
      message(FATAL_ERROR "Trilinos is configured for MPI, HDF5 is not.
      Did CMake find the correct libraries?
      Try setting HDF5_INCLUDE_DIRS and/or HDF5_LIBRARY_DIRS explicitly.
      ")
    endif()

    if (HDF5_FOUND)
      # Tell TriBITS that we found HDF5 and there no need to look any further!
      set(TPL_HDF5_INCLUDE_DIRS ${HDF5_INCLUDE_DIRS} CACHE PATH
        "HDF5 include dirs")
      set(TPL_HDF5_LIBRARIES ${HDF5_LIBRARIES} CACHE FILEPATH
        "HDF5 libraries")
      set(TPL_HDF5_LIBRARY_DIRS ${HDF5_LIBRARY_DIRS} CACHE PATH
        "HDF5 library dirs")
    endif()

  endif()

  #
  # Third, call tribits_tpl_find_include_dirs_and_libraries()
  #
  tribits_tpl_find_include_dirs_and_libraries( HDF5
    REQUIRED_HEADERS ${REQUIRED_HEADERS}
    REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES}
    )
  # NOTE: If find_package(HDF5 ...) was called and successfully found HDF5, then
  # tribits_tpl_find_include_dirs_and_libraries() will use the already-set
  # variables TPL_HDF5_INCLUDE_DIRS and TPL_HDF5_LIBRARIES and then print them
  # out (and set some other standard variables as well).  This is the final
  # "hook" into the TriBITS TPL system.

endif()

tribits_tpl_find_include_dirs_and_libraries( <tplName>
  REQUIRED_HEADERS <header0> <header1> ...
  REQUIRED_LIBS_NAMES <libname0> <libname1> ...
  MUST_FIND_ALL_LIBS
  )

tribits_tpl_find_include_dirs_and_libraries( <tplName>
  ...
  REQUIRED_LIBS_NAMES "<libname0> <libname0alt0> <libname0alt1> ..." ...
  ...
  )

tribits_tpl_find_include_dirs_and_libraries( BLAS
  REQUIRED_LIBS_NAMES "blas openblas atlas"
  ...
  )

10.5   How to add a new TriBITS Repository

To add a new TriBITS and/ git VC repository to a TriBITS project that already contains other extra repositories, do the following:

1. Add a row for the new TriBITS and/or git repo to the file <projectDir>/cmake/ExtraRepositoriesList.cmake. Commit this file.
2. Add an ignore for the extra repo name to the base project's git repo's .gitignore file (see Multi-Repository Support). Commit this file.
3. Set up the new package dependencies for the new package in the new TriBITS repo or make other adjustments.
4. Consider the potential for missing upstream repos and packages by using tribits_allow_missing_external_packages().

See <projectDir>/cmake/ExtraRepositoriesList.cmake for more details and links.

10.6   How to add a new TriBITS Package dependency

It is often the case where one will want to add a new dependency for an existing downstream package to an existing upstream (internal or external) TriBITS Package. This can either be a required dependency or an optional dependency. Here, we will refer to the downstream package as <packageName> with base directory <packageDir> and will refer to the upstream (internal or external) package as <upstreamPackageName>.

The process for adding a new dependency to an existing upstream package is as follows:

1. Add the name of the upstream package to the downstream package's Dependencies.cmake file: Add <upstreamPackagename> to the call of tribits_package_define_dependencies() in the downstream package's <packageDir>/cmake/Dependencies.cmake file. If this is to be a required library dependency, then <upstreamPackagename> is added to the LIB_REQUIRED_PACKAGES argument. Alternatively, if this is to be an optional library dependency, then <upstreamPackagename> is added to the LIB_OPTIONAL_PACKAGES argument. (For example, see the file packages/EpetaExt/cmake/Dependencies.cmake file in the ReducedMockTrilinos project.) If only the test and/or example sources, and not the package's core library sources, will have the required or optional dependency, then <upstreamPackagename> is added to the arguments TEST_REQUIRED_PACKAGES or TEST_OPTIONAL_PACKAGES, respectively.

2. For an optional dependency, add `HAVE_` preprocessor macro to the package's configured header file: If this is an optional dependency, typically a C/C++ processor macro will be added to the package's configured <packageDir>/cmake/<packageName>_config.h.in file using the line:

   #cmakedefine HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC>
   

   (see HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC>.)

   Warning, do not add optional defines for tests/examples to configured header files: If this is a test-only and/or example-only dependency then please do not add a #cmakedefine to the package's core <packageDir>/cmake/<packageName>_config.h.in file. Instead, add the #cmakedefine line to a configured header that is only included by sources for the tests/examples or just add a define on the compiler command line (see the DEFINES argument to tribits_add_library() and tribits_add_executable(), but see the warning about problems with add_definitions() in Miscellaneous Notes (tribits_add_library())). We don't want the package's header files to change or libraries to have to be rebuilt if tests/examples get enabled or disabled. Otherwise, the TriBITS CTest/CDash Driver process will result in unnecessary rebuilds of software over and over again.

3. Use the features of the upstream package in the source files of the downstream package sources and/or tests/examples: Usage of the features of the upstream package <upstreamPackageName> in the downstream package <packageName> will typically involve adding #include <upstreamPackageName>_<fileName> in the package's C/C++ source (or test/example) files (or the equivalent in Fortran). If it is an optional dependency, then these includes will typically be protected using preprocessor ifdefs, for example, as:

   #include "<packageName>_config.h"
   
   #if HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC>
   #  include "<upstreamPackageName>_<fileName>"
   #endif
   

4. For an optional dependency, use CMake if() statements based on ${PACKAGE_NAME}_ENABLE_<upstreamPackageName>: When a package PACKAGE_NAME has an optional dependency on an upstream package <upstreamPackageName> and needs to put in optional logic in a CMakeLists.txt file, then the if() statements should use the variable ${PACKAGE_NAME}_ENABLE_<upstreamPackageName> and not the variable ${PROJECT_NAME}_ENABLE_<upstreamPackageName> or TPL_ENABLE_<upstreamPackageName> (if <upstreamPackageName> is an external package/TPL). For example, to optionally enable a test that depends on the enable of the optional upstream dependent package <upstreamPackageName>, one would use:

   tribits_add_test( ...
     EXCLUDE_IF_NOT_TRUE  ${PACKAGE_NAME}_ENABLE_<upstreamPackageName>
     )
   

   or:

   if (${PACKAGE_NAME}_ENABLE_<upstreamPackageName>)
     tribits_add_test( ... )
   endif()
   

NOTE: TriBITS will automatically add the include directories for the upstream package to the compile lines for the downstream package source builds and will add the libraries for the upstream package to the link lines to the downstream package library and executable links. See documentation in the functions tribits_add_library() and tribits_add_executable(), and the DEPLIBS argument to these functions, for more details.

10.7   How to tentatively enable an external package/TPL

A TriBITS package can request the tentative enable of any of its optional external packagse/TPLs (see How to add a new TriBITS Package dependency). This is done by calling tribits_tpl_tentatively_enable() in the package's <packageDir>/cmake/Dependencies.cmake file. For example:

tribits_package_define_dependencies(
  ...
  LIB_OPTIONAL_TPLS  SomeTpl
  ...
  )

tribits_tpl_tentatively_enable(SomeTpl)

This will result is an attempt to find the components for the TPL SomeTpl. But if that attempt fails, then the TPL will be disabled and ${PACKAGE_NAME}_ENABLE_SomeTpl will be set to OFF.

10.8   How to insert a package into an upstream repo

Sometimes it is desired to insert a package from a downstream VC repo into an upstream TriBITS Repository in order for one or more packages in the upstream repo to define a dependency on that package. The way this is supported in TriBITS is to just list the inserted package into the PackagesList.cmake file of the upstream TriBITS repo after the packages it depends on and before the packages that will use it then call the tribits_allow_missing_external_packages() function to allow the package to be missing. This is demonstrated in TribitsExampleProject with the package InsertedPkg which is not included in the default TribitsExampleProject source tree. The TribitsExampleProject/PackagesList.cmake file looks like:

tribits_repository_define_packages(
  SimpleCxx          packages/simple_cxx         PT
  MixedLang          packages/mixed_lang         PT
  InsertedPkg        InsertedPkg                 ST
  WithSubpackages    packages/with_subpackages   PT
  WrapExternal       packages/wrap_external      ST
  )

tribits_disable_package_on_platforms(WrapExternal Windows)
tribits_allow_missing_external_packages(InsertedPkg)

In this example, InsertedPkg has a required dependency on SimpleCxx and the package WithSubpackagesB has an optional dependency on InsertedPkg. Therefore, the inserted package InsertedPkg has upstream and downstream dependencies on packages in the TribitsExampleProject repo.

The function tribits_allow_missing_external_packages() tells TriBITS to treat InsertedPkg the same as any other package if the directory TribitsExampleProject/InsertedPkg exists or to completely ignore the package InsertedPkg otherwise. In addition, TriBITS will automatically disable of all downstream package dependencies for the missing package (and print a note about the disables). NOTE: By default TriBITS will silently ignore missing inserted packages and disable optional support for the missing package. To see what packages are missing and being ignored, configure with:

-D <Project>_WARN_ABOUT_MISSING_EXTERNAL_PACKAGES=TRUE

The way one would set up TribitsExampleProject to enable InsertedPkg, if these were in separate VC (e.g. git) repos for example, would be to do:

$ git clone <some-url-base>/TribitsExampleProject
$ cd TribitsExampleProject
$ git clone <some-other-url-base>/ExteranlPkg
$ echo /InsertedPkg/ >> .git/info/excludes

Then, when you configure TribitsExampleProject, the package InsertedPkg would automatically appear and could then be enabled or disabled like any other TriBITS package. The TriBITS test Tribits_TribitsExampleProject_InsertedPkg demonstrates this.

Assuming that one would put the (new) external package in a separate VC repo, one would perform the following steps:

1. Pick a name for the new inserted external package <insertedPackageName> (see Globally unique TriBITS package names). (NOTE: The external package may already exist in which case the name would already be selected).
2. Create a new VC repo containing the new package <insertedPackageName> or add it to an existing extra VC repo. (Or, add the new package to an existing downstream repo but don't add it to the PackagesList.cmake file to downstream repo. That would define the package twice!)
3. Clone the downstream VC repo under the upstream TriBITS repository. (This is currently needed since TriBITS only allows package dirs to be contained under the repository directory.)
4. Insert the package into the <repoDir>/PackagesList.cmake file as described in How to add a new TriBITS Package except one must also call tribits_allow_missing_external_packages(<insertedPackageName>) as described above.
5. Flesh out the new package and use it in downstream packages just like it was any other package. But note that any downstream package that has a required dependency on <insertedPackageName> will always be hard disabled when the source for <insertedPackageName> is missing.
6. When configuring and building to get the package working, add -D<insertedPackageName>_ALLOW_MISSING_EXTERNAL_PACKAGE=FALSE so that TriBITS will catch mistakes in specifying the package directory. Otherwise, to see notes about ignoring missing inserted/external packages, set the variable -D<Project>_WARN_ABOUT_MISSING_EXTERNAL_PACKAGES=TRUE and TriBITS will print warnings about missing external packages.

10.9   How to put a TriBITS and raw CMake build system side-by-side

There are cases where it is advantageous to have a raw CMake build system and a TriBITS CMake build system sit side-by-side in a CMake project. There are various ways to accomplish this but a very simple way that has minimal impact on the raw CMake build system is described here. An example of how to accomplish this is shown in the example project RawAndTribitsHelloWorld. This CMake project is a copy of the TribitsHelloWorld project that puts a primary default raw CMake build system side-by-side with a secondary TriBITS CMake build system. The key aspects of this basic approach shown in this example are:

1. An if() statement must be added to the base project CMakeLists.txt file to switch between the two build systems. (This is required since the raw CMake commands cmake_minimum_required() and project() must exist in the base CMakeLists.txt file and not in and included *.cmake file.) The switch trigger in the if() statement can be any logic desired, but a simple way is to look for the ${PROJECT_NAME}_TRIBITS_DIR cache variable being set.
2. The TriBITS build system in every subdirectory is contained in files of the name CMakeLists.tribits.cmake beside the CMakeLists.txt files for the raw CMake build system. (The file CMakeLists.tribits.cmake ends with a *.cmake extension so that source editors pick it up as a CMake file.)
3. At the top of every raw CMake build system CMakeLists.txt file is a call to a simple macro include_tribits_build() which includes the CMakeLists.tribits.cmake file and then returns if doing a TriBITS build and otherwise does nothing for a raw CMake build.

The top file RawAndTribitsHelloWorld/CMakeLists.txt file demonstrates the basic approach:

cmake_minimum_required(VERSION 3.23.0 FATAL_ERROR)
include(${CMAKE_CURRENT_SOURCE_DIR}/ProjectName.cmake)

# Called at the top of every CMakeLists.txt file
macro(include_tribits_build)
  if (${PROJECT_NAME}_TRIBITS_DIR)
    include("${CMAKE_CURRENT_SOURCE_DIR}/CMakeLists.tribits.cmake")
    return()
  endif()
endmacro()

if (${PROJECT_NAME}_TRIBITS_DIR)

  # TriBITS CMake project
  project(${PROJECT_NAME} NONE)
  include("${${PROJECT_NAME}_TRIBITS_DIR}/TriBITS.cmake")
  # Only one package in this simple project so just enable it :-)
  set(${PROJECT_NAME}_ENABLE_HelloWorld ON CACHE BOOL "" FORCE)
  tribits_project()

else()

  # Raw CMake project
  project(RawHelloWorld)
  enable_testing()
  add_subdirectory(hello_world)

endif()

# NOTE: The cmake_minimum_required() and project() commands must be executed
# in the base CMakeLists.txt file and *NOT* in an included 

Then every raw CMakeLists.txt file starts with the command include_tribits_build() at the very top as shown in the example file RawAndTribitsHelloWorld/hello_world/CMakeLists.txt:

include_tribits_build()

# Build and install library
set(HEADERS hello_world_lib.hpp)
set(SOURCES hello_world_lib.cpp)
add_library(hello_world_lib ${SOURCES})
install(TARGETS hello_world_lib DESTINATION lib)
install(FILES ${HEADERS} DESTINATION include)

# Build and install user executable
add_executable(hello_world hello_world_main.cpp)
target_link_libraries(hello_world hello_world_lib)
install(TARGETS hello_world DESTINATION bin)

# Test the executable
add_test(test ${CMAKE_CURRENT_BINARY_DIR}/hello_world)
set_tests_properties(test PROPERTIES PASS_REGULAR_EXPRESSION "Hello World")

# Build and run some unit tests  
add_executable(unit_tests hello_world_unit_tests.cpp)
target_link_libraries(unit_tests hello_world_lib)
add_test(unit_test ${CMAKE_CURRENT_BINARY_DIR}/unit_tests)
set_tests_properties(unit_test
  PROPERTIES PASS_REGULAR_EXPRESSION "All unit tests passed")

To configure the project as a raw CMake project, just configure it as with any raw CMake project as:

cmake [options] <some_base_dir>/RawAndTribitsHelloWorld

To configure it as a TriBITS project, just set the cache var RawAndTribitsHelloWorld_TRIBITS_DIR to point to valid TriBITS source tree as:

cmake [options] \
  -DRawAndTribitsHelloWorld_TRIBITS_DIR=<tribits_dir> \
   <some_base_dir>/RawAndTribitsHelloWorld

A twist on this use case is for a package that only builds as a TriBITS package inside of some larger TriBITS project and not as its own TriBITS CMake project. In this case, some slight changes are needed to this example but the basic approach is nearly identical. One still needs an if() statement at the top of the first CMakeLists.txt file (this time for the package) and the macro include_tribits_build() needs to be defined at the top of that file as well. Then every CMakeLists.txt file in subdirectories just calls include_tribits_build(). That is it.

10.10   How to implement a TriBITS-compliant internal package using raw CMake

As described in TriBITS-Compliant Internal Packages, it is possible to create a raw CMake build system for a CMake package that can build under a parent TriBITS CMake project. The raw CMake code for such a package must provide the <Package>::all_libs target both in the current CMake build system and also in the generated <Package>Config.cmake file for the build directory and in the installed <Package>Config.cmake file. Every such TriBITS-compliant internal package therefore is also capable of installing a TriBITS-compliant external package <Package>Config.cmake file (see How to implement a TriBITS-compliant external package using raw CMake).

A raw CMake build system for a TriBITS-compliant internal package is demonstrated in the TribitsExampleProject2 project Package1 package. The base CMakeLists.txt file for building Package1 with a raw CMake build system (called package1/CMakeLists.raw.cmake in that directory) looks like:

cmake_minimum_required(VERSION 3.23.0 FATAL_ERROR)

if (COMMAND tribits_package)
  message("Configuring raw CMake package Package1")
else()
  message("Configuring raw CMake project Package1")
endif()

# Standard project-level stuff
project(Package1 LANGUAGES C CXX)
include(GNUInstallDirs)
find_package(Tpl1 CONFIG REQUIRED)
add_subdirectory(src)
if (Package1_ENABLE_TESTS)
  include(CTest)
  add_subdirectory(test)
endif()

# Stuff that TriBITS does automatically
include("${CMAKE_CURRENT_LIST_DIR}/cmake/raw/DefineAllLibsTarget.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/cmake/raw/GeneratePackageConfigFileForBuildDir.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/cmake/raw/GeneratePackageConfigFileForInstallDir.cmake")

As shown above, this simple CMake package contains the basic features of any CMake project/package including calling the cmake_minimum_required() and project() commands as well as including GNUInstallDirs. In this example, the project/package being built Package1 has a dependency on an external upstream package Tpl1 pulled in with find_package(Tpl1). Also in this example, the package has native tests it defines with include(CTest) and add_subdirectory() (if Package1_ENABLE_TESTS is set to ON).

The file package1/src/CMakeLists.raw.cmake (which gets included from package1/src/CMakeLists.txt) creates a library and executable for the package and has the contents:

# Create and install library 'package1'
add_library(Package1_package1 Package1.hpp Package1.cpp)
target_include_directories(Package1_package1
  PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>)
target_link_libraries(Package1_package1
  PRIVATE tpl1::tpl1 )
set_target_properties(Package1_package1 PROPERTIES
  EXPORT_NAME package1)
add_library(Package1::package1 ALIAS Package1_package1)
install(TARGETS Package1_package1
  EXPORT ${PROJECT_NAME}
  INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} )
install(
  FILES Package1.hpp
  DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} )

# Create and install executable 'package1-prg' 
add_executable(package1-prg Package1_Prg.cpp)
target_link_libraries(package1-prg PRIVATE Package1::package1)
install(
  TARGETS package1-prg
  EXPORT ${PROJECT_NAME}
  INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} )

This creates a single installable library target Package1_package1 which is aliased as Package1::package1 in the current CMake project and sets up to create the IMPORTED target Package1::package1 in the generated Package1ConfigTarget.cmake file, which gets included in the installed Package1Config.cmake (<Package>Config.cmake) file (as recommenced in the book "Professional CMake", see below). In addition, the above code creates the installable executable package1-prg.

The Package1::all_libs (<Package>::all_libs) target is defined and set up inside of the included file package1/cmake/raw/DefineAllLibsTarget.cmake which contains the code:

# Generate the all_libs target(s)
add_library(Package1_all_libs INTERFACE)
set_target_properties(Package1_all_libs
  PROPERTIES EXPORT_NAME all_libs)
target_link_libraries(Package1_all_libs
  INTERFACE Package1_package1)
install(TARGETS Package1_all_libs
  EXPORT ${PROJECT_NAME}
  COMPONENT ${PROJECT_NAME}
  INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} )
add_library(Package1::all_libs ALIAS Package1_all_libs)

The above code contains the ALIAS library target Package1::all_libs (<Package>::all_libs) for the current CMake project as well as sets up for the IMPORTED target Package1::all_libs (<Package>::all_libs) getting put in the generated Package1ConfigTargets.cmake file (see below).

The Package1Config.cmake (<Package>Config.cmake) file for the build directory is generated inside of the included file package1/cmake/raw/GeneratePackageConfigFileForBuildDir.cmake which has the contents:

if (COMMAND tribits_package)
  # Generate Package1Config.cmake file for the build tree (for internal
  # TriBITS-compliant package)
  set(packageBuildDirCMakePackagesDir
    "${${CMAKE_PROJECT_NAME}_BINARY_DIR}/cmake_packages/${PROJECT_NAME}")
  export(EXPORT ${PROJECT_NAME}
    NAMESPACE ${PROJECT_NAME}::
    FILE "${packageBuildDirCMakePackagesDir}/${PROJECT_NAME}ConfigTargets.cmake" )
  configure_file(
    "${CMAKE_CURRENT_LIST_DIR}/Package1Config.cmake.in"
    "${packageBuildDirCMakePackagesDir}/${PROJECT_NAME}/Package1Config.cmake"
    @ONLY )
endif()

The above code uses the export(EXPORT ...) command to generate the file Package1ConfigTargets.cmake for the build directory which provides the IMPORTED targets Package1::package1 and Package1::all_libs. The command configure_file(...) generates the Package1Config.cmake file that includes it for the build directory <buildDir>/cmake_packages/Package1/. (NOTE: The above code only runs when the package is being built from inside of a TriBITS project which defines the command tribits_package. So this code gets skipped when building Package1 as a stand-alone raw CMake project.)

Finally, the code for generating and installing the Package1Config.cmake file for the install directory CMAKE_PREFIX_PATH=<installDir> is specified in the included file package1/cmake/raw/GeneratePackageConfigFileForInstallDir.cmake with the contents:

# Generate and install the Package1Config.cmake file for the install tree
# (needed for both internal and external TriBITS package)
set(pkgConfigInstallDir "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
install(EXPORT ${PROJECT_NAME}
  DESTINATION "${pkgConfigInstallDir}"
  NAMESPACE ${PROJECT_NAME}::
  FILE ${PROJECT_NAME}ConfigTargets.cmake )
configure_file(
  "${CMAKE_CURRENT_SOURCE_DIR}/cmake/raw/Package1Config.cmake.in"
  "${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/Package1Config.install.cmake"
  @ONLY )
install(
  FILES "${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/Package1Config.install.cmake"
  RENAME "Package1Config.cmake"
  DESTINATION "${pkgConfigInstallDir}" )

The above uses the command install(EXPORT ...) to have CMake automatically generate and install the file Package1ConfigTargets.cmake in the install directory <installDir>/libs/cmake/Package1/ which provides the IMPORTED targets Package1::package1 and Package1::all_libs. The command configure_file() is used to generate the file Package1Config.install.cmake in the build directory from the template file Package1Config.cmake.in. Finally, the install() command is used in the file GeneratePackageConfigFileForInstallDir.cmake to set up the installation of the Package1Config.cmake file.

Note, the template file package1/cmake/raw/Package1Config.cmake.in (which is unique to Package1) is:

set(Tpl1_DIR "@Tpl1_DIR@")
find_package(Tpl1 CONFIG REQUIRED)
include("${CMAKE_CURRENT_LIST_DIR}/Package1ConfigTargets.cmake")

As shown in the all of the above code, there is a lot of boilerplate CMake code needed to correctly define the targets such that they get put into the installed Package1Config.cmake file using the correct namespace Package1:: and care must be taken to ensure that a consistent "export set" is used for this purpose. (For more details, see the book "Professional CMake".)

NOTE: One should compare the above raw CMakeLists files to the more compact TriBITS versions for the base package1/CMakeLists.txt file (called package1/CMakeLists.tribits.cmake in the base directory pacakge1/):

message("Configuring package Package1 as full TriBITS package")
tribits_package(Package1)
add_subdirectory(src)
tribits_add_test_directories(test)
tribits_package_postprocess()

and the TriBITS package1/src/CMakeLists.txt file (called package1/src/CMakeLists.tribits.cmake):

tribits_include_directories(${CMAKE_CURRENT_SOURCE_DIR})
tribits_add_library(package1  HEADERS  Package1.hpp  SOURCES  Package1.cpp)
tribits_add_executable(package1-prg  NOEXEPREFIX  NOEXESUFFIX
  SOURCES  Package1_Prg.cpp  INSTALLABLE )

This shows the amount of boilerplate code that TriBITS addresses automatically (which reduces the overhead of finer-grained packages and avoids common mistakes with tedious low-level CMake code).

10.11   How to implement a TriBITS-compliant external package using raw CMake

As described in TriBITS-Compliant External Packages, it is possible to create a raw CMake build system for a CMake package such that once it is installed, satisfies the requirements for a TriBITS-compliant external package. These installed packages provide a <Package>Config.cmake file that provides the required targets and behaviors as if it was produced by a TriBITS project. For most existing raw CMake projects that already produce a "Professional CMake" compliant <Package>Config.cmake file, that usually just means adding the IMPORTED target called <Package>::all_libs to the installed <Package>Config.cmake file.

A raw CMake build system for a TriBITS-compliant external package is demonstrated in the TribitsExampleProject2 project Package1 package. The base package1/CMakeLists.txt file for building Package1 with a raw CMake build system (called package1/CMakeLists.raw.cmake) for implementing a TriBITS-compliant internal package looks like:

cmake_minimum_required(VERSION 3.23.0 FATAL_ERROR)

if (COMMAND tribits_package)
  message("Configuring raw CMake package Package1")
else()
  message("Configuring raw CMake project Package1")
endif()

# Standard project-level stuff
project(Package1 LANGUAGES C CXX)
include(GNUInstallDirs)
find_package(Tpl1 CONFIG REQUIRED)
add_subdirectory(src)
if (Package1_ENABLE_TESTS)
  include(CTest)
  add_subdirectory(test)
endif()

# Stuff that TriBITS does automatically
include("${CMAKE_CURRENT_LIST_DIR}/cmake/raw/DefineAllLibsTarget.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/cmake/raw/GeneratePackageConfigFileForInstallDir.cmake")

Note that the raw build system this example is identical to the build system for the raw TriBITS-compliant internal package described in How to implement a TriBITS-compliant internal package using raw CMake. The only differences are:

1. The Package1Config.cmake (<Package>Config.cmake) file does not need to be generated for the build directory and therefore the code in cmake/raw/GeneratePackageConfigFileForBuildDir.cmake does not need to be included.
2. The ALIAS library target Package1::all_libs (<Package>::all_libs) does not need to be generated (but should be to be "Professional CMake" compliant).

Other than that, see How to implement a TriBITS-compliant internal package using raw CMake for how to implement a TriBITS-compliant external package.

10.12   How to use TriBITS testing support in non-TriBITS project

The TriBITS test support functions tribits_add_test() and tribits_add_advanced_test() can be used from any raw (i.e. non-TriBITS) CMake project. To do so, one just needs to include the TriBITS modules:

• <tribitsDir>/core/test_support/TribitsAddTest.cmake
• <tribitsDir>/core/test_support/TribitsAddAdvancedTest.cmake

and set the variable ${PROJECT_NAME}_ENABLE_TESTS to ON. For an MPI-enabled CMake project, the CMake variables MPI_EXEC, MPI_EXEC_PRE_NUMPROCS_FLAGS, MPI_EXEC_NUMPROCS_FLAG and MPI_EXEC_POST_NUMPROCS_FLAGS must also be set which define the MPI runtime program launcher command-line used in the TriBITS testing functions:

${MPI_EXEC} ${MPI_EXEC_PRE_NUMPROCS_FLAGS}
  ${MPI_EXEC_NUMPROCS_FLAG} <NP>
  ${MPI_EXEC_POST_NUMPROCS_FLAGS}
  <TEST_EXECUTABLE_PATH> <TEST_ARGS>

(NOTE: These variables are defined automatically in a TriBITS project when TPL_ENABLE_MPI is set to ON.)

This is demonstrated in the TribitsExampleProject2 project Package1 package. The base pacakge1/CMakeLists.txt file for building Package1 with a raw CMake build system using TriBITS testing functions (called package1/CMakeLists.raw.cmake) looks like:

cmake_minimum_required(VERSION 3.23.0 FATAL_ERROR)

if (COMMAND tribits_package)
  message("Configuring raw CMake package Package1")
else()
  message("Configuring raw CMake project Package1")
endif()

# Standard project-level stuff
project(Package1 LANGUAGES C CXX)
include(GNUInstallDirs)
find_package(Tpl1 CONFIG REQUIRED)
add_subdirectory(src)
if (Package1_ENABLE_TESTS)
  include(CTest)
  include("${CMAKE_CURRENT_LIST_DIR}/cmake/raw/EnableTribitsTestSupport.cmake")
  add_subdirectory(test)
endif()

The only difference between this base package1/CMakeLists.txt file and one for a raw CMake project is the inclusion of the file package1/cmake/raw/EnableTribitsTestSupport.cmake which has the contents:

set(Package1_USE_TRIBITS_TEST_FUNCTIONS  OFF  CACHE  BOOL
  "Use TriBITS testing functions")
set(Package1_TRIBITS_DIR  ""  CACHE  PATH
  "Path to TriBITS implementation base dir (e.g. TriBITS/tribits)")
if (Package1_USE_TRIBITS_TEST_FUNCTIONS  AND  Package1_TRIBITS_DIR)
  # Pull in and turn on TriBITS testing support
  include("${Package1_TRIBITS_DIR}/core/test_support/TribitsAddTest.cmake")
  include("${Package1_TRIBITS_DIR}/core/test_support/TribitsAddAdvancedTest.cmake")
  set(Package1_ENABLE_TESTS ON)
endif()

The key lines are:

include("${Package1_TRIBITS_DIR}/core/test_support/TribitsAddTest.cmake")
include("${Package1_TRIBITS_DIR}/core/test_support/TribitsAddAdvancedTest.cmake")

This defines the CMake functions tribits_add_test() and tribits_add_advanced_test(), respectively.

The above code demonstrates that CMAKE_MODULE_PATH does not need to be updated to use these TriBITS test_support modules. However, one is free to update CMAKE_MODULE_PATH and then include the modules by name only like:

list(PREPEND CMAKE_MODULE_PATH "${Package1_TRIBITS_DIR}/core/test_support")
include(TribitsAddTest)
include(TribitsAddAdvancedTest)

Once these TriBITS modules are included, one can use the TriBITS functions as demonstrated in the file package1/test/CMakeLists.tribits.cmake (which is included from the file package1/test/CMakeLists.txt) and has the contents:

tribits_add_test(package1-prg  NOEXEPREFIX  NOEXESUFFIX
  NAME Prg  DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/../src"  NUM_MPI_PROCS 1
  PASS_REGULAR_EXPRESSION "Package1 Deps: tpl1" )

tribits_add_advanced_test(Prg-advanced
  TEST_0
    EXEC package1-prg  DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/../src"
      NOEXEPREFIX  NOEXESUFFIX
    ARGS "something_extra"
    PASS_REGULAR_EXPRESSION_ALL
      "Package1 Deps: tpl1"
      "something_extra"
    ALWAYS_FAIL_ON_NONZERO_RETURN
  )

Note that in this example, the executable package1-prg was already created. If new test libraries and executables need to be created, then the raw CMake commands to create those will need to be added as well.

10.13   How to check for and tweak TriBITS "ENABLE" cache variables

TriBITS defines a number of special <XXX>_ENABLE_<YYY> variables for enabling/disabling various entities that allow for a default "undefined" empty "" enable status. Examples of these special variables include:

• ${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE> (Packages)
• TPL_ENABLE_<tplName> (External Packages/TPLs)
• <TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE> (Support for a package <TRIBITS_DEP_PACKAGE> in a downstream package <TRIBITS_PACKAGE>)
• <TRIBITS_PACKAGE>_ENABLE_TESTS (Package tests)
• <TRIBITS_PACKAGE>_ENABLE_EXAMPLES (Package examples)
• ${PROJECT_NAME}_ENABLE_TESTS (Tests for explicitly enabled packages)
• ${PROJECT_NAME}_ENABLE_EXAMPLES (Examples for explicitly enabled packages)

(see TriBITS Dependency Handling Behaviors).

To check for and tweak these special "ENABLE" variables, perform the following:

1. To check to see if an ENABLE variable has been enabled or disabled (either explicitly or through auto enable/disable logic), use:

   if ("${<XXX>_ENABLE_<YYY>}" STREQUAL "")
     # Variable has not been set to 'ON' or 'OFF' yet
     ...
   endif()
   

   This will work correctly independent of if the cache variable has been default defined or not.

2. To tweak the enable/disable of one or more of these variables after user input but before the step "Adjust package and TPLs enables and disables" in Full Processing of TriBITS Project Files:

1. To tweak the enables/disables for a TriBITS Repository (i.e. affecting all TriBITS projects) add enable/disable code to the file <repoDir>/cmake/RepositoryDependenciesSetup.cmake.
2. To tweak the enables/disables for a specific TriBITS Project (i.e. affecting only that TriBITS project) add enable/disable code to the file <projectDir>/cmake/ProjectDependenciesSetup.cmake.

For example, one might default disable a package if it has not been explicitly enabled (or disabled) in one of these files using logic like:

if (NOT ${PROJECT_NAME}_ENABLE_Fortran)
  if ("${${PROJECT_NAME}_ENABLE_<SomeFortranPackage>}" STREQUAL "")
    message("-- " "NOTE: Setting ${PROJECT_NAME}_ENABLE_<SomeFortranPackage>=OFF because"
      "${PROJECT_NAME}_ENABLE_Fortran = '${${PROJECT_NAME}_ENABLE_Fortran}'")
    set(${PROJECT_NAME}_ENABLE_<SomeFortranPackage> OFF)
  endif()
endif()

In order to understand the above steps for properly querying and tweaking these ENABLE variables, one must understand how TriBITS CMake code defines and interprets variables of this type.

First, note that most of these particular ENABLE variables are not BOOL cache variables but are actually STRING variables with the possible values of ON, OFF and empty "" (see the macro set_cache_on_off_empty()). Therefore, just because the value of a <XXX>_ENABLE_<YYY> variable is defined (e.g. if (DEFINED <XXX>_ENABLE_<YYY>) ... endif()) does not mean that it has been set to ON or OFF yet (or any non-empty values that evaluates to true or false in CMake). To see if an ENABLE variable is one of these types, look in the CMakeCache.txt. If the type of the variable <XXX>_ENABLE_<YYY> is STRING and you see another variable set with the name <XXX>_ENABLE_<YYY>_-STRINGS, then it is most likely this special type of ENABLE variable with a typical default value of empty "". However, if the cache variable is of type BOOL, then it is likely a standard bool variable that is not allowed to have a value of empty "".

Second, note that the value of empty "" evaluates to FALSE in CMake if() statements. Therefore, if one just wants to know if one of these variables evaluates to true, then just use if (<XXX>_ENABLE_<YYY>) ... endif().

Third, note that TriBITS will not define cache variables for these ENABLE variables until TriBITS processes the Dependencies.cmake files on the first configure (see Full TriBITS Project Configuration). On future reconfigures, these variables are all defined (but most will have a default value of empty "" stored in the cache).

The reason the files RepositoryDependenciesSetup.cmake and ProjectDependenciesSetup.cmake are the best places to put in these tweaks is because, as shown in Full Processing of TriBITS Project Files, they get processed after all of the user input has been read (in CMake cache variables set with -D<variable>=<value> and read in from ${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE files) but before TriBITS adjusts the package enables and disables (see Package Dependencies and Enable/Disable Logic). Also, these files get processed in Reduced Package Dependency Processing as well so they get processed in all contexts where enable/disable logic is applied.

However, if one wants to tweak these variables once packages are starting to be processed (in step "For each <packageDir> in all enabled top-level packages" in Full TriBITS Project Configuration), there are fewer situations where that can be done correctly as described in the next section.

10.14   How to tweak downstream TriBITS "ENABLE" variables during package configuration

There are cases where one may need to enable or disable some feature that TriBITS may have enabled by default (such as in "Adjust package and TPLs enables and disables" in Full Processing of TriBITS Project Files) and that decision can only be made while processing a package's <packageDir>/CMakeLists.txt file and not before. (And therefore the logic for this disable cannot be performed in the ProjectDependenciesSetup.cmake or RepositoryDependenciesSetup.cmake files as described in How to check for and tweak TriBITS "ENABLE" cache variables.) Also, there are cases where it is necessary to make this change visible to downstream packages. The main example is when optional support of an upstream package in a downstream package <DownstreamPackage>_ENABLE_<UpstreamPackage> must be changed in the package's <packageDir>/CMakeLists.txt file. But there are other examples such as support for a given data-type that may impact multiple downstream packages.

When the internal configuration of a package (i.e. while processing its <packageDir>/CMakeLists.txt file) determines that an optional feature <Package>_ENABLE_<YYY> must change the value previously set (e.g. that was set automatically by TriBITS during the "Adjust package and TPLs enables and disables" stage in Full Processing of TriBITS Project Files), one cannot use a simple set() statement. Changing the value of a <Package>_ENABLE_<YYY> variable inside a package's <packageDir>/CMakeLists.txt file using a raw set(<Package>_ENABLE_<YYY> <newValue>) statement only changes the variable's value inside the package's scope, but all other packages will see the old value of <Package>_ENABLE_<YYY>. To correctly change the value of one of these variables, instead use dual_scope_set() from the top-level <packageDir>/CMakeLists.txt file. To perform this disable more robustly than calling dual_scope_set() directly, use the provided macro tribits_disable_optional_dependency(). For example, to disable optional support for <UpstreamPackage> in <DownstreamPackage> in <DownstreamPackage> package's <packageDir>/CMakeLists.txt file based on some criteria, add the CMake code:

if (<some-condition>)
  tribits_disable_optional_dependency( <UpstreamPackage>
    "NOTE: ${PACKAGE_NAME}_ENABLE_<UpstreamPackage> being set to OFF because of <reason>" )
endif()

Calling dual_scope_set() in the package's top-level <packageDir>/CMakeLists.txt file sets the value in both the local scope of <packageDir>/CMakeLists.txt (and therefore propagated to all other CMakeLists.txt files in that package) and in base-level (global) project scope. (But this does not change the value of a cache variable <Package>_ENABLE_<YYY> that may have been set by the user or some other means which is the desired behavior; see TriBITS auto-enables/disables done using non-cache local variables.) In this way, any downstream package (configured after processing <packageDir>/CMakeLists.txt) will see the new value for <Package>_ENABLE_<YYY>.

It is also strongly recommended that a message be printed to CMake STDOUT using message("-- " "NOTE: <message>") when changing the value of one of these <Package>_ENABLE_<YYY> variables. The user may have set it explicitly or TriBITS may have printed automatic logic for setting it by default, and user needs to know why and where the value is being overridden.

NOTE: However, it is not allowed to try to change the value of a global enable of a upstream or downstream package by trying to change the value of <Project>_ENABLE_<Package> or TPL_ENABLE_<Package> in a <packageDir>/CMakeLists.txt file. Changing the value of these variables after the "Adjust package and TPLs enables and disables" stage in Full Processing of TriBITS Project Files will result in undefined behavior.

10.15   How to set up multi-repository support

The following steps describe how to set up support for TriBITS project managing multiple version control and TriBITS repositories by default (see Multi-Repository Support).

1) Add file <projectDir>/cmake/ExtraRepositoriesList.cmake and list out extra repos

For example, this file would contain something like:

tribits_project_define_extra_repositories(
  ExtraRepo1  ""  GIT  git@someurl.com:ExtraRepo1  ""  Continuous
  ExtraRepo2  "ExtraRepo1/ExtraRepos2"  GIT  git@someurl.com:ExtraRepo2
                                                 NOPACKAGES  Continuous
  ExtraRepo3  ""  GIT  git@someurl.com:ExtraRepo3  ""  Nightly
  )

NOTE: If one will not be using the checkin-test.py tool, or clone_extra_repos.py tool, or the TriBITS CTest/CDash Driver system, then one can leave the REPO_VCTYPE and REPO_URL fields empty (see tribits_project_define_extra_repositories() for details). (TriBITS Core does not have any dependencies on any specific VC tool. These fields are listed here to avoid duplicating the list of repos in another file when using these additional TriBITS tools.)

2) Set default values for ${PROJECT_NAME}_EXTRAREPOS_FILE and ${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE (and possibly ${PROJECT_NAME}_IGNORE_MISSING_EXTRA_REPOSITORIES) in the file <projectDir>/ProjectName.cmake

For example, add:

set(${PROJECT_NAME}_EXTRAREPOS_FILE  cmake/ExtraRepositoriesList.cmake
  CACHE  FILEPATH  "Set in ProjectName.cmake")
set(${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE  Continuous
  CACHE  STRING  "Set in ProjectName.cmake")

to the <projectDir>/ProjectName.cmake file. Otherwise, no extra repos will be defined or processed by default when configuring the project.

And if the project can operate without all of its extra repos, the project can set the following default in this file as well with:

set(${PROJECT_NAME}_IGNORE_MISSING_EXTRA_REPOSITORIES  TRUE
  CACHE  STRING  "Set in ProjectName.cmake")

Otherwise, all of the extra repos need to be present or the project configure will fail.

3) If using git as the VC tool, then set the variable ${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE_DEFAULT in the <projectDir>/ProjectName.cmake file

For example:

set(${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE_DEFAULT  TRUE)

4) If wanting a clone tool with git repos, set up a link to the clone_extra_repos.py tool in the base <projectDir>/ directory

Create a symlink to the script clone_extra_repos.py in the base project repo, for example with:

cd <projecDir>/
ln -s cmake/tribits/ci_support/clone_extra_repos.py .
git add clone_extra_repos.py
git commit

10.16   How to submit testing results to a CDash site

The following steps describe how to submit results to a CDash site using the TriBITS CTest/CDash Driver support.

1. Create a CDash project <ProjectName> on the targeted CDash site.

To do this, one must have an account on the CDash site and the permissions to create a new CDash project. The name of the project on CDash should generally match the name of the TriBITS project PROJECT_NAME but it does not have to. In fact, one can generally submit to any CDash project with any name so creating a new CDash project is actually optional.

NOTE: For open-source projects, Kitware provides the free CDash site my.cdash.org that allows a limited number of submits and data per day. But it should be enough to test out submitting to CDash.

2. Create CTestConfig.cmake and CTestCustom.cmake.in files for the project.

• The file <projectDir>/CTestConfig.cmake can be copied and pasted from TribitsExampleProject/CTestConfig.cmake. To customize for your project, you generally just need to update the variables CTEST_DROP_SITE, CTEST_PROJECT_NAME, and CTEST_DROP_LOCATION.
• The file <projectDir>/cmake/ctest/CTestCustom.cmake.in can be copied and pasted from TribitsExampleProject/cmake/ctest/CTestCustom.cmake.in and them modified as desired.

3. Test experimental submits with make dashboard.

Once the CDash project and the <projectDir>/CTestConfig.cmake and <projectDir>/cmake/ctest/CTestCustom.cmake.in files are created, one perform an experimental submission by just configuring the project as normal (except configuring additionally with -DCTEST_BUILD_FLAGS=-j8 and -DCTEST_PARALLEL_LEVEL=8 to use parallelism in the build and testing in the ctest -S script) and then running the build target:

make dashboard

That will configure, build, test, and submit results to the CDash site to the Experimental track of the target CDash project (see Dashboard Submissions).

To work out problems locally without spamming the CDash site, one can run with:

env CTEST_DO_SUBMIT=OFF make dashboard

To submit to a different CDash site and project, change the cache vars CTEST_DROP_SITE, CTEST_PROJECT_NAME, and CTEST_DROP_LOCATION at configure time. For example, to submit TribitsExampleProject results to a different CDash site, configure with:

cmake \
-DCTEST_DROP_SITE=testing.sandia.gov/cdash \
-DCTEST_PROJECT_NAME=TribitsExProj \
-DCTEST_DROP_LOCATION="/submit.php?project=TribitsExProj" \
[other cmake options] \
<baseDir>/TribitsExamplProject

and then run make dashboard.

4. Add custom CTest -S driver scripts.

For driving different builds and tests, one needs to set up one or more CTest -S driver scripts. There are various ways to do this but a simple approach that avoids duplication is to first create a file like TribitsExampleProject/cmake/ctest/TribitsExProjCTestDriver.cmake:

#
# Set the locations of things for this project
#


set(TRIBITS_PROJECT_ROOT "${CMAKE_CURRENT_LIST_DIR}/../..")
set(CTEST_SOURCE_NAME "TribitsExampleProject")
include("${TRIBITS_PROJECT_ROOT}/ProjectName.cmake")
if (NOT "$ENV{${PROJECT_NAME}_TRIBITS_DIR}" STREQUAL "")
  set(${PROJECT_NAME}_TRIBITS_DIR "$ENV{${PROJECT_NAME}_TRIBITS_DIR}")
endif()
if("${${PROJECT_NAME}_TRIBITS_DIR}" STREQUAL "")
  # If not set externally, then assume this is inside of tribits example
  # directory.
  set(${PROJECT_NAME}_TRIBITS_DIR "${CMAKE_CURRENT_LIST_DIR}/../../../..")
endif()

#
# Include the TriBITS file to get other modules included
#

include("${${PROJECT_NAME}_TRIBITS_DIR}/ctest_driver/TribitsCTestDriverCore.cmake")

function(tribitsexproj_ctest_driver)
  set_default_and_from_env( CTEST_BUILD_FLAGS "-j1 -i" )
  set_default_and_from_env( CTEST_PARALLEL_LEVEL "1" )
  tribits_ctest_driver()
endfunction()

and then create a set of CTest -S driver scripts that uses that file. One example is the file TribitsExampleProject/cmake/ctest/general_gcc/ctest_serial_debug.cmake:

include("${CMAKE_CURRENT_LIST_DIR}/../TribitsExProjCTestDriver.cmake")

set(COMM_TYPE SERIAL)
set(BUILD_TYPE DEBUG)
set(COMPILER_VERSION GCC)
set(BUILD_DIR_NAME ${COMM_TYPE}_${BUILD_TYPE})

set( EXTRA_CONFIGURE_OPTIONS
  "-DBUILD_SHARED_LIBS:BOOL=ON"
  "-DCMAKE_BUILD_TYPE=DEBUG"
  "-DCMAKE_C_COMPILER=gcc"
  "-DCMAKE_CXX_COMPILER=g++"
  "-DCMAKE_Fortran_COMPILER=gfortran"
  "-DTribitsExProj_ENABLE_Fortran=ON"
  "-DTribitsExProj_TRACE_ADD_TEST=ON"
  )

set_default_and_from_env(TribitsExProj_CMAKE_INSTALL_PREFIX "")
if (TribitsExProj_CMAKE_INSTALL_PREFIX)
  set(EXTRA_CONFIGURE_OPTIONS
    "${EXTRA_CONFIGURE_OPTIONS}"
    "-DCMAKE_INSTALL_PREFIX=${TribitsExProj_CMAKE_INSTALL_PREFIX}"
    )
endif()

set(CTEST_TEST_TYPE Continuous)

tribitsexproj_ctest_driver()

5. Test CTest -S driver scripts

Once a CTest -S driver script (like the ctest_serial_debug.cmake example shown above) is created, one can test it locally and then test a submit to CDash. To test the exact state of the repository locally, one can create a temporary base directory, symbolically link in the local project source directory, and then run the CTest -S script by setting CTEST_DO_SUBMIT=OFF. For example, the TribitsExampleProject CTest -S script can be run and tested locally by doing:

$ mkdir MOCK_TRIBITSEXPROJ_SERIAL_DEBUG
$ cd MOCK_TRIBITSEXPROJ_SERIAL_DEBUG/
$ ln -s $TRIBITS_DIR/examples/TribitsExampleProject .
$ env  CTEST_DASHBOARD_ROOT=$PWD \
     CTEST_DROP_SITE=testing.sandia.gov/cdash \
     CTEST_PROJECT_NAME=TribitsExProj \
     CTEST_DROP_LOCATION="/submit.php?project=TribitsExProj" \
     CTEST_TEST_TYPE=Experimental \
     CTEST_DO_SUBMIT=OFF \
     CTEST_DO_UPDATES=OFF \
     CTEST_START_WITH_EMPTY_BINARY_DIRECTORY=TRUE \
     ctest -V -S \
      $TRIBITS_DIR/examples/TribitsExampleProject/cmake/ctest/general_gcc/ctest_serial_debug.cmake \
     &> console.out

where TRIBITS_DIR is an env var that points to the location of the TriBITS/tribits directory on the local machine (and the location of the CDash site and project is changed, since the free my.cdash.org site can only accept as small number of builds each day).

Once that CTest -S driver script is working correctly without submitting to CDash, the above ctest -S command can be run with CTEST_DO_SUBMIT=ON which submit the CDash and then print the location on CDash for the submitted configure, build, and test results.

6. Set up automated runs of CTest -S driver scripts

The custom CTest -S driver scripts created above can be run and used to submit to CDash in a variety of ways:

• Cron jobs can be set up to run them at the same time every day.
• Jenkins jobs can be set up to run them based on various criteria.
• GitHub Actions can run them to respond to GitHub pushes or to test pull requests.
• GitLab CI can run them to respond to GitLab pushes or the test merge requests.
• Use the legacy TriBITS Dashboard Driver system (not recommended).

The setup of Jenkins, GitHub Actions, GitLab CI and other more sophisticated automated testing systems will not be described here. What will be briefly outlined below is the setup using cron jobs on a Linux machine. That is sufficient for most smaller projects and provides tremendous value.

To set up an automated build using a cron job, one will typically create a shell driver script that sets the env and then calls the ctest -S <script> command. Then one just adds a call to that shell driver script using crontab -e. That is about all there is to it.

10.17   How to submit testing results to a custom CDash Group

Following up on How to submit testing results to a CDash site, to submit build and test results to a custom "Group" on CDash (instead of just "Nightly", "Continuous" or "Experimental"), one just has to create the new group on CDash using the CDash GUI interface and then tell the ctest -S local driver to submit results to that CDash group. The steps for doing this are given below.

1. Create the new CDash group <special_group> for CDash project <ProjectName> on the targeted CDash site.

   If the CDash group <special_group> is not already created, then one can create it by first logging into CDash with an account that can modify the CDash project <ProjectName>. Once logged in, go to the project edit page and select "Settings" and "Groups". From there, create the new group <special_group>. Set the "Build Type" to either "Daily" or "Latest".

2. Set ${PROJECT_NAME}_TRACK=<special_group> with the CTest -S driver script.

   One can either do that by setting set(${PROJECT_NAME}_TRACK <special_group>) in the CTest -S *.cmake driver script itself or can set it in the environment when running the ctest -S driver script. For example:

   $ env <Project>_TRACK=<special_group> ... \
     ctest -V -S <ctest_driver>.cmake
   

   If the "build type" for the CDash group <special_group> was set to "Daily", then set CTEST_TEST_TYPE to Nightly. Otherwise, CTEST_TEST_TYPE can be set to Continuous or Experimental.

11.1   TriBITS Repository Contents

The TriBITS git repository is organized as a TriBITS Project, TriBITS Repository, and TriBITS Package all in the same base directory. The base contents are described in the file:

TriBITS/README.DIRECTORY_CONTENTS.rst

The part of TriBITS that is installed or snapshotted in contained in the subdirectory TriBITS/tribits/ and is described in the following section.

ProjectName.cmake   # PROJECT_NAME=TriBITS
CMakeLists.txt      # PROJECT_NAME = PACKAGE_NAME = TriBITS
PackagesList.cmake  # Lists just "TriBITS . PT"
TPLsList.cmake      # Lists only MPI
cmake/              # Dependencies.cmake, etc.

11.2   TriBITS System Project Dependencies

The core TriBITS system itself (see tribts/core/ in TriBITS/tribits/) which is used to configure, built, test, create tarballs, and install software has no dependencies other than a basic installation of CMake (which typically includes the executables cmake, ctest, and cpack). Great effort has been expended to implement all of this core functionality of TriBITS just using raw CMake. That means that anyone who needs to configure, build, and install software that uses TriBITS just needs a compatible CMake implementation. CMake is becoming iniquitous enough that many machines will already have a current-enough version of CMake installed by default on their systems and therefore no one will need to download or install any extra software when building and installing a project that uses TriBITS (assuming the necessary compilers etc. required by the project are also installed). If a current-enough version of CMake is not installed on a given system, it is easy to download the source code and all it needs is a basic C++ compiler to build and install.

However, note that a specific TriBITS project is free to use any newer CMake features it wants and therefore these projects will require newer versions of CMake than what is required by TriBITS (see discussion of cmake_minimum_required() in <projectDir>/CMakeLists.txt). But also note that specific TriBITS projects and packages will also require additional tools like compilers, Python (see Python Support), Perl, or many other such dependencies. It is just that TriBITS itself does not require any of these in order to perform the basic configure, build, test, and install of software. The goal of TriBITS is not to make the portability of software that uses it any worse than it already is but instead to make it easier in most cases (that after all is the whole goal of CMake).

While the TriBITS Core functionality to configure, build, test, and install software is written using only raw CMake, the more sophisticated development tools needed to implement the full TriBITS development environment require Python 2.7 (or higher including Python 3.x) (see Python Support). Python is needed for tools like checkin-test.py and gitdist. In addition, these python tools are used in tribits_ctest_driver() to drive automated testing and submits to CDash. Also note that git is the chosen version control tool for the TriBITS software development tools and all the VC-related functionality in TriBITS. But none of this is required for doing the most basic building, testing, or installation of a project using TriBITS Core.

11.3   Python Support

TriBITS Core does not require anything other than raw CMake. However, Python Utils, TriBITS CI Support, and other extended TriBITS components require Python. These extra TriBITS tools only require Python 3.6+. By default, when a TriBITS project starts to configure using CMake, it will try to find Python 3.6+ on the system (see Full Processing of TriBITS Project Files). If Python is found, it will set the global cache variable Python3_EXECUTABLE. If it is not found, then it will print a warning and Python3_EXECUTABLE will be empty. With this default behavior, if Python is found, then the TriBITS project can use it. Otherwise, it can do without it.

While the default behavior for finding Python described above is useful for many TriBITS projects, some TriBITS projects need different behavior such as:

1. The TriBITS project may not ever use Python so there is no need to look for it at all. In this case, the TriBITS project would set ${PROJECT_NAME}_USES_PYTHON to FALSE.
2. Some TriBITS projects require Python and should not even configure if it can't be found. In this case, the TriBITS project would set ${PROJECT_NAME}_REQUIRES_PYTHON to TRUE.
3. Some TriBITS projects may require a version of Python more recent than 3.6. In this case, the TriBITS project would set ${PROJECT_NAME}_Python3_FIND_VERSION to some value higher than 3.6. For example, may newer systems have higher versions of Python installed by default, and projects developed on such a system typically require this version or higher.

11.4   Project-Specific Build Reference

If a project that uses TriBITS is going to have a significant user base that will configure, build, and test the project, then having some documentation that explains how to do this would be useful. For this purpose, TriBITS provides a mechanism to quickly create a project-specific build reference document in restructured text (RST) format and with HTML and LaTeX/PDF outputs. This document are generally created in the base project source tree and given then name <Project>BuildReference.[rst,html,pdf]. This document consists of two parts. One part is a generic template document:

tribits/doc/TribitsBuildReferenceBody.rst

provided in the TriBITS source tree that uses the place-holder <Project> for the for the real project name. The second part is a project-specific template file:

<projectDir>/cmake/<Project>BuildReferenceTemplate.rst

which provides the outer RST document (with title, authors, abstract, introduction, other introductory sections). From these two files, the script:

tribits/doc/build_ref/create-project-build-quickref.py

is used to replace <Project> in the TribitsBuildReferenceBody.rst file with the real project name (read from the project's ProjectName.cmake file by default) and then generates the read-only files:

<projectDir>/
  <Project>BuildReference.rst
  <Project>BuildReference.html
  <Project>BuildReference.pdf

For a simple example of this, see:

tribits/doc/build_ref/create-build-ref.sh

A project-independent version of this file is provided in the TribitsBuildReference.[rst,html,pdf] which is referred to many times in this developers guide.

11.5   Project and Repository Versioning and Release Mode

TriBITS has built-in support for project and repository versioning and release-mode control. When the project contains the file <projectDir>/Version.cmake, it is used to define the project's official version. The idea is that when it is time to branch for a release, the only file that needs to be changed is the file <projectDir>/Version.cmake.

Each TriBITS repository can also contain a <repoDir>/Version.cmake file that sets version-related variables which TriBITS packages in that repository can use to derive development and release version information. If the TriBITS repository also contains a <repoDir>/Copyright.txt file, then the information in <repoDir>/Version.cmake and <repoDir>/Copyright.txt are used to configure a repository version header file:

${${REPOSITORY_NAME}_BINARY_DIR}/${REPOSITORY_NAME}_version.h

The configured header file ${REPOSITORY_NAME}_version.h defines C pre-processor macros that give the repository version number in several formats, which allows C/C++ code (or any software that uses the C preprocessor) to write conditional code like:

#if Trilinos_MAJOR_MINOR_VERSION > 100200
  /* Contains feature X */
  ...
#else
  /* Does not contain feature X */
  ...
#endif

Of course when the TriBITS project and the TriBITS repository are the same directory, the <projectDir>/Version.cmake and <repoDir>/Version.cmake files are the same file, which works just fine.

11.6   TriBITS Environment Probing and Setup

Part of the TriBITS Framework is to probe the environment, set up the compilers, and get ready to compile code. This was mentioned in Full Processing of TriBITS Project Files. This is executed by the TriBITS macro tribits_setup_env(). Some of the things this macro does are:

Probe and set up the environment:

• Set CMAKE_BUILD_TYPE default (if not already set)
• Set up for MPI (MPI compilers, etc.)
• Set up C, C++, and Fortran compilers using enable_language(<LANG>)
• include( <projectDir>/cmake/ProjectCompilerPostConfig.cmake )
• Find Perl (sets PERL_EXECUTABLE)
• Determine mixed language C/Fortran linking
• Set up OpenMP (with find_package(OpenMP))
• Set up optional Windows support
• Find Doxygen (sets DOXYGEN_EXECUTABLE)
• Perform some other configure-time tests (see cmake configure output)

At the completion of this part of the processing, the TriBITS CMake project is ready to compile code. All of the major variables set as part of this process are printed to the cmake stdout when the project is configured.

11.7   Tricky considerations for TriBITS-generated <tplName>Config.cmake files

An issue that comes up with external packages/TPLs like HDF5 that needs to be discussed here is the fact that FindTPL<tplName>.cmake module files create (See How to add a new TriBITS TPL) and TriBITS installs package config files of the name <tplName>Config.cmake. These TriBITS-generated package config files <tplName>Config.cmake could potentially be found by calls to find_package(<externalPkg>) (i.e. when <tplName> == <externalPkg> like with HDF5). These TriBITS-generated <tplName>Config.cmake files are primarily meant to provide a TriBITS-compliant external package for downstream TriBITS-compliant <Package>Config.cmake files. These TriBITS-generated <tplName>Config.cmake files will usually not behave the same way existing Find<tplName>.config find modules or native <tplName>Config.cmake package config files would behave as expected by downstream projects when found by find_package(<tplName>) commands called in some arbitrary downstream raw CMake project. Therefore, to avoid having an installed TriBITS-generated HDF5Config.cmake file, for example, being found by the inner call to find_package(HDF5 ...) in the file FindTPLHDF5.cmake (which could be disastrous), TriBITS employs two safeguards.

First, TriBITS-generated <tplName>Config.cmake package config files are placed into the build directory under:

<buildDir>/external_packages/<tplName>/<tplName>Config.cmake

and installed into the installation directory under:

<installDir>/lib/external_packages/<tplName>/<tplName>Config.cmake

so they will not be found by find_package(<tplName>) by default when <buildDir>/cmake_packages and/or <installDir>, respectively, are added to CMAKE_PREFIX_PATH.

Second, even if the directories <installDir>/lib/external_packages or <buildDir>/external_packages do get added to the search path somehow (e.g. by appending those to CMAKE_INSTALL_PREFIX), the companion TriBITS-generated <tplName>ConfigVersion.cmake files will set PACKAGE_VERSION_COMPATIBLE=OFF and result in find_package(<tplName>) not selecting the TriBITS-generated <tplName>Config.cmake file. (It turns out that CMake's find_package(<Package>) command always includes the file <Package>ConfigVersion.cmake, even if no version information is passed to the command find_package(<Package>). This allows special logic to be placed in the file <Package>ConfigVersion.cmake to determine if find_package(<Package>) will select a given <Package>Config.cmake file that is in the search path based on a number of different criteria such as in this case.)

11.8   Installation considerations

For the most part, installation is pretty straightforward with a TriBITS-based CMake project. TriBITS automatically puts in appropriate default install() commands to install header files, libraries, executables, and other commonly installed artifacts (such as TriBITS-autogenerated <Package>Config.cmake files). And packages can add their own custom install() commands to install items under CMAKE_INSTALL_PREFIX (or the subdirs under CMAKE_INSTALL_PREFIX mentioned in Setting the install prefix). However, there are some special situations that need to be addressed and some tweaks to built-in CMake support that need to be made.

One issue that can occur is that there are cases where a Unix/Linux system is set up not to honor the group sticky bit and therefore one cannot control what group owns the created installed files and directories (i.e. the default group will be used). Also, there are cases were one cannot easily control the default file or directory creation permissions using umask. And there are cases where one would like to recursively install a set of directories and files where some of these files may be scripts that need to have the execute permission set on them for them to work. The only to flexible accomplish that with CMake (if one does not know the exist list of those files or extensions of those files) is to pass in the SOURCE_PERMISSIONS option to the install(DIRECTORY ...) command. An example of this is shown in:

• TribitsExampleProject/packages/with_subpackages/b/CMakeLists.txt

that has:

install( DIRECTORY "${CMAKE_CURRENT_LIST_DIR}/stuff"
  DESTINATION "${CMAKE_INSTALL_PREFIX}/share/${PACKAGE_NAME}"
  USE_SOURCE_PERMISSIONS PATTERN "*~" EXCLUDE )

In this case, CMake will preserve the execute permission on any of the scripts contained under the stuff/ subdirectory but group and other permissions will not be set based on umask or the default CMake install permissions. Instead, these permissions are set based on the source directory permissions (which is often set to 700 or rwx------).

To address cases like this, TriBITS can automatically run chgrp and chmod on the created files and directories that are created during the install target as described in Setting install ownership and permissions. This is completely automatic and requires nothing for the TriBITS Project developers to do to enable support for this (other than to note the below warning).

WARNING: Do not add any install() commands after the tribits_project() command completes. Otherwise, any extra files or directories will not have their group and permissions fixed by these special TriBITS-added chgrp and chmod commands run at install time. Instead, try to put all install() commands inside of a package's <packageDir>/CMakeLists.txt file. Currently, there really is no good place to add repo-level or project-level install() commands. But if one had to sneak them in, they could add various install() commands to files like <projectDir>/CMakeLists.txt (before the tribits_project_() command), <repoDir>/cmake/CallbackSetupExtraOptions.cmake, <projectDir>/cmake/CallbackDefineProjectPackaging.cmake and/or <repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake. (Note that install commands from the former two files are run before install commands for the enabled packages while install commands from the latter two files are run after.)

One can also change what compilers are written into the generated <Project>Config.cmake and <Package>Config.cmake files for the build and the install trees. By default, the compilers pointed to in these Config.cmake files will be CMAKE_<LANG>_COMPILER where <LANG> = CXX, C, and Fortran. But one can change this by setting any of the following:

set(CMAKE_CXX_COMPILER_FOR_CONFIG_FILE_BUILD_DIR <path>)
set(CMAKE_C_COMPILER_FOR_CONFIG_FILE_BUILD_DIR <path>)
set(CMAKE_Fortran_COMPILER_FOR_CONFIG_FILE_BUILD_DIR <path>)
set(CMAKE_CXX_COMPILER_FOR_CONFIG_FILE_INSTALL_DIR <path>)
set(CMAKE_C_COMPILER_FOR_CONFIG_FILE_INSTALL_DIR <path>)
set(CMAKE_Fortran_COMPILER_FOR_CONFIG_FILE_INSTALL_DIR <path>)

before the Config.cmake files are generated. These can also be set in the CMake cache using, for example, -DCMAKE_CXX_COMPILER_FOR_CONFIG_FILE_INSTALL_DIR:FILEPATH=<path>.

This is used, for example, when compiler wrappers are used for the build tree and are set to CMAKE_<LANG>_COMPILER but when one wants to point to the original underlying compilers for the installed Config.cmake files.

11.9   RPATH Handling

As explained in Setting install RPATH, TriBITS changes the CMake defaults to write in the RPATH for shared libraries and executables so that they run right out of the install directory without needing to set paths in the environment (e.g. LD_LIBRARY_PATH). However, these defaults can be changed by changing setting different project defaults for the variables ${PROJECT_NAME}_SET_INSTALL_RPATH and CMAKE_INSTALL_RPATH_USE_LINK_PATH. But most projects should likely keep these defaults in place since they make it so that doing builds and installations on a single machine work correctly by default out of the box. For other installation/distribution use cases, the user is told how to manipulate CMake variables for those cases in Setting install RPATH.

11.10   Configure-time System Tests

CMake has good support for defining configure-time checks of the system to help in configuring the project. One can check for whether a header file exists, if the compiler supports a given data-type or language feature, or perform almost any other type of check that one can imagine that can be done using the configured compilers, libraries, system tools, etc. An example was given in TribitsExampleProject. Just follow that example, look at some of the built-in CMake configure-time test modules, and consult provided on-line CMake documentation in order to learn how to create a configure-time test for almost anything.

11.11   Creating Source Distributions

The TriBITS system uses CMake's built-in CPack support to create source distributions in a variety of zipped and tarred formats. (Note that the term "source tarball" or just "tarball" may be used below but should be interpreted as "source distribution".) TriBITS will automatically add support for CPack when the variable ${PROJECT_NAME}_ENABLE_CPACK_PACKAGING is set to ON. The commands for creating a source distribution are described in Creating a tarball of the source tree using the built-in package_source build target. The value added by TriBITS is that TriBITS will automatically exclude the source for any defined packages that are not enabled and TriBITS provides a framework for systematically excluding files and directories from individual repositories and packages. In addition, the source for non-enabled subpackages can also be excluded depending on the value of ${PROJECT_NAME}_EXCLUDE_DISABLED_SUBPACKAGES_FROM_DISTRIBUTION. All of this allows one to create distributions which only includes subsets of a larger project (even a single package in some cases).

Unlike other build systems (like autotools), CMake will put EVERYTHING into the source distribution (e.g. tarball) that is sitting in the source tree by default. Therefore, setting up for a source distribution usually means deciding what extra files and directories should be excluded. Beyond the directories for non-enabled packages, further files can be selected to be excluded on a package-by-package based and at the repository level (see below).

Individual packages can list additional files/directories under the package's source tree to be excluded from the project's source distribution using a call to tribits_exclude_files() in their <packageDir>/CMakeLists.txt file. Note that if the package is not enabled, these excludes will never get added! That is no problem if these excludes only apply to the given package since TriBITS will add an exclude for the entire package but is a problem if a package lists excludes for files outside of the package's source tree.

Additional files and entire directories can also be excluded at the repository level by listing them in the <repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake file which typically just appends the built-in CMake variable CPACK_SOURCE_IGNORE_FILES. However, if a repository is not processed, this file is never processed and therefore no files will be excluded from a repository sitting in the source tree that is not processed (see below).

There are a number of project-level settings that need to be defined and these are specified in the file <projectDir>/cmake/CallbackDefineProjectPackaging.cmake.

The TribitsExampleProject is set up for creating source distributions and this is demonstrated in one of the tests defined in:

TriBITS/test/core/ExamplesUnitTests/CMakeLists.txt

There are a few points of caution to note about creating source distributions.

NOTE: It is worth stressing again that EVERY file that is in the source tree will be included in the source distribution (tarball) unless there is an exclude regex matching it appended to the variable CPACK_SOURCE_IGNORE_FILES. TriBITS can only add excludes for defined non-enabled packaged. Every other file listed in the source tree will be included in the tarball.

NOTE: The entries in CPACK_SOURCE_IGNORE_FILES are interpreted as REGULAR EXPRESSIONS not globs so if you add "someFile.*" as an exclude, it will exclude every file in the entire source tree that has "someFile" in the name! This is because, in regex terminology, the trailing ".*" means "match any character zero or more times" and "someFile" can match anywhere in the file name path. Also, note that if you add in an exclude like "*.pyc" (i.e. trying to exclude all of the generated Python byte code files) that it will exclude every file that has "pyc" in the name and not just those with the file extension "pyc". For example, the exclude ".pyc" would exclude the files "puppyc", "lpycso", etc. If you want to exclude all files with extension "pyc", you have to add the exclude regex ".*[.]pyc$"! One's lack of understanding of this fact will cost someone hours of lost time debugging what happens when random files are missing when one tries to configure what is left. Sometimes, what is left will actually configure and might almost build!

NOTE: As warned in TriBITS Package Core Files and TriBITS Subpackage Core Files, Packages must have directories that are strictly independent of the directories of other packages. If they don't, then the source directory for an enabled package will get excluded from the source distribution if its directory is under the directory of a package that is not enabled. For example, if PackageA is enabled but its package directory packageb/packagea/ is under the package directory packageb/ for the disabled package PackageB, then every file and directory under packageb/ will be excluded from the source distribution (tarball), including everything under packageb/packagea/! It would be too expensive to put in an automated check for cases like this so package developers should just take care not to nest the directories of packages inside of each other to avoid problems like this.

NOTE: Extra repositories that are sitting in the source tree but not processed by TriBITS for some reason (e.g. due to explicitly listing in the variable ${PROJECT_NAME}_EXTRA_REPOSITORIES only a subset of the repositories listed in <projectDir>/cmake/ExtraRepositoriesList.cmake ) will get added to the source distribution in full by default even though there are no enabled packages from these repos. These non-processed repo dirs are like any other random directory sitting in the source tree, they will get copied over into the source distribution!.

NOTE: When debugging tarball creation problems, always configure with the variable <Project>_DUMP_CPACK_SOURCE_IGNORE_FILES=ON. If you don't see a regex listed for the file or directory you expect to be excluded, then that file/directory it will be included in the source distribution!

11.12   Using Git Bisect with checkin-test.py workflows

There are cases where a customer will do an update of an upstream project from a git repo and then find out that some feature or behavior is broken with respect to their usage. This can happen even if the upstream project's own test suite passes all of its tests. Depending on the situation, there may be hundreds to thousands of commits between the last known "good" working version of the code and pulled "bad" version. To help customers find the first commit that contains the changes which are causing the breakage, git supplies git bisect. This set of commands does a binary search of the commits in the range <good-sha>..<bad_sha> and finds the first commit that is "bad" (or a range of commits which contains the first "bad" commit if commits are skipped, as described below).

But the git bisect commands require that all of the commits in the range <good-sha>..<bad_sha> be complete commits that provide full working versions of the upstream project. However, many beginning git developers and even many experienced developers don't always create individual git commits that build and pass all of the upstream project's tests and therefore can create false "bad" commits during the binary search. This can happen when developers create intermediate "check-point" commits during the development process but did not squash the intermediate commits together to create cohesive working commits. This can also happen when experienced developers have a set of what they believe are all working commits but do not actually test all of the commits to know that they pass all of the upstream project's tests before pushing these commits to the main development branch. This lack of detailed testing of each and every individual commit can give rise to false "bad" commits which will result in git bisect reporting the wrong first "bad" commit.

Projects that use the checkin-test.py tool to push sets of commits to the main development branch have an advantage in the usage of git bisect. This is because the default mode of the checkin-test.py script is to amend the top commit message with a summary of what was tested and therefore marks a set of commits that are known to have more complete testing. For example, the checkin-test.py tool amends the top commit (after the final pull and rebase by default) as shown in the following Trilinos commit:

commit 71ce56bd2d268922fda7b8eca74fad0ffbd7d807
Author: Roscoe A. Bartlett <bartlettra@ornl.gov>
Date:   Thu Feb 19 12:04:11 2015 -0500

    Define HAVE_TEUCHOSCORE_CXX11 in TeuchosCore_config.h

    This makes TeuchosCore a good example for how Trilinos (or any TriBITS)
    subpackages should put in an optional dependency on C++11.

    Build/Test Cases Summary
    Enabled Packages: TeuchosCore
    Disabled Packages: [...]
    0) MPI_DEBUG => passed: passed=44,notpassed=0 (2.61 min)
    1) SERIAL_RELEASE => passed: passed=43,notpassed=0 (1.08 min)

Therefore, these special known-tested commits can be flagged by grepping the git log -1 HEAD output for the string "Build/Test Cases Summary". By bisecting on these commits, one has a lower chance of encountering false "bad" commits and has a higher chance of finding a smaller range of commits where the first true "bad" commit might be found. To aid in performing git bisect and only checking checkin-test.py-tested commits, the tool is_checkin_tested_commit.py is provided.

To demonstrate how the is_checkin_tested_commit.py tool can be used with git bisect, suppose that someone writes a customized script build_and_test_customer_code.sh that will build the upstream project and the downstream customer's code and then run a set of tests to see if the "bad" behavior seen by the customer code is the current HEAD version of the upstream project. Assume this script is copied into the upstream project's local git repo using:

$ cd <upstream-repo>/
$ cp ~/build_and_test_customer_code.sh .
$ cat /build_and_test_customer_code.sh >> .git/info/exclude

Now, one could use build_and_test_customer_code.sh directly with:

$ git bisect run ./build_and_test_customer_code.sh

but that would result in testing all the commits which may have a high chance of producing false "bad" commits as described above and fail to correctly bracket the location of the true first "bad" commit.

So instead, one can write a filtered testing script safe_build_and_test_customer_code.sh which calls is_checkin_tested_commit.py and build_and_test_customer_code.sh as follows:

#!/bin/bash
#
# Script: safe_build_and_test_customer_code.sh

$TRIBITS_DIR/ci_support/is_checkin_tested_commit.py
IS_CHECKIN_TESTED_COMMIT_RTN=$?
if [ "$IS_CHECKIN_TESTED_COMMIT_RTN" != "0" ] ; then
  exit 125 # Skip the commit because HEAD is not known to be tested!
fi

./build_and_test_customer_code.sh  # Rtn 0 "good", or [1, 124] if "bad"

The above test script safe_build_and_test_customer_code.sh will skip the testing of commits that are not marked by the checkin-test.py tool.

To demonstrate how to use the is_checkin_tested_commit.py script with git bisect, an example from Trilinos is used below. (Note that the current Trilinos public repository may have been filtered so the commit SHA1s shown below may not match what is in the current Trilinos repository. But one can use the commit summary message, author, and author date to find the updated SHA1s and then to update this example for the current repository.)

Consider a scenario where a customer application updates Trilinos from the commit:

d44c17d "Merge branch 'master' of software.sandia.gov:/space/git/Trilinos"
Author: Roscoe A. Bartlett <xxx@ornl.gov>
Date:   Tue May 26 12:43:25 2015 -0400

to the commit:

605b91b "Merge branch 'master' of software.sandia.gov:/git/Trilinos"
Author: Vitus Leung <xxx@sandia.gov>
Date:   Tue Sep 29 20:18:54 2015 -0600

and it is found that some critical feature broke or is not behaving acceptably for the customer code (but all of the tests for Trilinos pass just fine). This range of commits d44c17d..605b91b gives 2257 commits to search as shown by:

$ cd Trilinos/
$ git log-oneline d44c17d..605b91b | wc -l
2257

However, as described above, it is likely that doing git bisect on that full set of 2257 commits may result in hitting false "bad" commits and therefore result in a false bracketing of the first "bad" commit. This is where the usage of the checkin-test.py tool helps which is used by many (but not currently all) Trilinos developers to push changes to the Trilinos 'master' branch in the current single-branch workflow. The commits marked with the checkin-test.py tool are known (with some caveats mentioned below) to be working commits and for this the range of commits d44c17d..605b91b yields 166 commits as shown by:

$ git log-oneline --grep="Build/Test Cases Summary" d44c17d..605b91b | wc -l
166

That is an average of 2257/166 = 13.6 commits between commits pushed with the checkin-test.py tool. So bisecting on just the commits marked by checkin-test.py should bound the "bad" commit in a set of 13.6 commits on average. Bisecting on this set of 166 commits should likely give no false “bad” commits, and therefore result in the correct bracketing of the first "bad" commit.

Using the safe_build_and_test_customer_code.sh shown above, one would search for the first bad commit over this range using:

$ git bisect start 605b91b d44c17d
$ env DUMMY_TEST_COMMIT_BAD_SHA=83f05e8 \
    time git bisect run ./safe_build_and_test_customer_code.sh

and this would return the range of commits that contains the first "bad" commit (listed at the end of git bisect log output, see example below).

To provide a concrete example, suppose the commit that first introduced the problem in the range of commits d44c17d..605b91b was:

83f05e8 "MueLu: stop semi-coarsening if no z-layers are left."
Author: Tobias Wiesner <tawiesn@sandia.gov>
Date:   Wed Jul 1 14:54:20 2015 -0600

And instead of using the script safe_build_and_test_customer_code.sh, we use a dummy driver script dummy_test_commit.sh to simulate this which is provided in the set of TriBITS documentation:

$TRIBITS_DIR/doc/developers_guide/scripts/dummy_test_commit.sh

as:

#!/bin/bash
#
# Script: dummy_test_commit.sh
#
# This script simulates a test script used with 'git bisect run <script>' to
# show how to use the is_checkin_tested_commit.py script to skip commits that
# are not known to be tested with the checkin-test.py script.  To use this
# script, set the env variable DUMMY_TEST_COMMIT_BAD_SHA to the SHA1 of a
# commit that you are pretending is the bad commit in the range of commits
# <good-commit>..<bad-commit> and then run:
#
#   git bisect <bad-commit> <good-commit>
#   git bisect run ./dummy_test_commit.sh
#
# This should result in git-bisect bounding the commits around
# $DUMMY_TEST_COMMIT_BAD_SHA.  To see the sorted set of commits containing the
# first bad commit, run:
#
#   git bisect log | grep "possible first bad commit"
#

LOG_DUMMY_COMMIT=`git log --oneline HEAD ^$DUMMY_TEST_COMMIT_BAD_SHA^`
if [ "$LOG_DUMMY_COMMIT" == "" ] ; then
  echo "Commit is *before* bad commit $DUMMY_TEST_COMMIT_BAD_SHA!"
else
  echo "Commit is or after *after* bad commit $DUMMY_TEST_COMMIT_BAD_SHA!"
fi    

# Skip the commit if not tested with checkin-test.py script
$TRIBITS_DIR/ci_support/is_checkin_tested_commit.py
IS_CHECKIN_TESTED_COMMIT_RTN=$?
if [ "$IS_CHECKIN_TESTED_COMMIT_RTN" != "0" ] ; then
  exit 125 # Skip the commit because it was not known to be tested!
fi

echo "Building the current version ..."
if [ "$LOG_DUMMY_COMMIT" == "" ] ; then
echo "Commit is *before* bad commit $DUMMY_TEST_COMMIT_BAD_SHA! so marking good!"
  exit 0
else
  echo "Commit is or *after* bad commit $DUMMY_TEST_COMMIT_BAD_SHA so marking bad!"
  exit 1
fi    

This driver script allows one to simulate the usage of git bisect to understand how it works without having to actually build and test code. It is a useful training and experimentation tool.

Using git bisect (with git version 2.1.0) over the range of commits d44c17d..605b91b searching for the first "bad" commit is done by running the commands:

$ git bisect start 605b91b d44c17d
$ env DUMMY_TEST_COMMIT_BAD_SHA=83f05e8 \
    time git bisect run \
    $TRIBITS_DIR/doc/developers_guide/scripts/dummy_test_commit.sh \
    &> ../git_bisect_run.log
$ git2 bisect log &> ../git_bisect_log.log
$ cat ../git_bisect_log.log | grep "possible first bad commit" | \
    sed "s/possible first bad commit://g"  | sed "s/[a-z0-9]\{30\}\]/]/g"
$ git bisect reset

This set of commands yield the output:

Bisecting: 1128 revisions left to test after this (roughly 10 steps)
[9634d462dba77704b598e89ba69ba3ffa5a71471] Revert "Trilinos: remove _def.hpp [...]"

real  1m22.961s
user  0m57.157s
sys   3m40.376s

#  [165067ce53] MueLu: SemiCoarsenPFactory. Use strided maps to properly transfer [...]
#  [ada21a95a9] MueLu: refurbish LineDetectionFactory
#  [83f05e8970] MueLu: stop semi-coarsening if no z-layers are left.

Previous HEAD position was 83f05e8... MueLu: stop semi-coarsening if no z-layers are left.
Switched to branch 'master'

This output shows the dummy bad commit 83f05e8 in a set of just 3 commits, bounded in the set of commits 8b79832..165067c:

165067c "MueLu: SemiCoarsenPFactory. Use strided maps to properly [...]."
Author: Tobias Wiesner <tawiesn@sandia.gov>
Date:   Thu Jul 2 12:11:24 2015 -0600

8b79832 "Ifpack2: RBILUK: adding additional ETI types"
Author: Jonathan Hu <jhu@sandia.gov>
Date:   Thu Jul 2 14:17:40 2015 -0700

The set of commits that were actually tested by git bisect run <script> is shown by:

$ cat ../git_bisect_log.log | grep "\(good:\|bad:\)" | sed "s/[a-z0-9]\{30\}\]/]/g"
# bad: [605b91b012] Merge branch 'master' of software.sandia.gov:/git/Trilinos
# good: [d44c17d5d2] Merge branch 'master' of software.sandia.gov:/space/git/Trilinos
# good: [7e13a95774] Ifpack2: If the user does not provide the bandwidth of the banded [...]
# bad: [7335d8bc92] MueLu: fix documentation
# bad: [9997ecf0ba] Belos::LSQRSolMgr: Fixed bug in setParameters.
# bad: [b6e0453224] MueLu: add a nightly test for the combination of semicoarsening [...]
# bad: [165067ce53] MueLu: SemiCoarsenPFactory. Use strided maps to properly [...]
# good: [3b5453962e] Ifpack2: Nuking the old ETI system
# good: [8b79832f1d] Ifpack2: RBILUK: adding additional ETI types

This is only 9 commits out of the possible set of 166 checkin-test.py marked commits which is out of the total set of 2257 possible commits. With just 9 build/test cycles, it bounded the first "bad" commit in a set of 3 commits in this case. And it does not matter how sloppy or broken the intermediate commits are in Trilinos. All that matters is the usage of the checkin-test.py tool (another motivation for the usage of the checkin-test.py tool, see Pre-push Testing using checkin-test.py for others as well).

Note that above, we grep the output from git bisect log for the set of possible "bad" commits instead of just looking at the output from the git bisect run <script> command (which also lists the set of possible "bad" commits). This is because the direct output from the git bisect run <script> command (shown in the log file git_bisect_run.log) shows the set of possible bad commits at the end of the output but they are unsorted and give no other git commit information:

There are only 'skip'ped commits left to test.
The first bad commit could be any of:
83f05e89706590c4b384dd191f51ef4ab00ce9bb
ada21a95a991cd238581e5a6a96800d209a57924
165067ce538af2cd0bd403e2664171726ec86f3f
We cannot bisect more!
bisect run cannot continue any more

The problem with unsorted commits is that it is difficult to use an unsorted set to do further bisection. However, the output of the set of commits from git bisect log is sorted and also shows the commit summary message and therefore is much more useful. (But note that older versions of git don’t show this set of commits at the end of git bisect log so make sure and use an updated version of git, preferably >= 2.1.0.)

Now that one has the range of possible "bad" commits (just 3 in this example) doing a further manual bisection or even manual inspection of these commits may be enough to find the change that is causing the problem for the downstream customer application.

Without the usage of the checkin-test.py tool, one would not have an automated way to ensure that git bisect avoids false "bad" commits. This allows for less experienced developers to create commits and push to the main development branch but still ensure effective usage of git bisect. (This is another example where automated tools in TriBITS help to overcome lacking developer experience and discipline.)

11.13   Multi-Repository Almost Continuous Integration

The checkin-test.py tool can be used to the implement staged integration of the various repositories in a multi-repo TriBITS project (see Multi-Repository Support) . This is referred to here as Almost Continuous Integration (ACI). The basic concept of Almost Continuous Integration (ACI) is defined and described in the paper [Integration Strategies for CSE, 2009].

This topic is broken down into the following subsections:

• ACI Multi-Git/TriBITS Repo Integration Example
• ACI Local Sync Git Repo Setup
• ACI Integration Build Directory Setup
• ACI Sync Driver Script
• ACI Cron Job Setup
• Addressing ACI Failures and Summary

BaseProj/
    ExtraRepo1
    ExtraRepo2

url4.gov:/git/BaseProj
url4.gov:/git/ExtraRepo1
url4.gov:/git/ExtraRepo2

$ cd $SYNC_BASE_DIR
$ git clone url4.gov:/git/BaseProj
$ cd BaseProj
$ git clone url4.gov:/git/ExtraRepo1
$ git clone url4.gov:/git/ExtraRepo2

$ cd $SYNC_BASE_DIR/BaseProj
$ gitdist remote -v | grep -v push | grep -v "^$"
*** Base Git Repo: BaseProj
origin                url4.gov:/git/BaseProj (fetch)
*** Git Repo: ExtraRepo1
origin                url4.gov:/git/ExtraRepo1 (fetch)
public                url2.gov:/git/ExtraRepo1 (fetch)
*** Git Repo: ExtraRepo2
origin                url4.gov:/git/ExtraRepo2 (fetch)

$ cd $SYNC_BASE_DIR
$ mkdir BUILDS
$ mkdir BUILDS/CHECKIN

#!/bin/bash -e

# Set up the environment (i.e. PATH; needed for cron jobs)
...

SYNC_BASE_DIR=~/sync_base_dir
CHECKIN_TEST_WRAPPER=$SYNC_BASE_DIR/BaseProj/sampleScripts/checkin-test-foo.sh

cd $SYNC_BASE_DIR/BUILDS/CHECKIN

$CHECKIN_TEST_WRAPPER \
  --extra-pull-from=ExtraRepo1:public:master \
  --abort-gracefully-if-no-changes-to-push \
  --enable-extra-packages=Package1A \
  --send-build-case-email=only-on-failure \
  --send-email-to=base-proj-integrators@url4.gov \
  --send-email-to-on-push=base-proj-integrators@url4.gov \
  --no-append-test-results --no-rebase \
  --do-all --push \
  -j16 \
  --wipe-clean \
  "$@"

BaseProj/sampleScripts/checkin-test-foo.sh

# ----------------- minute (0 - 59)
# |  -------------- hour (0 - 23)
# |  |  ----------- day of month (1 - 31)
# |  |  |  -------- month (1 - 12)
# |  |  |  |  ----- day of week (0 - 7) (Sunday=0 or 7)
# |  |  |  |  |
# *  *  *  *  *  command to be executed
 00 20  *  *  *  ~/sync_base_dir/sync_ExtraRepo1.sh &> ~/sync_base_dir/sync_ExtraRepo1.out

~/sync_base_dir/sync_ExtraRepo1.sh

$ cd ~/sync_base_dir/
$ ln -s BaseProj/sampleScripts/sync_ExtraRepo1.sh .

11.14   Post-Push CI and Nightly Testing using checkin-test.py

While the post-push CI and Nightly testing processes using ctest -S scripts using tribits_ctest_driver() (see TriBITS CTest/CDash Driver) which posts results to a CDash server (see TriBITS CDash Customizations) is a very attractive testing system with many advantages, setting up a CDash server can be a bit difficult and a CDash server can require non-trivial storage and CPU resources (due to the MySQL DB of test results) and requires some amount of extra maintenance. As an intermediate approach, one can consider just using the project's checkin-test.py tool to implement basic post-push CI and/or Nightly testing servers using simple cron jobs and some other helper scripts. The checkin-test.py tool will robustly pull new commits, configure the project, build, run tests, and send out emails with results and pass/fail. A bunch of builds can be run at once using multiple builds specified in the --default-builds, --st-extra-builds, and --extra-build arguments, or different invocations of the checkin-test.py tool can be run in parallel for better machine utilization.

What one gives up with this approach over the full-blow CTest/CDash implementation is:

1. Test results over multiple builds/days are not archived in a database for later retrieval and query. Only the latest build's detailed results are accessible.
2. Detailed test results (pass or fail) cannot be easily looked up on a website. Only detailed results from the last build can be found and one has to go to the machine where the build was run from to see the results.

The process for setting up a basic nightly build using checkin-test.py as a cron job is a subset of the steps needed to set up a driver and cron job for the more complex ACI process described in ACI Sync Driver Script and ACI Cron Job Setup. Setting up a CI server using checkin-test.py is a little more involved than setting up a nightly build (and less desirable because it blows away old CI iteration results pretty fast) but can be done using existing tools.

For smaller, private, less-critical projects with few developers, setting up CI and Nightly testing using checkin-test.py may be quite adequate. In fact, post-push testing processes implemented with checkin-test.py are much more solid and feature-full than have been employed in many software projects that we have seen over the years that were larger, more public, had many developers, and were quite important to many users and development teams.

11.15   TriBITS Dashboard Driver

TriBITS also includes a system based on CMake/CTest/CDash to drive the builds of a TriBITS project and post results to another CDash project. This system is contained under the directory:

tribits/ctest/tdd/

If the TriBITS project name is <projectName>, the TDD driver CDash project is typically called <projectName>Driver. Using CTest to drive the Nightly builds of a TriBITS project makes sense because CTest can run different builds in parallel, can time-out builds that are taking too long, and will report results to a dashboard and submit notification emails when things fail. However, this is the most confusing and immature part of the TriBITS system. The TriBITS CTest/CDash Driver system using the tribits_ctest_driver() can be used without this TriBITS Dashboard Driver (TDD) system.

However, this TriBITS subsystem is not well tested with automated tests, is difficult to extend and test manually, and has other problems. Therefore, it is not recommended that projects adopt the usage of this subsystem. A simple set of cron jobs or a tool like Jenkins is likely a better option (if done carefully).

11.16   Regulated Backward Compatibility and Deprecated Code

The motivation and ideas behind Regulated Backward Compatibility and deprecated code are described in the TriBITS Lifecycle Model document. Here, the details of the implementation in TriBITS are given and how transitions between non-backward compatible major versions is accomplished.

This section describes the process for deprecating code within a major backward-compatible version sequence and finally removing deprecated code when transitioning to a new major version X to X+1 for the semantic versioning numbering scheme X.Y.Z. This information is given as a process with different phases.

The processing for managing deprecated code is as follows and more details are given below:

• Setting up support for deprecated code handling
• Deprecating code but maintaining backward compatibility
  • Deprecating C/C++ classes, structs, functions, typdefs, etc.
  • Deprecating preprocessor macros
  • Deprecating an entire header and/or source file
• Hiding deprecated code to certify and facilitate later removal
  • Hiding C/C++ entities
  • Hiding entire deprecated header and source files
• Physically removing deprecated code
  • Removing entire deprecated header and source files
  • Removing deprecated code from remaining files

tribits_add_show_deprecated_warnings_option()

<packageDir>/cmake/<PACKAGE_UCNAME>_config.h.in

@<PACKAGE_UCNAME>_DEPRECATED_DECLARATIONS@

-D<Project>_SHOW_DEPRECATED_WARNINGS=ON

-D<PackageName>_SHOW_DEPRECATED_WARNINGS=[ON|OFF]

-D<Project>_HIDE_DEPRECATED_CODE=ON

-D<PackageName>_HIDE_DEPRECATED_CODE=[ON|OFF]

// Deprecate a class (or struct)
class SOMEPACKAGE_DEPRECATED SomeClass { ... };

// Deprecate a function
SOMEPACKAGE_DEPRECATED int somefunc(...);

// Deprecate a typedef
SOMEPACKAGE_DEPRECATED typedef someTypeDef int;

// Deprecated old unsafe function taking raw pointers
somepackage_deprecated_msg(
  "Please use the safe somefunc(const Ptr<const std::string>&) instead!")
void somefunc(const std::string *str);

// New version take does not take raw pointers (and is therefore safer)
void somefunc(const Teuchos::Ptr<const std::string> &str);

"Please use the safe somefunc(const Ptr<const std::string>&) instead!"

// The definition of the macro
#define some_old_func_macro(ARG1, ARG2, ...) ...
...
// A use of the macro
some_old_func_macro(a, b, ...)

SOMEPACKAGE_DEPRECATED
inline void some_old_func_macro_is_deprecated()
{}

// The definition of the macro
#define some_old_func_macro(ARG1, ARG2, ...) \
  { \
    some_old_func_macro_is_deprecated(); \
    ... \
  }

...

// A use of the macro
some_old_func_macro(a, b, ...)

#ifdef __GNUC__
#  warning "This header <THIS_HEADER> is deprecated!  Use <NEW_HEADER> instead!"
#endif

#ifndef SOMEPACKAGE_HIDE_DEPRECATED_CODE
  // Deprecate a class (or struct)
  class SOMEPACKAGE_DEPRECATED SomeClass { ... };
  ...
#endif /* SOMEPACKAGE_HIDE_DEPRECATED_CODE */

set(HEADERS "")
set(SOURCES "")

set_and_inc_dirs(DIR ${CMAKE_CURRENT_SOURCE_DIR})
append_glob(HEADERS ${DIR}/*.hpp)
append_glob(SOURCES ${DIR}/*.cpp)

if (NOT ${PACKAGE_NAME}_HIDE_DEPRECATED_CODE)
  include_directories(${CMAKE_CURRENT_SOURCE_DIR}/deprecated)
  append_set(HEADERS
    SomeOldStuff.hpp
    )
  append_set(SOURCES
    SomeOldStuff.cpp
    )
endif()

...

tribits_add_library(
  <LIBRARY_NAME>
  HEADERS ${HEADERS}
  SOURCES ${SOURCES}
  )

if (NOT ${PACKAGE_NAME}_HIDE_DEPRECATED_CODE)
  include_directories(${CMAKE_CURRENT_SOURCE_DIR}/deprecated)
  append_set(HEADERS
    )
  append_set(SOURCES
    )
endif()

$ find . -name CMakeLists.txt -exec grep -nH HIDE_DEPRECATED_CODE {} \;

$ emacs -nw CMakeLists.txt  # Remove the references to the deprecated files
$ git rm SomeOldStuff.hpp SomeOldStuff.cpp
$ git commit -m "Removing deprecated files"

~/install/bin/unifdef

$ find . -name "*pp" -exec ~/install/bin/unifdef \
  -DSomePackage_HIDE_DEPRECATED_CODE {} -o {} \;

$ git diff -- .

$ git commit -m "Removing deprecated code blocks" -- .

11.17   Installation and Backward Compatibility Testing

TriBITS has some built-in support for installation testing and backward compatibility testing. The way it works is that one can install the headers, libraries, and executables for a TriBITS project and then configure the tests and examples in the TriBITS project against the installed headers/libraries/executables. In this mode, the TriBITS project's libraries and executables are not build and the header file locations to local source are not included.

When the same version of the project sources are used to build the tests/examples against the installed headers/libraries/executables, then this constitutes installation testing. When an older version of the project is used to build and run tests and examples against the headers/libraries/executables for a version of the project, then this constitutes backward compatibility testing which also includes installation testing of course.

11.18   Wrapping Externally Configured/Built Software

It is possible to take an external piece of software that uses any arbitrary build system and wrap it as a TriBITS package and have it integrate in with the package dependency infrastructure. The TribitsExampleProject package WrapExternal shows how this can be done. Support for this in TriBITS is slowly evolving but some key TriBITS features that have been added to support the arbitrary case include:

• tribits_determine_if_current_package_needs_rebuilt(): Uses brute-force searches for recently modified files in upstream packages to determine if the external piece of software needs to be rebuilt.
• tribits_write_flexible_package_client_export_files(): Write an export makefile or a XXXConfig.cmake file for usage by the wrapped externally configured and built software.

While it is possible to wrap nearly any externally configured and built piece of software as a TriBITS package, in most cases, it is usually better to just create a TriBITS build system for the software. For projects that use a raw CMake build system, a TriBITS build system can be created side-by-side with the existing raw CMake build using a number of approaches. The common approach that is not too invasive is to create a CMakeLists.tribits.txt file along side every native CMakeLists.txt file in the external software project and have the native CMakeLists.txt file defined like:

if (DOING_A_TRIBITS_BUILD)
  include("${CMAKE_CURRENT_SOURCE_DIR}/CMakeLists.tribits.txt")
  return()
endif()

# Rest of native CMakeLists.txt file ...

Experience from the CASL VERA project has shown that, overall, there is less hassle, less work, and better portability when creating a native TriBITS build, even if it is a secondary build system for a given piece of software.

The decision whether to just wrap the build system for the existing software or to create a (secondary) TriBITS build system for it depends on a number of factors.

Note that while it is generally recommended to create a TriBITS build for an existing piece of software, it is generally not recommended with switch over all of the tests to use CTest (unless the existing software is going to ditch their current test driver and reporting system). Instead, the external software's native test system can just be called by the wrapped TriBITS package in one or more CTest tests.

11.19   TriBITS directory snapshotting

Some TriBITS projects choose to snapshot the TriBITS/tribits/ directory source tree into their project's source tree, typically under <projectDir>/cmake/tribits/. The independent TriBITS/tribts/ source tree contains the tool snapshot_tribits.py (calls snapshot-dir.py) that allows one to update the snapshot of the TriBITS source tree as simply as:

$ cd <projectDir>/cmake/tribits/
$ <some-base-dir>/TriBITS/tribits/snapshot_tribits.py

This will create a git commit in the local <projectDir>/ git repo that looks like:

Automatic snapshot commit from tribits at f8c1682

Origin repo remote tracking branch: 'casl-dev-collab/tribits_reorg_26'
Origin repo remote repo URL: 'casl-dev-collab = git@casl-dev:collaboration/TriBITS'

At commit:

f8c1682 Assert TriBITS min CMake version in TriBITS itself
Author: Roscoe A. Bartlett <bartlettra@ornl.gov>
Date: Fri Dec 5 05:40:49 2014 -0500

This, of course, assumes that <projectDir>/ is a local git repo (or is in local git repo). If that is not the case, then one cannot use the script snapshot_tribits.py or must use it with the --skip-commit option.

See snapshot-dir.py --help for more details. Note the guidance on using a different branch for the snapshot sync followed by a merge. This allows for one to maintain local changes to TriBITS and use git to manage the merges. However, this will increase the changes of merge conflicts so one should consider just directly snapshotting into the master branch to avoid merge conflicts.

11.20   TriBITS Development Toolset

Most TriBITS projects need git, a compiler (e.g. GCC), MPI, and a number of other standard TPLs and other tools in order to develop on and test the project code. To this end, TriBITS contains some helper scripts for downloading, configuring, building, and installing packages like git, cmake, GCC, MPICH, and others needed to set up a development environment for a typical computational science software project. These tools are used to set up development environments on new machines for projects like Trilinos and CASL VERA. Scripts with names like install-gcc.py are defined which pull sources from public git repos then configure, build, and install into specified installation directories.

The script install_devtools.py is provided in the directory:

tribits/devtools_install/

To use this script, one just needs to create some scratch directory like:

$ mkdir scratch
$ cd scratch/

then install the tools using, for example:

$ install_devtools.py --install-dir=~/install/tribits_devtools \
  --parallel=16 --do-all

Then to access installed development environment, one just needs to source the script:

~/install/tribits_devtools/load_dev_env.sh

and then the installed versions of GCC, MPICH, CMake, and gitdist are placed in one's path.

See install_devtools.py --help for more details.

12.1   TriBITS Global Project Settings

TriBITS defines a number of global project-level settings that can be set by the user and can have their default determined by each individual TriBITS project. If a given TriBITS project does not define its own default, a reasonable default is set by the TriBITS system automatically. These options are defined and are set, for the most part, in the internal TriBITS function tribits_define_global_options_and_define_extra_repos() in the TriBITS CMake code file TribitsGlobalMacros.cmake which gets called inside of the tribits_project() macro. That function and that file are the definitive source the options that a TriBITS project takes and what the default values are but we strive to document them here as well. Many of these global options (i.e. cache variables) such as ${PROJECT_NAME}_<SOME_OPTION> allow the project to define a default by setting a local variable ${PROJECT_NAME}_<SOME_OPTION>_DEFAULT as:

set(${PROJECT_NAME}_<SOME_OPTION>_DEFAULT <someDefault>)

either in its top-level CMakeLists.txt file or in its ProjectName.cmake file (depends on what variable it is as to where it should be set). If ${PROJECT_NAME}_<SOME_OPTION>_DEFAULT is not set by the project, then TriBITS provides a reasonable default value. The TriBITS code that uses these defaults for this looks like:

if ("${${PROJECT_NAME}_<SOME_OPTION>_DEFAULT}" STREQUAL "")
  set(${PROJECT_NAME}_<SOME_OPTION>_DEFAULT <someDefault>)
endif()

advanced_set( ${PROJECT_NAME}_<SOME_OPTION>
  ${PROJECT_NAME}_<SOME_OPTION>_DEFAULT}
  CACHE BOOL "[documentation]."
  )

where <SOME_OPTION> is an option name, for example like TEST_CATEGORIES, and <someDefault> is the default set by TriBITS if the project does not define a default. In this way, if the project sets the variable ${PROJECT_NAME}_<SOME_OPTION>_DEFAULT before this code executes, then ${${PROJECT_NAME}_<SOME_OPTION>_DEFAULT} will be used as the default for the cache variable ${PROJECT_NAME}_<SOME_OPTION> which, of course, can be overridden by the user when calling cmake in a number of ways.

Most of these global options that can be overridden externally by setting the cache variable ${PROJECT_NAME}_<SOME_OPTION> should be documented in the Project-Specific Build Reference document. A generic version of this document is found in TribitsBuildReference. Some of the more unusual options that might only be of interest to developers mentioned below may not be documented in TribitsBuildReference.

The global project-level TriBITS options for which defaults can be provided by a given TriBITS project are:

• ${PROJECT_NAME}_ASSERT_CORRECT_TRIBITS_USAGE
• ${PROJECT_NAME}_ASSERT_DEFINED_DEPENDENCIES
• ${PROJECT_NAME}_C_Standard
• ${PROJECT_NAME}_CHECK_FOR_UNPARSED_ARGUMENTS
• ${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE_APPEND
• ${PROJECT_NAME}_CPACK_SOURCE_GENERATOR
• ${PROJECT_NAME}_CTEST_DO_ALL_AT_ONCE
• ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES
• ${PROJECT_NAME}_ELEVATE_ST_TO_PT
• ${PROJECT_NAME}_ENABLE_CPACK_PACKAGING
• ${PROJECT_NAME}_ENABLE_CXX
• ${PROJECT_NAME}_ENABLE_C
• ${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE
• ${PROJECT_NAME}_ENABLE_Fortran
• ${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES
• ${PROJECT_NAME}_ENABLE_SECONDARY_TESTED_CODE
• ${PROJECT_NAME}_EXCLUDE_DISABLED_SUBPACKAGES_FROM_DISTRIBUTION
• ${PROJECT_NAME}_GENERATE_EXPORT_FILE_DEPENDENCIES
• ${PROJECT_NAME}_GENERATE_VERSION_DATE_FILES
• ${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE
• ${PROJECT_NAME}_IMPORTED_NO_SYSTEM
• ${PROJECT_NAME}_INSTALL_LIBRARIES_AND_HEADERS
• ${PROJECT_NAME}_MAKE_INSTALL_GROUP_READABLE
• ${PROJECT_NAME}_MAKE_INSTALL_GROUP_WRITABLE
• ${PROJECT_NAME}_MAKE_INSTALL_WORLD_READABLE
• ${PROJECT_NAME}_MUST_FIND_ALL_TPL_LIBS
• ${PROJECT_NAME}_Python3_FIND_VERSION
• ${PROJECT_NAME}_REQUIRES_PYTHON
• ${PROJECT_NAME}_SET_INSTALL_RPATH
• ${PROJECT_NAME}_SHOW_GIT_COMMIT_PARENTS
• ${PROJECT_NAME}_SHOW_TEST_START_END_DATE_TIME
• ${PROJECT_NAME}_SKIP_INSTALL_PROJECT_CMAKE_CONFIG_FILES
• ${PROJECT_NAME}_TEST_CATEGORIES
• ${PROJECT_NAME}_TPL_SYSTEM_INCLUDE_DIRS
• ${PROJECT_NAME}_TRACE_ADD_TEST
• ${PROJECT_NAME}_USE_GNUINSTALLDIRS
• ${PROJECT_NAME}_USES_PYTHON
• DART_TESTING_TIMEOUT
• CMAKE_INSTALL_RPATH_USE_LINK_PATH
• MPI_EXEC_MAX_NUMPROCS
• TRIBITS_HANDLE_TRIBITS_DEPRECATED_CODE

These options are described below.

${PROJECT_NAME}_ASSERT_CORRECT_TRIBITS_USAGE

The CMake cache variable ${PROJECT_NAME}_ASSERT_CORRECT_TRIBITS_USAGE is used to define how some invalid TriBITS usage checks are handled. The valid values include 'FATAL_ERROR', 'SEND_ERROR', 'WARNING', and 'IGNORE'. The default value is 'FATAL_ERROR' for a project when ${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE=ON, which is best for development mode for a project that currently has no invalid usage patterns. The default is 'IGNORE' when ${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE=OFF. But a project with some existing invalid usage patterns might want to set, for example, a default of 'WARNING' in order to allow for a smooth upgrade of TriBITS. To do so, set:

set(${PROJECT_NAME}_ASSERT_CORRECT_TRIBITS_USAGE_DEFAULT WARNING)

in the project's base <projectDir>/ProjectName.cmake file.

${PROJECT_NAME}_ASSERT_DEFINED_DEPENDENCIES

To set ${PROJECT_NAME}_ASSERT_DEFINED_DEPENDENCIES a different default, set:

set(${PROJECT_NAME}_ASSERT_DEFINED_DEPENDENCIES_DEFAULT  <newDefault>)

in the project's base <projectDir>/ProjectName.cmake file, where <newDefault> can be FATAL_ERROR, SEND_ERROR, WARNING, NOTICE or IGNORE

Otherwise, the default is WARNING when ${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE is ON and if IGNORE if ${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE is OFF.

${PROJECT_NAME}_C_Standard

The variable ${PROJECT_NAME}_C_Standard is used define the C standard pass to the compiler in --std=<cstd> for GCC builds of the project. TriBITS sets the default as c99 but the project can set a new default in the project's base <projectDir>/CMakeLists.txt file with, for example:

set(${PROJECT_NAME}_C_Standard_DEFAULT c11)

${PROJECT_NAME}_CHECK_FOR_UNPARSED_ARGUMENTS

The variable ${PROJECT_NAME}_CHECK_FOR_UNPARSED_ARGUMENTS determines how unparsed and otherwise ignored arguments are handled in TriBITS functions that are called by the client TriBITS projects. These are arguments that are left over from parsing input options to functions and macros that take both positional arguments and keyword arguments/options handled with the cmake_parse_arguments() function. For example, for the a TriBITS function declared like:

tribits_copy_files_to_binary_dir(
  <targetName>
  [SOURCE_FILES <file1> <file2> ...]
  [SOURCE_DIR <sourceDir>]
  ...
  )

the arguments SOURCE_FILES <file1> <file2> ... and those that follow are parsed by the cmake_parse_arguments() function while the argument <targetName> is a positional argument. The problem is that any arguments passed between the first <targetName> argument and the specified keyword arguments like SOURCE_FILES and SOURCE_DIR are returned as unparsed arguments and are basically ignored (which is what happened in earlier versions of TriBITS). For example, calling the function as:

tribits_copy_files_to_binary_dir( FooTestCopyFiles
  ThisArgumentIsNotParsedAndIsIgnored
  SOURCE_FILES file1.cpp file2.cpp ...
  ...
  )

would result in the unparsed argument ThisArgumentIsNotParsedAndIsIgnored.

The value of ${PROJECT_NAME}_CHECK_FOR_UNPARSED_ARGUMENTS determines how that ignored argument is handled. If the value is WARNING, then it will just result in a message(WARNING ...) command that states the warning but configure is allowed to be completed. This would be the right value to allow an old TriBITS project to keep configuring until the warnings can be cleaned up. If the value is SEND_ERROR, then message(SEND_ERROR ...) is called. This will result in the configure failing but will allow configure to continue until the end (or a FATAL_ERROR is raised). This would be the right value when trying to upgrade a TriBITS project where you wanted to see all of the warnings when upgrading TriBITS (so you could fix them all in one shot). Finally, the value of FATAL_ERROR will result in message(FATAL_ERROR ...) being called which will halt configure right away. This is the best value when developing on a TriBITS project that is already clean but you want to catch new developer-inserted errors right away.

The default value for ${PROJECT_NAME}_CHECK_FOR_UNPARSED_ARGUMENTS is WARNING, so that it will be backward compatible for TriBITS projects that might have previously undetected unparased and therefore ignored argument . However, a project can change the default by setting, for example:

set(${PROJECT_NAME}_CHECK_FOR_UNPARSED_ARGUMENTS_DEFAULT FATAL_ERROR)

in the <projectDir>/ProjectName.cmake file.

The user of a TriBITS project should not be able to trigger this unparsed arguments condition so this variable is not documented in the TriBITS Build Reference. But it is still a CMake cache var that is documented in the CMakeCache.txt file and can be set by the user or developer if desired.

${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE_APPEND

The variable ${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE_APPEND is used to define the absolute path to a file (or a list of files) that should be included after the files listed in ${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE. This variable can be used by the TriBITS project to define, for example, a standard set of development environments in the base <projectDir>/CMakeLists.txt file with:

set(${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE_APPEND_DEFAULT
  "${CMAKE_CURRENT_LIST_DIR}/cmake/StdDevEnvs.cmake")

before the tribits_project() command. By including this file(s) after the file(s) listed in ${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE, the user can override the variables set in this appended file(s). But it is important that these variables best set after the users options have been set but before the Package and TPL dependency analysis is done (because this might enable or disable some TPLs).

${PROJECT_NAME}_CPACK_SOURCE_GENERATOR

The variable ${PROJECT_NAME}_CPACK_SOURCE_GENERATOR determines the CPack source generation types that are created when the package_source target is run. The TriBITS default is set to TGZ. However, this default can be overridden by setting, for example:

set(${PROJECT_NAME}_CPACK_SOURCE_GENERATOR_DEFAULT "TGZ;TBZ2")

This variable should generally be set in the file:

<projectDir>/cmake/CallbackDefineProjectPackaging.cmake

instead of in the base-level CMakeLists.txt file so that it goes along with rest of the project-specific CPack packaging options.

${PROJECT_NAME}_CTEST_DO_ALL_AT_ONCE

The variable ${PROJECT_NAME}_CTEST_DO_ALL_AT_ONCE determines if the CTest driver scripts using tribits_ctest_driver() configure, build, test and submit results to CDash all-at-once for all of the packages being tested or if instead is done package-by-package. Currently, the default is set to FALSE for the package-by-package mode (for historical reasons) but the default can be set to TRUE by setting:

set(${PROJECT_NAME}_CTEST_DO_ALL_AT_ONCE_DEFAULT "TRUE")

in the project's <projectDir>/ProjectName.cmake file. (This default must be changed in the <projectDir>/ProjectName.cmake file and NOT the <projectDir>/CMakeLists.txt file because the latter is not directly processed in CTest -S driver scripts using tribits_ctest_driver().)

In general, a project should change the default to TRUE when using a newer CDash installation with CDash versions 3.0+ that can accommodate the results coming from ctest -S and display them package-by-package very nicely. Otherwise, most projects are better off with package-by-package mode since it results in nicer display on CDash for older CDash versions.

${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES

If ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON (the TriBITS default value), then any explicitly enabled packages that have disabled upstream required packages or TPLs will be disabled. If ${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=OFF, then an configure error will occur. For more details also see TribitsBuildReference and Disables trump enables where there is a conflict. A project can define a different default value by setting:

set(${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES_DEFAULT FALSE)

${PROJECT_NAME}_ELEVATE_ST_TO_PT

If ${PROJECT_NAME}_ELEVATE_ST_TO_PT is set to ON, then all ST packages will be elevated to PT packages. The TriBITS default is obviously OFF. The default can be changed by setting:

set(${PROJECT_NAME}_ELEVATE_ST_TO_PT_DEFAULT ON)

There are projects, especially meta-projects, where the distinction between PT and ST code is not helpful or the assignment of PT and ST packages in a repository is not appropriate with respect to the outer meta-project. An example project like this CASL VERA. Changing the default to ON allows any and packages to be considered in pre-push testing.

${PROJECT_NAME}_ENABLE_CPACK_PACKAGING

If ${PROJECT_NAME}_ENABLE_CPACK_PACKAGING is ON, then CPack support is enabled and some TriBITS code is run that is needed to set up data-structures that are used by the built-in CMake target package_source. The TriBITS default is OFF with the idea that the average developer or user will not be wanting to create source distributions with CPack. However, this default can be changed by setting:

set(${PROJECT_NAME}_ENABLE_CPACK_PACKAGING_DEFAULT ON)

${PROJECT_NAME}_ENABLE_CXX

If ${PROJECT_NAME}_ENABLE_CXX is ON, then C++ language support for the project will be enabled and the C++ compiler must be found. By default, TriBITS sets this to ON for all systems. A project never requires C++ can set this to off by default by setting:

set(${PROJECT_NAME}_ENABLE_CXX_DEFAULT FALSE)

${PROJECT_NAME}_ENABLE_C

If ${PROJECT_NAME}_ENABLE_C is ON, then C language support for the project will be enabled and the C compiler must be found. By default, TriBITS sets this to ON for all systems. A project never requires C can set this to off by default by setting:

set(${PROJECT_NAME}_ENABLE_C_DEFAULT FALSE)

If a project does not have any native C code a good default would be:

set(${PROJECT_NAME}_ENABLE_C_DEFAULT FALSE)

NOTE: It is usually not a good idea to always force off C, or any compiler, because extra repositories and packages might be added by someone that might require the compiler and we don't want to unnecessarily limit the generality of a given TriBITS build. Setting the default for all platforms should be sufficient.

${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE

The variable ${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE switches the TriBITS project from development mode to release mode. The default for this variable ${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE_DEFAULT should be set in the project's <projectDir>/Version.cmake file and switched from ON to OFF when creating a release (see Project and Repository Versioning and Release Mode). When ${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE is ON, several other variables are given defaults appropriate for development mode. For example, ${PROJECT_NAME}_ASSERT_DEFINED_DEPENDENCIES is set to FATAL_ERROR by default in development mode but is set to IGNORE by default in release mode. In addition, strong compiler warnings are enabled by default in development mode but are disabled by default in release mode. This variable also affects the behavior of tribits_set_st_for_dev_mode().

${PROJECT_NAME}_ENABLE_Fortran

If ${PROJECT_NAME}_ENABLE_Fortran is ON, then Fortran support for the project will be enabled and the Fortran compiler(s) must be found. By default, TriBITS sets this to ON .

If a project does not have any native Fortran code a good default would be:

set(${PROJECT_NAME}_ENABLE_Fortran_DEFAULT OFF)

This default can be set in <projectDir>/ProjectName.cmake or <projectDir>/CMakeLists.txt.

On WIN32 systems, the default for ${PROJECT_NAME}_ENABLE_Fortran_DEFAULT is set to OFF since it can be difficult to get a Fortran compiler for native Windows.

Given that a native Fortran compiler is not supported by default on Windows and on most Mac OSX systems, projects that have optional Fortran code may decide to set the default depending on the platform by setting, for example:

if ( (WIN32 AND NOT CYGWIN) OR (CMAKE_HOST_SYSTEM_NAME STREQUAL "Darwin") )
  message(STATUS "Warning: Setting ${PROJECT_NAME}_ENABLE_Fortran=OFF by default"
   " because this is Windows (not cygwin) and we assume to not have Fortran!")
  set(${PROJECT_NAME}_ENABLE_Fortran_DEFAULT OFF)
else()
  set(${PROJECT_NAME}_ENABLE_Fortran_DEFAULT ON)
endif()

NOTE: It is usually not a good idea to always force off Fortran, or any compiler, because extra repositories and packages might be added by someone that might require the compiler and we don't want to unnecessarily limit the generality of a given TriBITS build. Setting the default for all platforms should be sufficient.

${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES

If ${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES is set to ON, then <PackageName>Config.cmake files are created at configure time in the build tree and installed into the install tree. These files are used by external CMake projects to pull in the list of compilers, compiler options, include directories and libraries. The TriBITS default is OFF but a project can change the default by setting, for example:

set(${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES_DEFAULT ON)

A project would want to leave off the creation and installation of <PackageName>Config.cmake files if it was only installing and providing executables (see ${PROJECT_NAME}_INSTALL_LIBRARIES_AND_HEADERS). However, if it is wanting to provide libraries for other projects to use, then it should turn on the default generation of these files.

${PROJECT_NAME}_ENABLE_SECONDARY_TESTED_CODE

If ${PROJECT_NAME}_ENABLE_SECONDARY_TESTED_CODE is ON, then packages and subpackages marked as ST in the <repoDir>/PackagesList.cmake file will be implicitly enabled along with the PT packages. Additional code and tests may also be enabled using this option. The TriBITS default is OFF but this can be changed by setting:

set(${PROJECT_NAME}_ENABLE_SECONDARY_TESTED_CODE_DEFAULT ON)

in the <projectDir>/ProjectName.cmake file.

${PROJECT_NAME}_EXCLUDE_DISABLED_SUBPACKAGES_FROM_DISTRIBUTION

If ${PROJECT_NAME}_EXCLUDE_DISABLED_SUBPACKAGES_FROM_DISTRIBUTION is TRUE, then the directories for subpackages that are not enabled are left out of the source tarball. This reduces the size of the tarball as much as possible but does require that the TriBITS packages and subpackages be properly set up to allow disabled subpackages from being excluded. The TriBITS default is TRUE but this can be changed by setting:

set(${PROJECT_NAME}_EXCLUDE_DISABLED_SUBPACKAGES_FROM_DISTRIBUTION_DEFAULT FALSE)

${PROJECT_NAME}_GENERATE_EXPORT_FILE_DEPENDENCIES

If ${PROJECT_NAME}_GENERATE_EXPORT_FILE_DEPENDENCIES is ON, then the data-structures needed to generate <PackageName>Config.cmake files are created. These data structures are also needed in order to generate export makefiles on demand using the function tribits_write_flexible_package_client_export_files(). The default in TriBITS is to turn this ON automatically by default if ${PROJECT_NAME}_ENABLE_INSTALL_CMAKE_CONFIG_FILES is ON. Else, by default, TriBITS sets this to OFF. The only reason for the project to override the default is to set it to ON as with:

set(${PROJECT_NAME}_GENERATE_EXPORT_FILE_DEPENDENCIES_DEFAULT ON)

is so that the necessary data-structures are generated in order to use the function tribits_write_flexible_package_client_export_files().

${PROJECT_NAME}_GENERATE_VERSION_DATE_FILES

If ${PROJECT_NAME}_GENERATE_VERSION_DATE_FILES is ON, then the files VersionDate.cmake and <RepoName>_version_date.h will get generated and the generated file <RepoName>_version_date.h will get installed for each TriBITS version-controlled repository when the local directories are git repositories. The default is OFF but the project can change that by setting:

set(${PROJECT_NAME}_GENERATE_VERSION_DATE_FILES ON)

in the <projectDir>/ProjectName.cmake file.

${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE

If ${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE is ON, then the file <Project>RepoVersion.txt will get generated as a byproduct of configuring with CMake. See Multi-Repository Support and <Project>_GENERATE_REPO_VERSION_FILE. The default is OFF but the project can change that by setting:

set(${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE_DEFAULT ON)

in the <projectDir>/ProjectName.cmake file.

Note that if a git exectauble cannot be found at configure time, then the default ${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE_DEFAULT will be overridden to OFF. But if the user sets ${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE=ON in the cache and git can't be found, then an configure-time error will occur.

${PROJECT_NAME}_IMPORTED_NO_SYSTEM

By default, include directories from IMPORTED library targets from the TriBITS project's installed <Package>Config.cmake files will be considered SYSTEM headers and therefore be included on the compile lines of downstream CMake projects with -isystem with most compilers. However, if ${PROJECT_NAME}_IMPORTED_NO_SYSTEM is set to ON (only supported for CMake versions 3.23 or greater), then all of the IMPORTED library targets exported into the set of installed <Package>Config.cmake files will have the IMPORTED_NO_SYSTEM property set. This will cause downstream customer CMake projects to apply the include directories from these IMPORTED library targets as non-system include directories. On most compilers, that means that the include directories will be listed on the compile lines with -I instead of with -isystem. (See more details in the TriBITS Build Reference for <Project>_IMPORTED_NO_SYSTEM.)

The default value set by TriBITS itself is OFF but a TriBITS project can change the default value to ON by adding:

if (CMAKE_VERSION VERSION_GREATER_EQUAL 3.23)
  set(${PROJECT_NAME}_IMPORTED_NO_SYSTEM_DEFAULT ON)
endif()

in the <projectDir>/ProjectName.cmake file. (NOTE: The above if() statement ensures that a configure error will not occur if a version of CMake less than 3.23 is used. But if the TriBITS project minimum CMake version is 3.23 or greater, then the above if() statement guard can be removed.)

${PROJECT_NAME}_INSTALL_LIBRARIES_AND_HEADERS

If ${PROJECT_NAME}_INSTALL_LIBRARIES_AND_HEADERS is set to ON, then any defined libraries or header files that are listed in calls to tribits_add_library() or tribits_install_headers() will be installed (unless options are passed into tribits_add_library() that disable installs). If set to OFF, then headers and libraries will not be installed by default and only INSTALLABLE executables added with tribits_add_executable() will be installed. However, as described in TribitsBuildReference, shared libraries will always be installed if enabled since they are needed by the installed executables.

For a TriBITS project that is primarily delivering libraries (e.g. Trilinos), then it makes sense to leave the TriBITS default which is ON or explicitly set:

set(${PROJECT_NAME}_INSTALL_LIBRARIES_AND_HEADERS_DEFAULT  ON)

For a TriBITS project that is primarily delivering executables (e.g. VERA), then it makes sense to set the default to:

set(${PROJECT_NAME}_INSTALL_LIBRARIES_AND_HEADERS_DEFAULT  OFF)

${PROJECT_NAME}_MAKE_INSTALL_GROUP_READABLE ${PROJECT_NAME}_MAKE_INSTALL_GROUP_WRITABLE ${PROJECT_NAME}_MAKE_INSTALL_WORLD_READABLE

Determines the permissions for directories and files created during the execution of the of the install and isntall_package_by_package targets.

To make the created directories by only group readable for the project by default, set:

set(${PROJECT_NAME}_MAKE_INSTALL_WORLD_READABLE_DEFAULT TRUE)

To make the created directories by only group writable (and readable) for the project by default, set:

set(${PROJECT_NAME}_MAKE_INSTALL_WORLD_WRITABLE_DEFAULT TRUE)

To make the created directories by world readable for the project by default, set:

set(${PROJECT_NAME}_MAKE_INSTALL_WORLD_READABLE_DEFAULT TRUE)

On non-Windows systems, these set permissions for all files and directories from the the user-set base directory ${PROJECT_NAME}_SET_GROUP_AND_PERMISSIONS_ON_INSTALL_BASE_DIR on down. For more details see Installation considerations.

These defaults can be set in the <projectDir>/ProjectName.cmake file.

${PROJECT_NAME}_MUST_FIND_ALL_TPL_LIBS

Determines if all of the libraries listed in <tplName>_LIBRARY_NAMES for a given TPL must be found for each enabled TPL. By default, this is FALSE which means that the determination if all of the listed libs for a TPL should be found is determined by the MUST_FIND_ALL_LIBS option to the tribits_tpl_find_include_dirs_and_libraries() function in the TPL find module. To change the default for this, set:

set(${PROJECT_NAME}_MUST_FIND_ALL_TPL_LIBS_DEFAULT  TRUE)

in the <projectDir>/ProjectName.cmake file.

${PROJECT_NAME}_Python3_FIND_VERSION

Determines the version of Python that is looked for. TriBITS requires at least version "3.6". A particular TriBITS project can require a higher version of TriBITS and this is set using, for example:

set(${PROJECT_NAME}_Python3_FIND_VERSION_DEFAULT "3.8")

in the <projectDir>/ProjectName.cmake file (See Python Support). The user can force a more recent version of Python by configuring with, for example:

-D <Project>_Python3_FIND_VERSION="3.8"

${PROJECT_NAME}_REQUIRES_PYTHON

If the TriBITS project requires Python, set:

set(${PROJECT_NAME}_REQUIRES_PYTHON  TRUE)

in the <projectDir>/ProjectName.cmake file (See Python Support). The default is implicitly FALSE.

${PROJECT_NAME}_SET_INSTALL_RPATH

The cache variable ${PROJECT_NAME}_SET_INSTALL_RPATH is used to define the default RPATH mode for the TriBITS project (see Setting install RPATH for details). The TriBITS default is to set this to TRUE but the TriBITS project can be set the default to FALSE by setting:

set(${PROJECT_NAME}_SET_INSTALL_RPATH_DEFAULT FALSE)

in the project's <projectDir>/ProjectName.cmake file (see RPATH Handling).

${PROJECT_NAME}_SHOW_GIT_COMMIT_PARENTS

The cache variable ${PROJECT_NAME}_SHOW_GIT_COMMIT_PARENTS results in the repo version file showing the parent commits for each repo commit. By default, this variable is set to OFF but projects can set to to ON by default by setting:

set(${PROJECT_NAME}_SHOW_GIT_COMMIT_PARENTS_DEFAULT ON)

in the project's ProjectName.cmake file. (That way, it will also impact cmake -P scripts don't configure the project itself to be built.)

${PROJECT_NAME}_SHOW_TEST_START_END_DATE_TIME

The cache variable ${PROJECT_NAME}_SHOW_TEST_START_END_DATE_TIME determines if the start and end date/time for each advanced test (i.e. added with tribits_add_advanced_test()) is printed or not with each test. If set to TRUE this also causes in the timing for each TEST_<IDX> block to be printed as well. The TriBITS default is OFF but a TriBITS project can change this default by setting:

set(${PROJECT_NAME}_SHOW_TEST_START_END_DATE_TIME_DEFAULT ON)

The implementation of this feature currently uses execute_process(date) and therefore will work on many (but perhaps not all) Linux/Unix/Mac systems and not on Windows systems.

NOTE: In a future version of CTest, this option may turn on start and end date/time for regular tests added with tribits_add_test() (which uses a raw command with add_test()).

${PROJECT_NAME}_SKIP_INSTALL_PROJECT_CMAKE_CONFIG_FILES

To change the default value of the ${PROJECT_NAME}_SKIP_INSTALL_PROJECT_CMAKE_CONFIG_FILES to TRUE, for example, for a TriBITS project, set:

set(${PROJECT_NAME}_SKIP_INSTALL_PROJECT_CMAKE_CONFIG_FILES_DEFAULT TRUE)

in the project's <projectDir>/CMakeLists.txt or <projectDir>/ProjectName.cmake files.

${PROJECT_NAME}_SKIP_EXTRAREPOS_FILE

The cache variable ${PROJECT_NAME}_SKIP_EXTRAREPOS_FILE is set in the <projectDir>/ProjectName.cmake file as:

set(${PROJECT_NAME}_SKIP_EXTRAREPOS_FILE TRUE)

for projects that don't have a <projectDir>/cmake/ExtraRepositoriesList.cmake file. This variable needs to be set when using the CTest driver script and does not need to be set for the basic configure and build process.

${PROJECT_NAME}_TEST_CATEGORIES

The cache variable ${PROJECT_NAME}_TEST_CATEGORIES determines what tests defined using tribits_add_test() and tribits_add_advanced_test() will be added for ctest to run (see Test Test Category). The TriBITS default is NIGHTLY for a standard local build. The checkin-test.py tool sets this to BASIC by default. A TriBITS project can override the default for a basic configure using, for example:

set(${PROJECT_NAME}_TEST_CATEGORIES_DEFAULT BASIC)

The justification for having the default Test Test Category be NIGHTLY instead of BASIC is that when someone is enabling a package to develop on it or install it, we want them by default to be seeing the full version of the test suite (shy of the Test Test Category HEAVY tests which can be very expensive) for the packages they are explicitly enabling. Typically they will not be enabling forward/downstream dependent packages so the cost of running the test suite should not be too prohibitive. This all depends on how good of a job the development teams do in making their test suites run fast and keeping the cost of running the tests down. See the section TriBITS Automated Testing for a more detailed discussion.

${PROJECT_NAME}_TPL_SYSTEM_INCLUDE_DIRS

If ${PROJECT_NAME}_TPL_SYSTEM_INCLUDE_DIRS is set to TRUE, then the SYSTEM flag will be passed into the include_directories() command for TPL include directories for every TPL for every package, by default. On some systems this will result in include directories being passed to the compiler with -isystem instead of -I. This helps to avoid compiler warning coming from TPL header files for C and C++. However, with CMake version 3.2 and less, this also results in -isystem being passed to the Fortran compiler (e.g. gfortran) as well. This breaks the reading of Fortran module files (perhaps a bug in gfortran). Because of this issue with Fortran, the TriBITS default for this option is set to FALSE but a project can override the default using:

set(${PROJECT_NAME}_TPL_SYSTEM_INCLUDE_DIRS_DEFAULT  TRUE)

(This would be a good default if the project has not Fortran files or has not Fortran files that use modules provided by TPLs).

However, if a package or subpackage sets:

set(${PACKAGE_NAME}_SKIP_TPL_SYSTEM_INCLUDE_DIRS  TRUE)

in its CMakeLists.txt files before the tribits_add_library() or tribits_add_executable() commands are called in that package, then SYSTEM will not be passed into include_directories() for TPL include dirs. This is how some TriBITS packages with Fortran files that use Fortran modules avoid passing in -isystem to the Fortran compiles and thereby avoid the defect with gfortran described above. If CMake version 3.3 or greater is used, this variable is not required.

NOTE: Currently, a TriBITS package must have a direct dependency on a TPL to have -isystem added to a TPL's include directories on the compile lines for that package. That is, the TPL must be listed in the LIB_REQUIRED_TPLS or LIB_OPTIONAL_TPLS arguments passed into the tribits_package_define_dependencies() function in the package's <packageDir>/cmake/Dependencies.cmake file. In addition, to have -isystem added to the include directories for a TPL when compiling the tests for an package, it must be listed in the TEST_REQUIRED_TPLS or TEST_OPTIONAL_TPLS arguments. This is a limitation of the TriBITS implementation that will be removed in a future version of TriBITS.

${PROJECT_NAME}_TRACE_ADD_TEST

If ${PROJECT_NAME}_TRACE_ADD_TEST is set to TRUE, then a single line will be printed for each call to tribits_add_test() and tribits_add_advanced_test() for if the test is added or not and if not then why. The default is set based on the value of ${PROJECT_NAME}_VERBOSE_CONFIGURE but a project can override the default by setting:

set(${PROJECT_NAME}_TRACE_ADD_TEST_DEFAULT  TRUE)

${PROJECT_NAME}_USE_GNUINSTALLDIRS

If ${PROJECT_NAME}_USE_GNUINSTALLDIRS is set to TRUE, then the default install paths will be determined by the standard CMake module GNUInstallDirs. Otherwise, platform independent install paths are used by default.

A project can use the paths given the cmake module GNUInstallDirs by default by setting:

set(${PROJECT_NAME}_USE_GNUINSTALLDIRS_DEFAULT TRUE)

in the project's top-level <projectDir>/CMakeLists.txt file or its <projectDir>/ProjectName.cmake file. The default is FALSE.

${PROJECT_NAME}_USES_PYTHON

If the TriBITS project can use Python, but does not require it, set:

set(${PROJECT_NAME}_USES_PYTHON  TRUE)

in the <projectDir>/ProjectName.cmake file (see Python Support). The default for a TriBITS project is implicitly TRUE. To explicitly state that Python is never needed, set:

set(${PROJECT_NAME}_USES_PYTHON  FALSE)

DART_TESTING_TIMEOUT

The cache variable DART_TESTING_TIMEOUT is a built-in CMake variable that provides a default timeout for all tests (see Setting test timeouts at configure time). By default, TriBITS defines this to be 1500 seconds (which is also the raw CMake default) but the project can change this default, from 1500 to 300 for example, by setting the following in the project's <projectDir>/ProjectName.cmake or <projectDir>/CMakeLists.txt file:

set(DART_TESTING_TIMEOUT_DEFAULT 300)

CMAKE_INSTALL_RPATH_USE_LINK_PATH

The cache variable CMAKE_INSTALL_RPATH_USE_LINK_PATH is a built-in CMake variable that determines if the paths for external libraries (i.e. from TPLs) is put into the installed library RPATHS (see RPATH Handling). TriBITS sets the default for this to TRUE but a project can change the default back to FALSE by setting the following in the project's <projectDir>/ProjectName.cmake file:

set(CMAKE_INSTALL_RPATH_USE_LINK_PATH_DEFAULT FALSE)

MPI_EXEC_MAX_NUMPROCS

The variable MPI_EXEC_MAX_NUMPROCS gives the maximum number of processes for an MPI test that will be allowed as defined by tribits_add_test() and tribits_add_advanced_test(). The TriBITS default is set to be 4 (for no good reason really but it needs to stay that way for backward compatibility). This default can be changed by setting:

set(MPI_EXEC_MAX_NUMPROCS_DEFAULT <newDefaultMax>)

While this default can be changed for the project as a whole on all platforms, it is likely better to change this default on a machine-by-machine basis to correspond to the load that can be accommodated by a given machine (or class of machines). For example if a given machine has 64 cores, a reasonable number for MPI_EXEC_MAX_NUMPROCS_DEFAULT is 64.

TRIBITS_HANDLE_TRIBITS_DEPRECATED_CODE

Determines how the function tribits_deprecated() behaves. To change the default behavor, such as call message(FATAL_ERROR ...), set:

set(TRIBITS_HANDLE_TRIBITS_DEPRECATED_CODE_DEFAULT  FATAL_ERROR)

in the project's <projectDir>/ProjectName.cmake file, or <projectDir>/CMakeLists.txt file, or on the individual package basis in its <packageDir>/CMakeLists.txt file.

12.2   TriBITS Macros and Functions

The following subsections give detailed documentation for the CMake macros and functions that make up the core TriBITS system. These are what are used by TriBITS project developers in their CMakeLists.txt and other files. All of these functions and macros should be automatically available when processing the project's and package's variables files if used properly. Therefore, no explicit include() statements should be needed other than the initial include of the TriBITS.cmake file in the top-level <projectDir>/CMakeLists.txt file so the command tribits_project() can be executed.

tribits_add_advanced_test(
  <testNameBase>
  TEST_0 (EXEC <execTarget0> | CMND <cmndExec0>) ...
  [TEST_1 (EXEC <execTarget1> | CMND <cmndExec1>) ...]
  ...
  [TEST_N (EXEC <execTargetN> | CMND <cmndExecN>) ...]
  [OVERALL_WORKING_DIRECTORY (<overallWorkingDir> | TEST_NAME)]
  [SKIP_CLEAN_OVERALL_WORKING_DIRECTORY]
  [FAIL_FAST]
  [RUN_SERIAL]
  [KEYWORDS <keyword1> <keyword2> ...]
  [COMM [serial] [mpi]]
  [OVERALL_NUM_MPI_PROCS <overallNumProcs>]
  [OVERALL_NUM_TOTAL_CORES_USED <overallNumTotalCoresUsed>]
  [CATEGORIES <category0> <category1> ...]
  [HOST <host0> <host1> ...]
  [XHOST <host0> <host1> ...]
  [HOSTTYPE <hosttype0> <hosttype1> ...]
  [XHOSTTYPE <hosttype0> <hosttype1> ...]
  [EXCLUDE_IF_NOT_TRUE <varname0> <varname1> ...]
  [DISABLED <messageWhyDisabled>]
  [FINAL_PASS_REGULAR_EXPRESSION "<regex>" |
    FINAL_FAIL_REGULAR_EXPRESSION "<regex>"]
  [ENVIRONMENT <var1>=<value1> <var2>=<value2> ...]
  [TIMEOUT <maxSeconds>]
  [LIST_SEPARATOR <sep>]
  [ADDED_TEST_NAME_OUT <testName>]
  )

TEST_<idx>
   (EXEC <exeRootName> [NOEXEPREFIX] [NOEXESUFFIX] [ADD_DIR_TO_NAME]
          [DIRECTORY <dir>]
      | CMND <cmndExec>)
   [ARGS "<arg0>" "<arg1>" ... "<argn>"]
   [MESSAGE "<message>"]
   [WORKING_DIRECTORY <workingDir>]
   [SKIP_CLEAN_WORKING_DIRECTORY]
   [NUM_MPI_PROCS <numProcs>]
   [NUM_TOTAL_CORES_USED <numTotalCoresUsed>]
   [OUTPUT_FILE <outputFile>]
   [NO_ECHO_OUTPUT]]
   [PASS_ANY
     | PASS_REGULAR_EXPRESSION "<regex0>" "<regex1>" ...
     | PASS_REGULAR_EXPRESSION_ALL "<regex0>" "<regex1>" ...
     | STANDARD_PASS_OUTPUT ]
   [FAIL_REGULAR_EXPRESSION "<regex0>" "<regex1>" ...]
   [ALWAYS_FAIL_ON_NONZERO_RETURN | ALWAYS_FAIL_ON_ZERO_RETURN]
   [WILL_FAIL]

TEST_<idx>
  COPY_FILES_TO_TEST_DIR <file0> <file1> ... <filen>
  [SOURCE_DIR <srcDir>]
  [DEST_DIR <destDir>]