Finding C++ Build Bottlenecks with ClangBuildAnalyzer

Background

Recently, a C++ project with only a few dozen files took a very long time to compile. Even with multi-threaded builds on an M1 Pro, it still took nearly 2–4 minutes. After putting up with it for a while, I decided to find out why.

As projects grow and dependencies become more complex, C++ build times increase. The open-source tool ClangBuildAnalyzer can effectively analyze time spent in files, functions, and header inclusion expansion, providing a reference for build-time optimization.

Of course, besides ClangBuildAnalyzer, there are other tools as well. In my opinion, its advantages are that it is simple to use, cross-platform, open source, and easy to build. The downside is that it only supports Clang.

For C++ build optimization, Meituan's engineering team has a very in-depth article worth studying and applying in practice: C++服务编译耗时优化原理及实践

Build

GitHub: https://github.com/aras-p/ClangBuildAnalyzer — download the source and build with CMake:

mkdir build && cd build
cmake .. && make

The ClangBuildAnalyzer executable will be generated in the build directory.

Preparation

To analyze build time, you also need to add -ftime-trace to your project's Clang compile flags. If you use Xcode, add the -ftime-trace option under Xcode project -> Build Settings -> Other C++ Flags.

Usage

Start tracing — run in the terminal:

./ClangBuildAnalyzer --start <artifacts_folder>

<artifacts_folder> refers to the directory where compiled intermediate .o (object) files are generated. ClangBuildAnalyzer will start tracing and analyzing these files to determine build time and dependencies.

Run the build:

Compile your project at this point.

Stop tracing — run in the terminal:

./ClangBuildAnalyzer --stop <artifacts_folder> analy_log.log

<artifacts_folder> should be the same as in --start. analy_log.log is the trace log file and can be named as you like.

Analyze build time:

./ClangBuildAnalyzer --analyze analy_log.log

Build time analysis report:

With this report, you can target build optimization. After each round of optimization, run the analysis again until build time is acceptable. The report roughly includes:

Total parsing time;
Total codegen and optimization time;
Per-file compile time;
Template instantiation time;
Function/method compile time;
Header dependency inclusion time;

Analyzing build trace from 'artifacts/FullCapture.bin'...
**** Time summary:
Compilation (7664 times):
  Parsing (frontend):         2118.9 s
  Codegen & opts (backend):   1204.1 s

**** Files that took longest to parse (compiler frontend):
  5084 ms: cycles_scene.build/RelWithDebInfo/volume.o
  4471 ms: extern_ceres.build/RelWithDebInfo/covariance_impl.o
  4225 ms: bf_intern_libmv.build/RelWithDebInfo/resect.o
  4121 ms: bf_blenkernel.build/RelWithDebInfo/volume_to_mesh.o
 
**** Files that took longest to codegen (compiler backend):
 47123 ms: bf_blenkernel.build/RelWithDebInfo/volume.o
 39617 ms: bf_blenkernel.build/RelWithDebInfo/volume_to_mesh.o
 37488 ms: bf_modifiers.build/RelWithDebInfo/MOD_volume_displace.o
  30676 ms: bf_gpu.build/RelWithDebInfo/gpu_shader_create_info.o

**** Templates that took longest to instantiate:
 11172 ms: fmt::detail::vformat_to<char> (142 times, avg 78 ms)
  6662 ms: std::__scalar_hash<std::_PairT, 2>::operator() (3549 times, avg 1 ms)
  6281 ms: std::__murmur2_or_cityhash<unsigned long, 64>::operator() (3549 times, avg 1 ms)
  5757 ms: std::basic_string<char>::basic_string (3597 times, avg 1 ms)
  5541 ms: blender::CPPType::to_static_type_tag<float, blender::VecBase<float, ... (70 times, avg 79 ms)

**** Template sets that took longest to instantiate:
 32421 ms: std::unique_ptr<$> (30461 times, avg 1 ms)
 30098 ms: Eigen::MatrixBase<$> (8639 times, avg 3 ms)
 27524 ms: Eigen::internal::call_assignment_no_alias<$> (2397 times, avg 11 ms)

**** Functions that took longest to compile:
 28359 ms: gpu_shader_create_info_init (source/blender/gpu/intern/gpu_shader_create_info.cc)
  4090 ms: ccl::GetConstantValues(ccl::KernelData const*) (intern/cycles/device/metal/kernel.mm)
  3996 ms: gpu_shader_dependency_init (source/blender/gpu/intern/gpu_shader_dependency.cc)

**** Function sets that took longest to compile / optimize:
 10606 ms: bool openvdb::v10_0::tree::NodeList<$>::initNodeChildren<$>(openvdb:... (470 times, avg 22 ms)
  9640 ms: void tbb::interface9::internal::dynamic_grainsize_mode<$>::work_bala... (919 times, avg 10 ms)
  9459 ms: void tbb::interface9::internal::dynamic_grainsize_mode<$>::work_bala... (715 times, avg 13 ms)
  7279 ms: blender::Vector<$>::realloc_to_at_least(long long) (1840 times, avg 3 ms)
 
**** Expensive headers:
261580 ms: /Developer/SDKs/MacOSX13.1.sdk/usr/include/c++/v1/algorithm (included 3389 times, avg 77 ms), included via:
  341x: BKE_context.h BLI_string_ref.hh string 
  180x: DNA_mesh_types.h BLI_math_vector_types.hh array 
  125x: DNA_space_types.h DNA_node_types.h DNA_node_tree_interface_types.h BLI_function_ref.hh BLI_memory_utils.hh 
  ...

188777 ms: /Developer/SDKs/MacOSX13.1.sdk/usr/include/c++/v1/string (included 3447 times, avg 54 ms), included via:
  353x: BKE_context.h BLI_string_ref.hh 
  184x: DNA_mesh_types.h BLI_offset_indices.hh BLI_index_mask.hh BLI_linear_allocator.hh BLI_string_ref.hh 
  131x: DNA_node_types.h DNA_node_tree_interface_types.h BLI_span.hh 
  ...

174792 ms: source/blender/makesdna/DNA_node_types.h (included 1653 times, avg 105 ms), included via:
  316x: ED_screen.hh DNA_space_types.h 
  181x: DNA_space_types.h 
  173x: <direct include>
  ...