80 lines
5.4 KiB
Markdown
80 lines
5.4 KiB
Markdown
bazel-zig-cc and llvm
|
||
---------------------
|
||
|
||
bazel-zig-cc has a performance issue when compiling many files. This repository
|
||
reproduces that.
|
||
|
||
The test
|
||
--------
|
||
|
||
Compiles 64 small binaries with [bazel-zig-cc][1] and [llvm14][2]. The tests
|
||
were run on an x86_64 8-core machine running Ubuntu 22.04.
|
||
|
||
Results
|
||
-------
|
||
|
||
Benchmark 1: bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=local //...
|
||
Time (mean ± σ): 9.257 s ± 0.287 s [User: 0.083 s, System: 0.026 s]
|
||
Range (min … max): 8.818 s … 9.515 s 5 runs
|
||
|
||
Benchmark 2: bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=local //...
|
||
Time (mean ± σ): 9.613 s ± 0.214 s [User: 0.077 s, System: 0.038 s]
|
||
Range (min … max): 9.418 s … 9.866 s 5 runs
|
||
|
||
Benchmark 3: bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=sandboxed --experimental_reuse_sandbox_directories //...
|
||
Time (mean ± σ): 13.314 s ± 0.201 s [User: 0.109 s, System: 0.028 s]
|
||
Range (min … max): 13.093 s … 13.536 s 5 runs
|
||
|
||
Benchmark 4: bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=sandboxed --experimental_reuse_sandbox_directories //...
|
||
Time (mean ± σ): 13.829 s ± 0.376 s [User: 0.093 s, System: 0.046 s]
|
||
Range (min … max): 13.573 s … 14.489 s 5 runs
|
||
|
||
Benchmark 5: bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=sandboxed //...
|
||
Time (mean ± σ): 35.101 s ± 0.365 s [User: 0.103 s, System: 0.028 s]
|
||
Range (min … max): 34.747 s … 35.673 s 5 runs
|
||
|
||
Benchmark 6: bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=sandboxed //...
|
||
Time (mean ± σ): 21.071 s ± 0.378 s [User: 0.091 s, System: 0.034 s]
|
||
Range (min … max): 20.611 s … 21.635 s 5 runs
|
||
|
||
Summary
|
||
'bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=local //...' ran
|
||
1.04 ± 0.04 times faster than 'bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=local //...'
|
||
1.44 ± 0.05 times faster than 'bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=sandboxed --experimental_reuse_sandbox_directories //...'
|
||
1.49 ± 0.06 times faster than 'bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=sandboxed --experimental_reuse_sandbox_directories //...'
|
||
2.28 ± 0.08 times faster than 'bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=sandboxed //...'
|
||
3.79 ± 0.12 times faster than 'bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=sandboxed //...'
|
||
|
||
|
||
| Command | Mean [s] | Min [s] | Max [s] | Relative |
|
||
|:---|---:|---:|---:|---:|
|
||
| `bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=local //...` | 9.257 ± 0.287 | 8.818 | 9.515 | 1.00 |
|
||
| `bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=local //...` | 9.613 ± 0.214 | 9.418 | 9.866 | 1.04 ± 0.04 |
|
||
| `bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=sandboxed --experimental_reuse_sandbox_directories //...` | 13.314 ± 0.201 | 13.093 | 13.536 | 1.44 ± 0.05 |
|
||
| `bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=sandboxed --experimental_reuse_sandbox_directories //...` | 13.829 ± 0.376 | 13.573 | 14.489 | 1.49 ± 0.06 |
|
||
| `bazel build --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux --spawn_strategy=sandboxed //...` | 35.101 ± 0.365 | 34.747 | 35.673 | 3.79 ± 0.12 |
|
||
| `bazel build --platforms=@zig_sdk//libc_aware/platform:linux_amd64_gnu.2.28 --spawn_strategy=sandboxed //...` | 21.071 ± 0.378 | 20.611 | 21.635 | 2.28 ± 0.08 |
|
||
|
||
Explanation:
|
||
|
||
* `--spawn_strategy=local` is the baseline. This is the theoretically fastest
|
||
approach (no sandbox), but unfit for production.
|
||
* `--spawn_strategy=sandboxed` is the default for Bazel. This is what happens
|
||
if one does not pass any arguments.
|
||
* `--spawn_strategy=sandboxed --experimental_reuse_sandbox_directories` is an
|
||
optimization which is meaningful on sandboxes with many files. As of Bazel 6
|
||
this has been [promoted to stable][3], so safe to use and, as one can see in
|
||
the numbers, recommended with both bazel-zig-cc and a nontrivial sysroot. As
|
||
a result, this is the most important benchmark to look at.
|
||
|
||
Flame graphs and discussion
|
||
---------------------------
|
||
|
||
Flame graphs in results/. Only of historical interest, because the original
|
||
performance issues have been resolved with a combination of zig and
|
||
bazel-zig-cc changes.
|
||
|
||
[1]: https://sr.ht/~motiejus/bazel-zig-cc
|
||
[2]: https://github.com/grailbio/bazel-toolchain
|
||
[3]: https://github.com/bazelbuild/bazel/issues/16138
|