motiejus
/
hermetic_cc_toolchain
1
Fork 0
Code Releases Activity
hermetic_cc_toolchain/ci/test

33 lines
1.1 KiB
Plaintext
Raw Normal View History

ci/test: add tests for exact expected artifacts. Cache reuse in zig is currently quite fragile (we are using a forked version with github.com/ziglang/zig/issues/13051), so ensure that it does not regress when zig SDK upgrades.
2022-10-13 12:39:17 +03:00
#!/usr/bin/env bash
Re-license portions of the code to Apache 2.0 Your old version of the software released under the old license can still be used under the terms of the old license. I have received the following statement from the past contributors Luis Holanda, Jeremy Volkman and Fabian Hahn: I hereby confirm that I am the copyright holder or authorized by the copyright holder of this contribution. As such I hereby confirm that all contributions made to bazel-zig-cc by me or on behalf of me, hereby is licensed under the Apache 2.0 License [http://www.apache.org/licenses/LICENSE-2.0]. I am aware that my previous contributions will still be available under the MIT license as well. I confirm that I am aware of the bazel-zig-cc teams intention to release bazel-zig-cc under the Apache 2.0 License from release [1.0] and onwards, and that the bazel-zig-cc no longer will accept contributions made under the MIT and that any future submissions from myself will be considered to be licensed under Apache 2.0 unless I expressly state otherwise. Copyright and re-licensing for Google and Uber employees has been resolved internally. Since not all contributors took action to re-license the code yet, portions of bazel-zig-cc remain licensed as MIT. I have started this project during my personal time with my personal resources. I am now handing over all copyrights to this code to Uber Technologies, Inc.
2023-03-02 23:42:35 +02:00
# Copyright 2023 Uber Technologies, Inc.
# Licensed under the Apache License, Version 2.0
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
set -xeuo pipefail
ci/test: add tests for exact expected artifacts. Cache reuse in zig is currently quite fragile (we are using a forked version with github.com/ziglang/zig/issues/13051), so ensure that it does not regress when zig SDK upgrades.
2022-10-13 12:39:17 +03:00
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
BAZEL_ZIG_CC_CACHE_PREFIX=${BAZEL_ZIG_CC_CACHE_PREFIX:-/tmp/bazel-zig-cc}
mkdir -p "${BAZEL_ZIG_CC_CACHE_PREFIX}"
# check a very hermetic setup with a single target. Re-building all of
# them takes a long time, so using only one. If we ever decide to build all
# targets, we will need to exclude Go, since go dynamically links to glibc on
# linux.
bazel build "$@" \
--experimental_use_hermetic_linux_sandbox \
--sandbox_add_mount_pair=/proc \
//test/c:which_libc_linux_amd64_gnu.2.19
# then test everything else with the standard sandbox
ci/test: add tests for exact expected artifacts. Cache reuse in zig is currently quite fragile (we are using a forked version with github.com/ziglang/zig/issues/13051), so ensure that it does not regress when zig SDK upgrades.
2022-10-13 12:39:17 +03:00
bazel test "$@" ...
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
# Ensure that github.com/ziglang/zig/issues/13050 does not regress
ci/test: show how bad the artifact situation is
2023-02-24 12:41:32 +02:00
find "$BAZEL_ZIG_CC_CACHE_PREFIX" -name mutex_destructor.o -execdir file '{}' \; | \
sort | uniq -c | sort -rn > /tmp/got_cache
diff -u ci/testdata/want_cache /tmp/got_cache || {
>&2 echo "ERROR: unexpected artifacts. This is TODO."
# TODO: Go 1.20 regressed this. Find a way to re-enable. See README.
#exit 1
exit 0
}