motiejus/hermetic_cc_toolchain
1
Fork 0
Code Releases Activity
macos
hermetic_cc_toolchain/ci/launcher

28 lines
748 B
Plaintext
Raw Permalink Normal View History

zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
#!/usr/bin/env bash
Re-license portions of the code to Apache 2.0 Your old version of the software released under the old license can still be used under the terms of the old license. I have received the following statement from the past contributors Luis Holanda, Jeremy Volkman and Fabian Hahn: I hereby confirm that I am the copyright holder or authorized by the copyright holder of this contribution. As such I hereby confirm that all contributions made to bazel-zig-cc by me or on behalf of me, hereby is licensed under the Apache 2.0 License [http://www.apache.org/licenses/LICENSE-2.0]. I am aware that my previous contributions will still be available under the MIT license as well. I confirm that I am aware of the bazel-zig-cc teams intention to release bazel-zig-cc under the Apache 2.0 License from release [1.0] and onwards, and that the bazel-zig-cc no longer will accept contributions made under the MIT and that any future submissions from myself will be considered to be licensed under Apache 2.0 unless I expressly state otherwise. Copyright and re-licensing for Google and Uber employees has been resolved internally. Since not all contributors took action to re-license the code yet, portions of bazel-zig-cc remain licensed as MIT. I have started this project during my personal time with my personal resources. I am now handing over all copyrights to this code to Uber Technologies, Inc.
2023-03-02 23:42:35 +02:00
# Copyright 2023 Uber Technologies, Inc.
# Licensed under the Apache License, Version 2.0
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
set -xeuo pipefail
More CI (#3) - ci only: use a separate zig cache dir for each run - use `tools/bazel` everywhere. - remove arm64 tests. They don't give enough value to be worth the brittle environment. - remove windows execution, except for launcher. Ditto. I will probably bring them back.
2023-03-06 17:32:48 +02:00
echo "--- which zig"
ZIG=${ZIG:-$(tools/bazel run "$@" --run_under=echo @zig_sdk//:zig)}
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
More CI (#3) - ci only: use a separate zig cache dir for each run - use `tools/bazel` everywhere. - remove arm64 tests. They don't give enough value to be worth the brittle environment. - remove windows execution, except for launcher. Ditto. I will probably bring them back.
2023-03-06 17:32:48 +02:00
echo "--- compile launcher.zig for various architectures"
CI: fix tests
2023-01-31 04:01:01 +02:00
for target in \
aarch64-linux-gnu.2.19 \
aarch64-macos-none \
x86_64-linux-gnu.2.19 \
x86_64-macos-none \
x86_64-windows-gnu
do
$ZIG build-exe -fno-emit-bin -target $target toolchain/launcher.zig
done
More CI (#3) - ci only: use a separate zig cache dir for each run - use `tools/bazel` everywhere. - remove arm64 tests. They don't give enough value to be worth the brittle environment. - remove windows execution, except for launcher. Ditto. I will probably bring them back.
2023-03-06 17:32:48 +02:00
echo "--- zig fmt --check toolchain/launcher.zig"
launcher: remove --color=on
2023-02-24 12:01:49 +02:00
$ZIG fmt --check toolchain/launcher.zig
ci/launcher: check fmt
2023-02-24 11:53:10 +02:00
More CI (#3) - ci only: use a separate zig cache dir for each run - use `tools/bazel` everywhere. - remove arm64 tests. They don't give enough value to be worth the brittle environment. - remove windows execution, except for launcher. Ditto. I will probably bring them back.
2023-03-06 17:32:48 +02:00
echo "--- zig test toolchain/launcher.zig"
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
# until bazel-zig-cc gets a zig toolchain, run launcher's unit tests here.
$ZIG test toolchain/launcher.zig
Copy Permalink