motiejus/hermetic_cc_toolchain
1
Fork 0
Code Releases Activity
main
hermetic_cc_toolchain/ci/launcher

33 lines
863 B
Plaintext
Raw Permalink Normal View History

zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
#!/usr/bin/env bash
Re-license portions of the code to Apache 2.0 Your old version of the software released under the old license can still be used under the terms of the old license. I have received the following statement from the past contributors Luis Holanda, Jeremy Volkman and Fabian Hahn: I hereby confirm that I am the copyright holder or authorized by the copyright holder of this contribution. As such I hereby confirm that all contributions made to bazel-zig-cc by me or on behalf of me, hereby is licensed under the Apache 2.0 License [http://www.apache.org/licenses/LICENSE-2.0]. I am aware that my previous contributions will still be available under the MIT license as well. I confirm that I am aware of the bazel-zig-cc teams intention to release bazel-zig-cc under the Apache 2.0 License from release [1.0] and onwards, and that the bazel-zig-cc no longer will accept contributions made under the MIT and that any future submissions from myself will be considered to be licensed under Apache 2.0 unless I expressly state otherwise. Copyright and re-licensing for Google and Uber employees has been resolved internally. Since not all contributors took action to re-license the code yet, portions of bazel-zig-cc remain licensed as MIT. I have started this project during my personal time with my personal resources. I am now handing over all copyrights to this code to Uber Technologies, Inc.
2023-03-02 23:42:35 +02:00
# Copyright 2023 Uber Technologies, Inc.
revert "Re-license portions of the code to Apache 2.0" (#35) This (partially) reverts commit dfdb1f26805c1ccf6d1113276d40c8ce4c1fa17e
2023-04-20 03:23:25 +03:00
# Licensed under the MIT License
Re-license portions of the code to Apache 2.0 Your old version of the software released under the old license can still be used under the terms of the old license. I have received the following statement from the past contributors Luis Holanda, Jeremy Volkman and Fabian Hahn: I hereby confirm that I am the copyright holder or authorized by the copyright holder of this contribution. As such I hereby confirm that all contributions made to bazel-zig-cc by me or on behalf of me, hereby is licensed under the Apache 2.0 License [http://www.apache.org/licenses/LICENSE-2.0]. I am aware that my previous contributions will still be available under the MIT license as well. I confirm that I am aware of the bazel-zig-cc teams intention to release bazel-zig-cc under the Apache 2.0 License from release [1.0] and onwards, and that the bazel-zig-cc no longer will accept contributions made under the MIT and that any future submissions from myself will be considered to be licensed under Apache 2.0 unless I expressly state otherwise. Copyright and re-licensing for Google and Uber employees has been resolved internally. Since not all contributors took action to re-license the code yet, portions of bazel-zig-cc remain licensed as MIT. I have started this project during my personal time with my personal resources. I am now handing over all copyrights to this code to Uber Technologies, Inc.
2023-03-02 23:42:35 +02:00
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
set -xeuo pipefail
More CI (#3) - ci only: use a separate zig cache dir for each run - use `tools/bazel` everywhere. - remove arm64 tests. They don't give enough value to be worth the brittle environment. - remove windows execution, except for launcher. Ditto. I will probably bring them back.
2023-03-06 17:32:48 +02:00
echo "--- which zig"
ZIG=${ZIG:-$(tools/bazel run "$@" --run_under=echo @zig_sdk//:zig)}
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
More CI (#3) - ci only: use a separate zig cache dir for each run - use `tools/bazel` everywhere. - remove arm64 tests. They don't give enough value to be worth the brittle environment. - remove windows execution, except for launcher. Ditto. I will probably bring them back.
2023-03-06 17:32:48 +02:00
echo "--- compile launcher.zig for various architectures"
CI: fix tests
2023-01-31 04:01:01 +02:00
for target in \
aarch64-linux-gnu.2.19 \
aarch64-macos-none \
x86_64-linux-gnu.2.19 \
x86_64-macos-none \
x86_64-windows-gnu
do
launcher: use `-mcpu=baseline` This log message has been seen in a Github Actions worker: running <...>/c++ failed: signal: illegal instruction (core dumped) I conclude that the launcher was compiled on a newer CPU than used on the worker at the time. Reported-by: mp@edgeless.systems Fixes #22
2023-03-21 12:43:08 +02:00
if [[ $target == aarch64-macos-none ]]; then
mcpu=apple_a14
else
mcpu=baseline
fi
$ZIG build-exe -fno-emit-bin -target $target -mcpu=$mcpu toolchain/launcher.zig
CI: fix tests
2023-01-31 04:01:01 +02:00
done
More CI (#3) - ci only: use a separate zig cache dir for each run - use `tools/bazel` everywhere. - remove arm64 tests. They don't give enough value to be worth the brittle environment. - remove windows execution, except for launcher. Ditto. I will probably bring them back.
2023-03-06 17:32:48 +02:00
echo "--- zig fmt --check toolchain/launcher.zig"
launcher: remove --color=on
2023-02-24 12:01:49 +02:00
$ZIG fmt --check toolchain/launcher.zig
ci/launcher: check fmt
2023-02-24 11:53:10 +02:00
More CI (#3) - ci only: use a separate zig cache dir for each run - use `tools/bazel` everywhere. - remove arm64 tests. They don't give enough value to be worth the brittle environment. - remove windows execution, except for launcher. Ditto. I will probably bring them back.
2023-03-06 17:32:48 +02:00
echo "--- zig test toolchain/launcher.zig"
Rename bazel-zig-cc to hermetic_cc_toolchain (#36) As it says on the tin. Long live hermetic_cc_toolchain!
2023-04-21 17:00:03 +03:00
# until hermetic_cc_toolchain gets a zig toolchain, run launcher's unit tests here.
zig launcher: replace shell wrappers with a binary Until now we needed to maintain two versions of the zig launcher: one for Windows and one for everything else. This was problematic for two reasons: 1. I do not know powershell and thus keep breaking the Windows wrapper all the time (see git history of Fabian fixing stuff that I broke). 2. This makes bazel-zig-cc dependent on the system shell, making it not really hermetic. So the recently added `--experimental_use_hermetic_linux_sandbox` does not work with bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`, `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`. Switching to a Zig-based wrapper solves both issues, and we can do this: bazel build "$@" \ --experimental_use_hermetic_linux_sandbox \ --sandbox_add_mount_pair=/proc \ <...> Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to keep that. I will look into reducing even that dependency separately. Not all is nice and shiny though: this commit replaces ~80 LOC worth of shell scripts wrappers with a singe ~300 LOC zig program, which is arguably harder to understand. However, it is easier to change, at least for me, because it's a single file with unit tests! Most importantly, the gnarly code (which resolves paths and sets environment variables) is cross-platform. Thanks to Fabian Hahn for testing this on Windows and pointing out errors.
2022-12-20 17:00:50 +02:00
$ZIG test toolchain/launcher.zig
Copy Permalink