1
Fork 0
Commit Graph

199 Commits

Author SHA1 Message Date
Jeremy Volkman f35f5d3eeb buildifier 2023-04-26 07:32:43 +03:00
Jeremy Volkman 5a7e58af4e Move -Wl,-S to a separate strip_debug_symbols feature 2023-04-26 07:32:43 +03:00
Jeremy Volkman cff2b2660b Re-add linkopts and set -Wl,-headerpad_max_install_names on macos 2023-04-24 09:48:27 -07:00
Motiejus Jakštys 9e7efd9a1b remaining linker_version_scripts cleanup
This was missed in #30
2023-04-24 16:16:00 +03:00
Motiejus Jakštys 1f50f76cc6 bump zig sdk to 0.11.0-dev.2619+bd3e248c7
From https://github.com/marler8997/zig-unofficial-releases

I also updated the "Release Process" wiki instructing how to use it.
Long live Mr. Marler!

The launcher got a facelift because of this error:

    /code/zig-linux-x86_64-0.11.0-dev.2619+bd3e248c7/lib/std/fmt.zig:2013:9: error: function called at runtime cannot return value at comptime
            return &buf;
            ^~~~~~~~~~~
    referenced by:
        test.launcher:parseArgs: toolchain/launcher.zig:334:31
        remaining reference traces hidden; use '-freference-trace' to see all reference traces

Zig gets confused by the requirement in that test in a runtime context.
2023-04-24 15:48:07 +03:00
Motiejus Jakštys cb46744cb8 launcher: use `-mcpu=baseline`
This log message has been seen in a Github Actions worker:

      running <...>/c++ failed: signal: illegal instruction (core dumped)

I conclude that the launcher was compiled on a newer CPU than used on
the worker at the time.

Reported-by: mp@edgeless.systems

Fixes #22
2023-04-24 12:38:23 +03:00
Goni Zahavy f164afe89a Removed leftovers of `compiler_extra_includes` 2023-04-24 12:09:35 +03:00
Goni Zahavy 19c735c41a remove glibc hacks and update Zig SDK 2023-04-24 12:09:35 +03:00
Motiejus Jakštys 92067b5d85 remove @bazel_skylib dependency (again)
This was accidentally included in
d61aab183e434d48783dbaa2fdb9e39714217637
2023-04-24 12:02:35 +03:00
Motiejus Jakštys 9779650a78 Work around a known race in Zig build system
More information in the commit message and
https://github.com/ziglang/zig/issues/14815
2023-04-24 10:10:58 +03:00
Motiejus Jakštys 97a4846b58 remove `@bazel_skylib` dependency
We are using only a single trivial function `paths`, which can be
bundled. Makes things like test-zigcc[1] easier: less dependencies to
worry about.

[1]: https://git.jakstys.lt/motiejus/test-zig-cc/
2023-04-24 10:10:30 +03:00
Motiejus Jakštys e0e7a4ca46
Rename bazel-zig-cc to hermetic_cc_toolchain (#36)
As it says on the tin.

Long live hermetic_cc_toolchain!
2023-04-21 10:00:03 -04:00
Motiejus Jakštys 73a9ceccfb
revert "Re-license portions of the code to Apache 2.0" (#35)
This (partially) reverts commit dfdb1f2680
2023-04-19 20:23:25 -04:00
Motiejus Jakštys 4a42b46a99 launcher miscompilation workaround
Sometimes the launcher fails to compile with the following error
messsage:

```
error: FileNotFound
```

We cannot reproduce this in a controlled environment, but see it
happening in the wild often enough to receive repeated questions.

Since this has been escalated to Zig Software Foundation, the most
meaningful thing we can ask our users to do is apply a workaround and
wait. Let's do just that.
2023-03-14 13:52:13 +02:00
Motiejus Jakštys dfdb1f2680
Re-license portions of the code to Apache 2.0
Your old version of the software released under the old license can
still be used under the terms of the old license.

I have received the following statement from the past contributors
Luis Holanda, Jeremy Volkman and Fabian Hahn:

  I hereby confirm that I am the copyright holder or authorized by the
  copyright holder of this contribution. As such I hereby confirm that
  all contributions made to bazel-zig-cc by me or on behalf of me,
  hereby is licensed under the Apache 2.0 License
  [http://www.apache.org/licenses/LICENSE-2.0]. I am aware that my
  previous contributions will still be available under the MIT license
  as well. I confirm that I am aware of the bazel-zig-cc teams intention
  to release bazel-zig-cc under the Apache 2.0 License from release
  [1.0] and onwards, and that the bazel-zig-cc no longer will accept
  contributions made under the MIT and that any future submissions from
  myself will be considered to be licensed under Apache 2.0 unless I
  expressly state otherwise.

Copyright and re-licensing for Google and Uber employees has been
resolved internally.

Since not all contributors took action to re-license the code yet,
portions of bazel-zig-cc remain licensed as MIT.

I have started this project during my personal time with my personal
resources. I am now handing over all copyrights to this code to Uber
Technologies, Inc.
2023-03-06 11:05:13 +02:00
Motiejus Jakštys 9621da8ad6 zig sdk: use mirror.bazel.build
The current version of Zig SDK was uploaded to mirror.bazel.build via
https://github.com/bazelbuild/bazel/issues/17635 . Thanks Google!
2023-03-01 16:25:16 +02:00
Motiejus Jakštys 6ebe4747ae upgrade zig 2023-02-24 11:50:41 +02:00
Motiejus Jakštys 52b4f4b436 Merge branch 'motiejus_go1.20' 2023-02-24 11:43:00 +02:00
Motiejus Jakštys b4d067adbe launcher: support multi-for loop syntax
At the moment we need both old and new zig, so doing a way that's
compatible with both.
2023-02-23 09:50:53 +02:00
Motiejus Jakštys 53d89ba627 wip go 1.20 2023-02-10 15:44:56 +02:00
Motiejus Jakštys 11f7e1ad82 CI: fix tests 2023-01-30 18:01:01 -08:00
Motiejus Jakštys 3ba59fcb78 launcher: formatting
no real changes. Code looks a bit less ugly now.
2023-01-30 17:17:51 -08:00
Motiejus Jakštys 8d1e1c9fa6 res_search: fix for aarch64
On linux_aarch64 `res_search` is `__res_search@GLIBC_2.17`, which is
different from the x86_64 counterpart `__res_search@GLIBC_2.2.5`.

Also fix tests.
2023-01-18 22:39:35 +02:00
Motiejus Jakštys 7b0de33070 work-around res_search on older glibcs
`res_search` became a proper symbol only in glibc 2.34. Until that it
was re-defined in headers, hitting the (in-)famous
https://github.com/ziglang/zig/issues/9485

glibc 2.34+:

    resolv/resolv.h:int             res_search (const char *, int, int, unsigned char *, int)

Old glibc:

    resolv/resolv.h:#define res_search              __res_search

Also, remove the toplevel includes, as they should never be included in
the first place: the headers are included explicitly when needed.
2023-01-18 15:45:47 +02:00
Motiejus Jakštys eced0109ca zig launcher: replace shell wrappers with a binary
Until now we needed to maintain two versions of the zig launcher: one
for Windows and one for everything else. This was problematic for two
reasons:
1. I do not know powershell and thus keep breaking the Windows wrapper
   all the time (see git history of Fabian fixing stuff that I broke).
2. This makes bazel-zig-cc dependent on the system shell, making it not
   really hermetic. So the recently added
   `--experimental_use_hermetic_linux_sandbox` does not work with
   bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`,
   `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`.

Switching to a Zig-based wrapper solves both issues, and we can do this:

    bazel build "$@" \
        --experimental_use_hermetic_linux_sandbox \
        --sandbox_add_mount_pair=/proc \
        <...>

Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to
keep that. I will look into reducing even that dependency separately.

Not all is nice and shiny though: this commit replaces ~80 LOC worth of
shell scripts wrappers with a singe ~300 LOC zig program, which is
arguably harder to understand. However, it is easier to change, at least
for me, because it's a single file with unit tests! Most importantly,
the gnarly code (which resolves paths and sets environment variables) is
cross-platform.

Thanks to Fabian Hahn for testing this on Windows and pointing out
errors.
2023-01-03 11:46:58 +02:00
Motiejus Jakštys 872bf302b0 reduce number of zig tools
* cc: bazel supports only cpp
* wasm-ld: we do not support wasm here. Can be re-added with proper target config.
* coff: we use ld64.lld for some reason (I don't know enough about Windows to tell)

Also, now creating the tools where it makes sense to add for the target only.
2022-12-18 06:13:32 +02:00
Motiejus Jakštys bd243aad71 simplify zig wrapper
While the original intention to be "xdg-friendly" was honorable, it
never worked in practice. Bazel has a tendency to remove almost all
environment variables during the build, causing only the fallback to
remain (i.e. all zig's cache to be put to /tmp/bazel-zig-cc).

If we just accept the world as is, we can get rid of half of the shell
wrappers.
2022-12-18 05:51:55 +02:00
Jeremy Volkman cc8f113489 Use artifact_name_pattern to specify proper macos and windows artifact names.
On macos, dynamic libraries are generated as "libfoo.dylib".

On windows, executables end with ".exe", static libraries end with ".lib",
and dynamic libraries end with ".dll".
2022-12-18 05:29:04 +02:00
Jeremy Volkman 6ee01496be Use -mcpu=apple_m1 when targeting macos + aarch64 2022-12-18 05:12:24 +02:00
Jeremy Volkman cba7c5fcae Rename _target_darwin to _target_macos
This aligns with bazel's @platforms//os:macos
2022-12-18 05:11:05 +02:00
Jeremy Volkman 0db5d2d9e6 Adds supports_dynamic_linker feature
With this feature, cc_library generates a .so as well as a .a and the
"dynamic_library" output group is enabled.
2022-12-18 05:07:59 +02:00
Motiejus Jakštys 947402c843 bump zig to 0.11.0-dev.811+8ff9284c4
includes 8ff9284c46, a fix for cache poisoning when building glibc stubs
2022-12-15 22:10:09 +02:00
Motiejus Jakštys 79812dc567 includes: add `libcxxabi`
Otherwise v8 complains.
2022-12-14 00:02:23 +02:00
Motiejus Jakštys 66b9635ca8 lint 2022-12-13 18:26:56 +02:00
Motiejus Jakštys 00e126ec85 don't forget libcxx and ar 2022-12-13 17:19:42 +02:00
Motiejus Jakštys 65a8d7c4bb toolchain files: reduce the number of files in the sandbox
We already know which headers are necessary, so it's wasteful to put the
full zig_sdk to every sandbox.

This reduces the number of files in `external/zig_sdk` from 15k to 4k or
6k, depending on the action.
2022-12-13 16:40:03 +02:00
Motiejus Jakštys 1fb93e089c toolchain wrapper: zig_exe path is relative
Just like we know the path to zig_lib_dir, we do know the path to
zig_exe. Lets use that.

This was surfaced by experimenting with
`--experimental_use_hermetic_linux_sandbox`.
`repository_ctx.path("zig")` would return the real file system path to
zig (outside of the sandbox), which is not good when the sandbox is
hermetic.

Co-developed-by: Fabian Hahn <fabian@hahn.graphics>
2022-12-11 18:28:38 +02:00
Fabian Hahn 43b6e1c0c1 fix windows include root path 2022-12-11 18:28:19 +02:00
Motiejus Jakštys 70ef4dd54e remove zig_include_root
It is now consistent across all 5 platforms we support:

    $ for x in *324*.tar.xz; do tar -tf $x | grep libc/ | head -5; done
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    $ for x in *324*.zip; do unzip -l $x | grep libc/ | head -5; done
            0  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
            0  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/
       301018  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.11.tbd
       309693  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.12.tbd
       326079  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.13.tbd
2022-12-07 12:17:44 +02:00
Motiejus Jakštys 6bd588c07b upgrade to zig 0.11.0-dev.324+f61c5f3f5
includes a zig upstream fix for absolute directories: https://github.com/ziglang/zig/issues/13050
2022-11-29 13:51:57 +02:00
Motiejus Jakštys 88e7e47ed2 zig_include_root: fixes
- do not pass where it's unnecessary
- remove trailing slash. That conflicts with ziglang/zig/pull/13596
2022-11-21 06:08:22 +02:00
Motiejus Jakštys dd4239407e lint 2022-11-17 15:47:46 +02:00
Motiejus Jakštys 87fb993408 zig build: add -target automatically 2022-11-17 14:04:25 +02:00
Motiejus Jakštys d195402030 zig build-exe, build-lib, build-obj
This change makes it possible to build zig sources from Bazel rules by
calling the right tool.
2022-11-17 13:51:31 +02:00
Motiejus Jakštys 50103497f6 upgrade zig sdk
The uber1 was broken on x86_64-macos
2022-10-19 11:38:46 +03:00
Motiejus Jakštys ae2746ebeb zig_include_root: support windows too
No real change, just make the logic consistent with Linux/Darwin.

This prepares us for windows-aarch64, which will probably have lib in
lib/zig.
2022-10-13 06:11:25 +03:00
Motiejus Jakštys fed463e1d0 bump zig to 4301+uber1
This is a patched zig release with 13051[1]. While we are waiting for a
resolution on #13050 (which @andrewrk is working on).

[1]: https://patch-diff.githubusercontent.com/raw/ziglang/zig/pull/13051
2022-10-13 05:57:56 +03:00
Motiejus Jakštys e5eff0c911 zig include root: it depends on the OS
Previously the toolchain wrappers used to assume the zig_lib_dir is
always in "lib". However, it depends on the OS: macos-x86_64 and
linux-aarch64 have it in "lib/zig".

Update the wrapper scripts to reflect that.
2022-10-11 14:57:44 +03:00
Motiejus Jakštys 5743865c26 Revert "resolve ZIG_LIB_DIR before passing it to zig"
This reverts commit 3a8c2eecfe.

Causes the dependency paths to be absolute.
2022-10-05 14:48:19 +03:00
Motiejus Jakštys 3a8c2eecfe resolve ZIG_LIB_DIR before passing it to zig
That way the zig's cache keys will be the same for all sandboxes.

Also see https://github.com/ziglang/zig/issues/13050#issuecomment-1268074032
2022-10-05 14:24:19 +03:00