1
Commit Graph

501 Commits

Author SHA1 Message Date
Motiejus Jakštys
1fb93e089c toolchain wrapper: zig_exe path is relative
Just like we know the path to zig_lib_dir, we do know the path to
zig_exe. Lets use that.

This was surfaced by experimenting with
`--experimental_use_hermetic_linux_sandbox`.
`repository_ctx.path("zig")` would return the real file system path to
zig (outside of the sandbox), which is not good when the sandbox is
hermetic.

Co-developed-by: Fabian Hahn <fabian@hahn.graphics>
2022-12-11 18:28:38 +02:00
Fabian Hahn
43b6e1c0c1 fix windows include root path 2022-12-11 18:28:19 +02:00
Motiejus Jakštys
8bc78753e2 ci/test: quote {}
This is recommended in the man page
2022-12-11 07:20:14 +02:00
Motiejus Jakštys
6feba23aee build.yml: add file
this will fix the last test in ci/test
2022-12-11 07:12:43 +02:00
Motiejus Jakštys
70ef4dd54e remove zig_include_root
It is now consistent across all 5 platforms we support:

    $ for x in *324*.tar.xz; do tar -tf $x | grep libc/ | head -5; done
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-linux-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-linux-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-macos-aarch64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/ntddstor-uuid.c
    zig-macos-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/mingw/libsrc/bth-uuid.c
    $ for x in *324*.zip; do unzip -l $x | grep libc/ | head -5; done
            0  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/
            0  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/
       301018  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.11.tbd
       309693  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.12.tbd
       326079  2022-11-28 19:26   zig-windows-x86_64-0.11.0-dev.324+f61c5f3f5/lib/libc/darwin/libSystem.13.tbd
2022-12-07 12:17:44 +02:00
Motiejus Jakštys
9eaebe0ad2 rules_go: cherry-pick a gorace workaround 2022-11-30 21:10:08 +02:00
Motiejus Jakštys
bf88ddb2d5 ci/test: fail on error 2022-11-29 14:23:45 +02:00
Motiejus Jakštys
6bd588c07b upgrade to zig 0.11.0-dev.324+f61c5f3f5
includes a zig upstream fix for absolute directories: https://github.com/ziglang/zig/issues/13050
2022-11-29 13:51:57 +02:00
Motiejus Jakštys
0f981d591e WORKSPACE: bump go to 1.19.3 2022-11-28 11:06:03 +02:00
Motiejus Jakštys
88e7e47ed2 zig_include_root: fixes
- do not pass where it's unnecessary
- remove trailing slash. That conflicts with ziglang/zig/pull/13596
2022-11-21 06:08:22 +02:00
Motiejus Jakštys
dd4239407e lint 2022-11-17 15:47:46 +02:00
Motiejus Jakštys
87fb993408 zig build: add -target automatically 2022-11-17 14:04:25 +02:00
Motiejus Jakštys
d195402030 zig build-exe, build-lib, build-obj
This change makes it possible to build zig sources from Bazel rules by
calling the right tool.
2022-11-17 13:51:31 +02:00
Motiejus Jakštys
50103497f6 upgrade zig sdk
The uber1 was broken on x86_64-macos
2022-10-19 11:38:46 +03:00
Motiejus Jakštys
abf00a525c contrib/build-and-archive: fix lib dir for macos-x86_64 2022-10-13 15:50:19 +03:00
Motiejus Jakštys
557b726d62 shellcheck 2022-10-13 12:48:43 +03:00
Motiejus Jakštys
7383c4984e ci/test: add tests for exact expected artifacts.
Cache reuse in zig is currently quite fragile (we are using a forked
version with github.com/ziglang/zig/issues/13051), so ensure that it
does not regress when zig SDK upgrades.
2022-10-13 12:41:07 +03:00
Motiejus Jakštys
eba4adbbb4 WORKSPACE: upgrade to go1.19.2 2022-10-13 12:30:06 +03:00
Motiejus Jakštys
ae2746ebeb zig_include_root: support windows too
No real change, just make the logic consistent with Linux/Darwin.

This prepares us for windows-aarch64, which will probably have lib in
lib/zig.
2022-10-13 06:11:25 +03:00
Motiejus Jakštys
fed463e1d0 bump zig to 4301+uber1
This is a patched zig release with 13051[1]. While we are waiting for a
resolution on #13050 (which @andrewrk is working on).

[1]: https://patch-diff.githubusercontent.com/raw/ziglang/zig/pull/13051
2022-10-13 05:57:56 +03:00
Motiejus Jakštys
e5eff0c911 zig include root: it depends on the OS
Previously the toolchain wrappers used to assume the zig_lib_dir is
always in "lib". However, it depends on the OS: macos-x86_64 and
linux-aarch64 have it in "lib/zig".

Update the wrapper scripts to reflect that.
2022-10-11 14:57:44 +03:00
Motiejus Jakštys
359cb0fe6d README: fix copy/pasta 2022-10-08 15:01:00 +03:00
c0a58691eb README: rollback to v0.9.2
v1.0-rc* all have a performance regression due to
https://github.com/ziglang/zig/issues/13050; follow along there.
2022-10-07 12:28:07 +02:00
Motiejus Jakštys
5743865c26 Revert "resolve ZIG_LIB_DIR before passing it to zig"
This reverts commit 3a8c2eecfe.

Causes the dependency paths to be absolute.
2022-10-05 14:48:19 +03:00
Motiejus Jakštys
3a8c2eecfe resolve ZIG_LIB_DIR before passing it to zig
That way the zig's cache keys will be the same for all sandboxes.

Also see https://github.com/ziglang/zig/issues/13050#issuecomment-1268074032
2022-10-05 14:24:19 +03:00
Motiejus Jakštys
c5e3c25015 tests: add linux_arm64_musl, simplify BUILD 2022-10-03 16:24:59 +03:00
Motiejus Jakštys
c9d2442143 tests: remove linux_amd64_gnu.2.31
It does not add anything beyond linux_amd64_gnu.2.28.
2022-10-03 06:48:12 +03:00
Motiejus Jakštys
50ca49181f Zig cache hacks/optimizations
Zig does not always share `libc++.a` and the glibc shims. We have observed
to be caused for 2 reasons:

- For some commands Go `chdir`s to `/tmp/`. This causes `ZIG_LIB_DIR` to be
  absolute, which blows the cache key to find the right `libc++.a` (because Zig
  thinks we are using a different lib dir to compile libc++). This is currently
  worked around in `toolchain/defs.bzl` by overfitting to Go and returning
  early from that particular invocation.
- Sometimes Bazel's sandbox messes up Zig's cache keys. If one runs without the
  sandbox (`--spawn_strategy=standalone`), the cache hit rate and thus the
  build time are much better.

This is actively investigated. I am adding `--spawn_strategy=standalone`
to CI to see the speedup that it provides. I will rollback it later.
2022-10-03 05:50:53 +03:00
Motiejus Jakštys
f30a3afdde remove redundant flags
- -target
- linkopts
2022-10-01 23:27:58 +03:00
Motiejus Jakštys
b1209d0165 gohack only in tools/<arch>/c++ 2022-10-01 18:53:38 +03:00
Motiejus Jakštys
8919c5b703 fix tool paths for different OSes/environments 2022-10-01 14:39:59 +03:00
Motiejus Jakštys
9b510810cc split all tools to per-codename subdirectories
Go ignores CFLAGS for some commands. That causes unnecessary build of
glibc shims (and libc++.a).

In the GCC days cross-compiler toolchains used to expose a "tool" per
architecture. Let's do the same here. Then Go cannot cheat with skipping
CFLAGS which we normally *always* expect.

The wrapper code is getting gnarly, I know. But it still fits in my head
somehow, so we can still keep adding to it.
2022-10-01 14:35:27 +03:00
Motiejus Jakštys
e96f5bb59c lint 2022-10-01 05:31:50 +03:00
Motiejus Jakštys
e9638443c6 tools wrapper: fix for windows
'common' is not a thing in Windows; but it needs to be defined.

This makes v1.0.0-rc3 broken for Windows, and not useful (yet) for
anyone else but me. Revert the README update too.
2022-10-01 05:12:45 +03:00
Motiejus Jakštys
53975784ea ci: catch easy problems early 2022-09-30 15:18:21 +03:00
Motiejus Jakštys
c4ab1cfaea buildifier 2022-09-30 15:17:49 +03:00
Motiejus Jakštys
873a703fc1 Update release notes for v1.0.0-rc3 2022-09-30 15:03:51 +03:00
Motiejus Jakštys
9e8d89b40d CGo-specific: reduce unneeded compilation
This adds an ugly workaround for
https://go-review.googlesource.com/c/go/+/436884 ; fingers crossed.

Impact: ~20-25s reduced first build time for a particular
architecture+glibc.
2022-09-30 14:24:56 +03:00
Motiejus Jakštys
ae57dac00d ci/lint: make it fail in CI
It would "fix" the files on CI and not fail. ci/lint should verify they
are OK first, and give guidance to fix them.

also, `shellcheck -x` is no longer necessary; we can use the plain one
now.
2022-09-29 12:11:34 +03:00
Motiejus Jakštys
b9ded4caeb Update release notes for v1.0.0-rc2 2022-09-29 11:32:57 +03:00
Fabian Hahn
04b95526ae fix Windows host support with relative paths 2022-09-29 11:32:19 +03:00
Motiejus Jakštys
d841763745 update versions and sizes 2022-09-27 15:43:24 +03:00
Motiejus Jakštys
530dd1a825 Update release notes for v1.0.0-rc1 2022-09-27 15:41:06 +03:00
Motiejus Jakštys
08b02cb7d2 rename absolute_path to zig_path
this is not an absolute path.
2022-09-27 15:37:28 +03:00
Motiejus Jakštys
0302fb630a defs: fix import paths
Empirically these need to come from most specfic to least specific.

The error message is as follows:

    In file included from test/c/main.c:1:
    In file included from external/zig_sdk/lib/libcxx/include/stdio.h:107:
    In file included from external/zig_sdk/lib/libc/include/generic-glibc/stdio.h:38:
    external/zig_sdk/lib/libc/include/generic-glibc/bits/types.h:139:3: error:
    # error
      ^
    external/zig_sdk/lib/libc/include/generic-glibc/bits/types.h:145:1: error: unknown type name '__STD_TYPE'
    __STD_TYPE __DEV_T_TYPE __dev_t;        /* Type of device numbers.  */

Dissected `generic-glibc/bits/types.h:#error`:

    #if __WORDSIZE == 32
    <...>
    # define __STD_TYPE		__extension__ typedef
    #elif __WORDSIZE == 64
    <...>
    # define __STD_TYPE		typedef
    #else
    # error
    #endif

So we do not have the `__WORDSIZE`. Where does that come from? Probably from a
directory that has an `x86_64` in it. How does that get included? Let's start
with `lib/libcxx/include/stdio.h`:

	16 #include_next <stdio.h>

Now previously our `c++` command line looked like this:
    external/zig_sdk/tools/c++ \
    <...>
    -Iexternal/zig_sdk/lib/include \
    -Iexternal/zig_sdk/lib/libcxx/include \
    -Iexternal/zig_sdk/lib/libcxxabi/include \
    -Iexternal/zig_sdk/lib/libunwind/include \
    -Iexternal/zig_sdk/lib/libc/include/generic-glibc \
    -Iexternal/zig_sdk/lib/libc/include/any-linux-any \
    -Iexternal/zig_sdk/lib/libc/include/x86_64-linux-gnu \
    -Iexternal/zig_sdk/lib/libc/include/x86_64-linux-any \
    -Iexternal/zig_sdk/lib/libc/include/x86-linux-any \
    -Iexternal/zig_sdk/glibc-hacks \
    <...>

So the next place it will find `stdio.h` is in `generic-glibc`, which already
uses the `__WORDSIZE`. If we make the "next" include to be the arch-specific
one instead of the generic-glibc, things start compiling again.

Fix the same fo musl.
2022-09-27 10:15:02 +03:00
Ken Micklas
ab59c18d60 WIP: use relative paths for tools 2022-09-27 10:14:59 +03:00
Motiejus Jakštys
f6b16c386f upgrade to zig 0.10.0-dev.4166+cae76d829
- llvm 15
- uses less space for /tmp/bazel-zig-cc
2022-09-25 13:51:59 +03:00
Motiejus Jakštys
9dbba15064 partially revert c023c217a5
The `toolchain/private/cc_toolchains.bzl` changes were added by accident.
2022-09-22 13:32:17 +03:00
Motiejus Jakštys
026d234bc2 Revert "bump to zig 0.10.0-dev.3999+fda6d4477"
This reverts commit f161caf870.

This version has a bug that's been fixed upstream, but a new nightly did
not come out yet: https://github.com/ziglang/zig/issues/12858
2022-09-22 11:57:39 +03:00
Motiejus Jakštys
b3a8b5b169 update README
- update fixed upstream issues
- add the mailing list e-mail address
2022-09-22 11:57:13 +03:00