Subject: [PATCH bazel-zig-cc 0/1] zig launcher: replace shell wrappers with a binary

From: ~motiejus <[email-masked]>
Date: Fri, 23 Dec 2022 11:04:31 +0000

TLDR: I am looking to replace the two (unix and windows) shell scripts
with a single program with unit tests. Motivations and more discussion
in the commit message. If there are any concerns with it, it is a great
time to raise them.

Fabian, I did my best to test this "on windows" without actually having
a windows host. Please let me know if it works for you. Hopefully I
don't break the windows wrapper ever again!

Motiejus Jakštys (1):
  zig launcher: replace shell wrappers with a binary

 .build.yml                 |   9 +-
 ci/launcher                |  13 +
 ci/test                    |  25 +-
 toolchain/defs.bzl         | 128 +++------
 toolchain/launcher.zig     | 531 +++++++++++++++++++++++++++++++++++++
 toolchain/private/defs.bzl |   2 +-
 6 files changed, 613 insertions(+), 95 deletions(-)
 create mode 100755 ci/launcher
 create mode 100644 toolchain/launcher.zig

-- 
2.34.5

---

From: ~motiejus <[email-masked]>
Date: Tue, 20 Dec 2022 17:00:50 +0200

From: Motiejus Jakštys <[email-masked]>

Until now we needed to maintain two versions of the zig launcher: one
for Windows and one for everything else. This was problematic for two
reasons:
1. I do not know powershell and thus keep breaking the Windows wrapper
   all the time (see git history of Fabian fixing stuff that I broke).
2. This makes bazel-zig-cc dependent on the system shell, making it not
   really hermetic. So the recently added
   `--experimental_use_hermetic_linux_sandbox` does not work with
   bazel-zig-cc, unless we bind-mount a bunch of stuff: `/usr`, `/bin`,
   `/lib`, `/usr/lib:/lib`, `/usr/lib64:/lib64` and `/proc`.

Switching to a Zig-based wrapper solves both issues, and we can do this:

    bazel build "$@" \
        --experimental_use_hermetic_linux_sandbox \
        --sandbox_add_mount_pair=/proc \
        <...>

Zig itself still depends on `/proc` for `/proc/self/exe`, so we need to
keep that. I will look into reducing even that dependency separately.

Not all is nice and shiny though: this commit replaces ~80 LOC worth of
shell scripts wrappers with a singe ~250 LOC zig program, which is
arguably harder to understand. However, it is easier to change, at least
for me, because it's a single file with unit tests! Most importantly,
the gnarly code (which resolves paths and sets environment variables) is
cross-platform.
---
 .build.yml                 |   9 +-
 ci/launcher                |  13 +
 ci/test                    |  25 +-
 toolchain/defs.bzl         | 128 +++------
 toolchain/launcher.zig     | 531 +++++++++++++++++++++++++++++++++++++
 toolchain/private/defs.bzl |   2 +-
 6 files changed, 613 insertions(+), 95 deletions(-)
 create mode 100755 ci/launcher
 create mode 100644 toolchain/launcher.zig

diff --git a/.build.yml b/.build.yml
index 664b847..e12a558 100644
--- a/.build.yml
+++ b/.build.yml
@@ -31,6 +31,13 @@ tasks:
   - list_toolchains_platforms: |
       cd bazel-zig-cc; . .envrc
       ./ci/list_toolchains_platforms
+  - test_launcher: |
+      cd bazel-zig-cc; . .envrc
+      ./ci/launcher --color=yes --curses=yes
   - test: |
       cd bazel-zig-cc; . .envrc
-      ./ci/test --color=yes --curses=yes
+      export BAZEL_ZIG_CC_CACHE_PREFIX=/tmp/bazel-zig-cc-2
+      ./ci/test \
+        --color=yes --curses=yes \
+        --repo_env BAZEL_ZIG_CC_CACHE_PREFIX=$BAZEL_ZIG_CC_CACHE_PREFIX \
+        --sandbox_writable_path "$BAZEL_ZIG_CC_CACHE_PREFIX"
diff --git a/ci/launcher b/ci/launcher
new file mode 100755
index 0000000..2c57c8f
--- /dev/null
+++ b/ci/launcher
@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+set -xeuo pipefail
+
+# until bazel-zig-cc gets a zig toolchain, run launcher's unit tests here.
+bazel run "$@" @zig_sdk//:zig -- test toolchain/launcher.zig
+
+# ReleaseSafe because of https://github.com/ziglang/zig/issues/14036
+bazel run "$@" @zig_sdk//:zig -- test \
+    -OReleaseSafe \
+    -target x86_64-windows-gnu \
+    --test-cmd wine64-stable \
+    --test-cmd-bin \
+    toolchain/launcher.zig
diff --git a/ci/test b/ci/test
index e3f4cf5..4cbc3ca 100755
--- a/ci/test
+++ b/ci/test
@@ -1,16 +1,29 @@
 #!/usr/bin/env bash
-set -euo pipefail
+set -xeuo pipefail
+
+BAZEL_ZIG_CC_CACHE_PREFIX=${BAZEL_ZIG_CC_CACHE_PREFIX:-/tmp/bazel-zig-cc}
+mkdir -p "${BAZEL_ZIG_CC_CACHE_PREFIX}"
 
 bazel test "$@" ...
 
-# /tmp/bazel-zig-cc should be empty for the test below to be valid.
-# This test ensures that github.com/ziglang/zig/issues/13050 does not
-# regress
-find /tmp/bazel-zig-cc -name mutex_destructor.o -execdir file '{}' \; | \
+bazel clean; bazel shutdown
+
+# smoke-test a very hermetic setup with a single target. Re-building all of
+# them takes a long time, so using only one. If we ever decide to build all
+# targets, we will need to exclude Go, since go dynamically links to glibc on
+# linux.
+bazel build "$@" \
+    --experimental_use_hermetic_linux_sandbox \
+    --sandbox_add_mount_pair=/proc \
+    //test/c:which_libc_linux_amd64_gnu.2.19
+
+# $BAZEL_ZIG_CC_CACHE_PREFIX should be empty for the test below to be valid.
+# Ensure that github.com/ziglang/zig/issues/13050 does not regress
+find "$BAZEL_ZIG_CC_CACHE_PREFIX" -name mutex_destructor.o -execdir file '{}' \; | \
     sort | uniq -c | sort -rn > /tmp/got_cache
 
 diff -u ci/testdata/want_cache /tmp/got_cache || {
     >&2 echo "ERROR: unexpected artifacts."
-    >&2 echo "Was /tmp/bazel-zig-cc empty before the test?"
+    >&2 echo "Was $BAZEL_ZIG_CC_CACHE_PREFIX empty before the test?"
     exit 1
 }
diff --git a/toolchain/defs.bzl b/toolchain/defs.bzl
index dfd1ef8..483d9fa 100644
--- a/toolchain/defs.bzl
+++ b/toolchain/defs.bzl
@@ -1,3 +1,4 @@
+load("@bazel_skylib//lib:paths.bzl", "paths")
 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 load("@bazel_tools//tools/build_defs/repo:utils.bzl", "read_user_netrc", "use_netrc")
 load("@bazel-zig-cc//toolchain/private:defs.bzl", "target_structs", "zig_tool_path")
@@ -79,83 +80,8 @@ def toolchains(
 _ZIG_TOOLS = [
     "c++",
     "ar",
-    "build-exe",
-    "build-lib",
-    "build-obj",
 ]
 
-_ZIG_TOOL_WRAPPER_WINDOWS_CACHE = """@echo off
-if exist "external\\zig_sdk\\lib\\*" goto :have_external_zig_sdk_lib
-set ZIG_LIB_DIR=%~dp0\\..\\..\\lib
-set ZIG_EXE=%~dp0\\..\\..\\zig.exe
-goto :set_zig_lib_dir
-:have_external_zig_sdk_lib
-set ZIG_LIB_DIR=external\\zig_sdk\\lib
-set ZIG_EXE=external\\zig_sdk\\zig.exe
-:set_zig_lib_dir
-set ZIG_LOCAL_CACHE_DIR={cache_prefix}\\bazel-zig-cc
-set ZIG_GLOBAL_CACHE_DIR=%ZIG_LOCAL_CACHE_DIR%
-"%ZIG_EXE%" "{zig_tool}" {maybe_target} %*
-"""
-
-_ZIG_TOOL_WRAPPER_CACHE = """#!/bin/sh
-set -e
-if [ -d external/zig_sdk/lib ]; then
-    ZIG_LIB_DIR=external/zig_sdk/lib
-    ZIG_EXE=external/zig_sdk/zig
-else
-    ZIG_LIB_DIR="$(dirname "$0")/../../lib"
-    ZIG_EXE="$(dirname "$0")/../../zig"
-fi
-export ZIG_LIB_DIR
-export ZIG_LOCAL_CACHE_DIR="{cache_prefix}/bazel-zig-cc"
-export ZIG_GLOBAL_CACHE_DIR="{cache_prefix}/bazel-zig-cc"
-{maybe_gohack}
-exec "$ZIG_EXE" "{zig_tool}" {maybe_target} "$@"
-"""
-
-# The hackery below will be deleted after Go 1.20 is released (in particular,
-# if/after https://go-review.googlesource.com/c/go/+/436884 )
-# Arg-messing snippet from
-# https://web.archive.org/web/20100129154217/http://www.seanius.net/blog/2009/03/saving-and-restoring-positional-params
-_ZIG_TOOL_GOHACK = """
-quote(){ echo "$1" | sed -e "s,','\\\\'',g"; }
-for arg in "$@"; do saved="${saved:+$saved }'$(quote "$arg")'"; done
-while [ "$#" -gt 6 ]; do shift; done
-if [ "$*" = "-Wl,--no-gc-sections -x c - -o /dev/null" ]; then
-  # This command probes if `--no-gc-sections` is accepted by the linker.
-  # Since it is executed in /tmp, the ZIG_LIB_DIR is absolute,
-  # glibc stubs and libc++ cannot be shared with other invocations (which use
-  # a relative ZIG_LIB_DIR).
-  exit 0;
-fi
-eval set -- "$saved"
-"""
-
-def _zig_tool_wrapper(zig_tool, is_windows, cache_prefix, zigtarget):
-    if zig_tool in ["c++", "build-exe", "build-lib", "build-obj"]:
-        maybe_target = "-target {}".format(zigtarget)
-    else:
-        maybe_target = ""
-
-    if not cache_prefix:
-        if is_windows:
-            cache_prefix = "C:\\Temp\\bazel-zig-cc"
-        else:
-            cache_prefix = "/tmp/bazel-zig-cc"
-
-    kwargs = dict(
-        zig_tool = zig_tool,
-        cache_prefix = cache_prefix,
-        maybe_gohack = _ZIG_TOOL_GOHACK if (zig_tool == "c++" and not is_windows) else "",
-        maybe_target = maybe_target,
-    )
-
-    if is_windows:
-        return _ZIG_TOOL_WRAPPER_WINDOWS_CACHE.format(**kwargs)
-    else:
-        return _ZIG_TOOL_WRAPPER_CACHE.format(**kwargs)
-
 def _quote(s):
     return "'" + s.replace("'", "'\\''") + "'"
 
@@ -217,22 +143,50 @@ def _zig_repository_impl(repository_ctx):
         sha256 = zig_sha256,
     )
 
+    cache_prefix = repository_ctx.os.environ.get("BAZEL_ZIG_CC_CACHE_PREFIX", "")
+    if cache_prefix == "":
+        if os == "windows":
+            cache_prefix = "C:\\Temp\\bazel-zig-cc"
+        else:
+            cache_prefix = "/tmp/bazel-zig-cc"
+
+    repository_ctx.template(
+        "tools/launcher.zig",
+        Label("//toolchain:launcher.zig"),
+        executable = False,
+        substitutions = {
+            "{BAZEL_ZIG_CC_CACHE_PREFIX}": cache_prefix,
+        },
+    )
+
+    ret = repository_ctx.execute(
+        [
+            paths.join("..", "zig"),
+            "build-exe",
+            "-OReleaseSafe",
+            "launcher.zig",
+        ] + (["-static"] if os == "linux" else []),
+        working_directory = "tools",
+        environment = {
+            "ZIG_LOCAL_CACHE_DIR": cache_prefix,
+            "ZIG_GLOBAL_CACHE_DIR": cache_prefix,
+        },
+    )
+    if ret.return_code != 0:
+        fail("compilation failed:\nreturn_code={}\nstderr={}\nstdout={}".format(
+            ret.return_code,
+            ret.stdout,
+            ret.stderr,
+        ))
+
+    exe = ".exe" if os == "windows" else ""
     for target_config in target_structs():
         for zig_tool in _ZIG_TOOLS + target_config.tool_paths.values():
-            zig_tool_wrapper = _zig_tool_wrapper(
-                zig_tool,
-                os == "windows",
-                repository_ctx.os.environ.get("BAZEL_ZIG_CC_CACHE_PREFIX", ""),
+            tool_path = zig_tool_path(os).format(
+                zig_tool = zig_tool,
                 zigtarget = target_config.zigtarget,
             )
-
-            repository_ctx.file(
-                zig_tool_path(os).format(
-                    zig_tool = zig_tool,
-                    zigtarget = target_config.zigtarget,
-                ),
-                zig_tool_wrapper,
-            )
+            repository_ctx.symlink("tools/launcher{}".format(exe), tool_path)
 
     repository_ctx.file(
         "glibc-hacks/fcntl.map",
diff --git a/toolchain/launcher.zig b/toolchain/launcher.zig
new file mode 100644
index 0000000..0cf86e8
--- /dev/null
+++ b/toolchain/launcher.zig
@@ -0,0 +1,531 @@
+// A wrapper for `zig` subcommands.
+//
+// In simple cases it is usually enough to:
+//
+//      zig c++ -target <triple> <...>
+//
+// However, there are some caveats:
+//
+// * Sometimes toolchains (looking at you, Go, see an example in
+// https://github.com/golang/go/pull/55966) skip CFLAGS to the underlying
+// compiler. Doing that may carry a huge cost, because zig may need to spend
+// ~30s compiling libc++ for an innocent feature test. Having an executable per
+// target platform (like GCC does things, e.g. aarch64-linux-gnu-<tool>) is
+// what most toolchains are designed to work with. So we need a wrapper per
+// zig sub-command per target. As of writing, the layout is:
+//   tools/
+//   ├── x86_64-linux-gnu.2.34
+//   │   ├── ar
+//   │   ├── c++
+//   │   └── ld.lld
+//   ├── x86_64-linux-musl
+//   │   ├── ar
+//   │   ├── c++
+//   │   └── ld.lld
+//   ├── x86_64-macos-none
+//   │   ├── ar
+//   │   ├── c++
+//   │   └── ld64.lld
+//   ...
+// * ZIG_LIB_DIR controls the output of `zig c++ -MF -MD <...>`. Bazel uses
+// command to understand which input files were used to the compilation. If any
+// of the files are not in `external/<...>/`, Bazel will understand and
+// complain that the compiler is using undeclared directories on the host file
+// system. We do not declare prerequisites using absolute paths, because that
+// busts Bazel's remote cache.
+// * BAZEL_ZIG_CC_CACHE_PREFIX is configurable per toolchain instance, and
+// ZIG_GLOBAL_CACHE_DIR and ZIG_LOCAL_CACHE_DIR must be set to its value for
+// all `zig` invocations.
+//
+// Originally this was a Bash script, then a POSIX shell script, then two
+// scripts (one with pre-defined BAZEL_ZIG_CC_CACHE_PREFIX, one without). Then
+// Windows came along with two PowerShell scripts (ports of the POSIX shell
+// scripts), which I kept breaking. Then Bazel 6 came with
+// `--experimental_use_hermetic_linux_sandbox`, which hermetizes the sandbox to
+// the extreme: the sandbox has nothing that is not declared. /bin/sh and its
+// dependencies (/lib/x86_64-linux-gnu/libc.so.6 on my system) are obviously
+// not declared. So one can either declare those dependencies, bundle a shell
+// to execute the wrapper, or port the shell logic to a cross-platform program
+// that compiles to a static binary. By a chance we happen to already ship a
+// toolchain of a language that could compile such program. And behold, the
+// program is below.
+
+const builtin = @import("builtin");
+const std = @import("std");
+const fs = std.fs;
+const mem = std.mem;
+const process = std.process;
+const ChildProcess = std.ChildProcess;
+const ArrayList = std.ArrayList;
+const ArrayListUnmanaged = std.ArrayListUnmanaged;
+const Allocator = std.mem.Allocator;
+const sep = fs.path.sep_str;
+
+const EXE = switch (builtin.target.os.tag) {
+    .windows => ".exe",
+    else => "",
+};
+
+const CACHE_DIR = "{BAZEL_ZIG_CC_CACHE_PREFIX}";
+
+const usage_cpp =
+    \\Usage: <...>/tools/<target-triple>/{[zig_tool]s}{[exe]s} <args>...
+    \\
+    \\Wraps the "zig" multi-call binary. It determines the target platform from
+    \\the directory where it was called. Then sets ZIG_LIB_DIR,
+    \\ZIG_GLOBAL_CACHE_DIR, ZIG_LOCAL_CACHE_DIR and then calls:
+    \\
+    \\  zig c++ -target <target-triple> <args>...
+;
+
+const usage_other =
+    \\Usage: <...>/tools/<target-triple>/{[zig_tool]s}{[exe]s} <args>...
+    \\
+    \\Wraps the "zig" multi-call binary. It sets ZIG_LIB_DIR,
+    \\ZIG_GLOBAL_CACHE_DIR, ZIG_LOCAL_CACHE_DIR, and then calls:
+    \\
+    \\  zig {[zig_tool]s} <args>...
+;
+
+const Action = enum {
+    early_ok,
+    early_err,
+    exec,
+};
+
+const ExecParams = struct {
+    args: ArrayListUnmanaged([]const u8),
+    env: process.EnvMap,
+};
+
+const ParseResults = union(Action) {
+    early_ok,
+    early_err: []const u8,
+    exec: ExecParams,
+};
+
+pub fn main() u8 {
+    var allocator = if (builtin.link_libc)
+        std.heap.c_allocator
+    else blk: {
+        var gpa = std.heap.GeneralPurposeAllocator(.{}){};
+        break :blk gpa.allocator();
+    };
+    var arena_allocator = std.heap.ArenaAllocator.init(allocator);
+    defer arena_allocator.deinit();
+    const arena = arena_allocator.allocator();
+
+    var argv_it = process.argsWithAllocator(arena) catch |err| {
+        std.debug.print("error parsing args: {s}\n", .{@errorName(err)});
+        return 1;
+    };
+
+    const action = parseArgs(arena, fs.cwd(), &argv_it) catch |err| {
+        std.debug.print("error: {s}\n", .{@errorName(err)});
+        return 1;
+    };
+
+    switch (action) {
+        .early_ok => return 0,
+        .early_err => |msg| {
+            std.io.getStdErr().writeAll(msg) catch {};
+            return 1;
+        },
+        .exec => |params| {
+            if (builtin.os.tag == .windows) {
+                return spawnWindows(arena, params);
+            } else {
+                return execUnix(arena, params);
+            }
+        },
+    }
+}
+
+fn spawnWindows(arena: Allocator, params: ExecParams) u8 {
+    var proc = ChildProcess.init(params.args.items, arena);
+    proc.env_map = &params.env;
+    const ret = proc.spawnAndWait() catch |err| {
+        std.debug.print(
+            "error spawning {s}: {s}\n",
+            .{ params.args.items[0], @errorName(err) },
+        );
+        return 1;
+    };
+
+    switch (ret) {
+        .Exited => |code| return code,
+        else => |other| {
+            std.debug.print("abnormal exit: {any}\n", .{other});
+            return 1;
+        },
+    }
+}
+
+fn execUnix(arena: Allocator, params: ExecParams) u8 {
+    const err = process.execve(arena, params.args.items, &params.env);
+    std.debug.print(
+        "error execing {s}: {s}\n",
+        .{ params.args.items[0], @errorName(err) },
+    );
+    return 1;
+}
+
+// argv_it is an object that has such method:
+//     fn next(self: *Self) ?[]const u8
+// in non-testing code it is *process.ArgIterator.
+// Leaks memory: the name of the first argument is arena not by chance.
+fn parseArgs(
+    arena: Allocator,
+    cwd: fs.Dir,
+    argv_it: anytype,
+) error{OutOfMemory}!ParseResults {
+    const arg0 = argv_it.next() orelse
+        return fatal(arena, "error: argv[0] cannot be null", .{});
+
+    const zig_tool = blk: {
+        const b = fs.path.basename(arg0);
+        if (builtin.target.os.tag == .windows and
+            std.ascii.eqlIgnoreCase(".exe", b[b.len - 4 ..]))
+            break :blk b[0 .. b.len - 4];
+
+        break :blk b;
+    };
+    const maybe_target = getTarget(arg0) catch |err| switch (err) {
+        error.BadParent => {
+            const fmt_args = .{ .zig_tool = zig_tool, .exe = EXE };
+            if (mem.eql(u8, zig_tool, "c++")) {
+                return fatal(arena, usage_cpp, fmt_args);
+            } else return fatal(arena, usage_other, fmt_args);
+        },
+        else => |e| return e,
+    };
+
+    const root = blk: {
+        var dir = cwd.openDir(
+            "external" ++ sep ++ "zig_sdk" ++ sep ++ "lib",
+            .{ .access_sub_paths = false, .no_follow = true },
+        );
+
+        if (dir) |*dir_exists| {
+            dir_exists.close();
+            break :blk "external" ++ sep ++ "zig_sdk";
+        } else |_| {}
+
+        // directory does not exist or there was an error opening it
+        const here = fs.path.dirname(arg0) orelse ".";
+        break :blk try fs.path.join(arena, &[_][]const u8{ here, "..", ".." });
+    };
+
+    const zig_lib_dir = try fs.path.join(arena, &[_][]const u8{ root, "lib" });
+    const zig_exe = try fs.path.join(
+        arena,
+        &[_][]const u8{ root, "zig" ++ EXE },
+    );
+
+    var env = process.getEnvMap(arena) catch |err| {
+        return fatal(
+            arena,
+            "error getting process environment: {s}",
+            .{@errorName(err)},
+        );
+    };
+    try env.put("ZIG_LIB_DIR", zig_lib_dir);
+    try env.put("ZIG_LOCAL_CACHE_DIR", CACHE_DIR);
+    try env.put("ZIG_GLOBAL_CACHE_DIR", CACHE_DIR);
+
+    // args is the path to the zig binary and args to it.
+    var args = ArrayListUnmanaged([]const u8){};
+    try args.appendSlice(arena, &[_][]const u8{ zig_exe, zig_tool });
+    if (maybe_target) |target|
+        try args.appendSlice(arena, &[_][]const u8{ "-target", target });
+
+    while (argv_it.next()) |arg|
+        try args.append(arena, arg);
+
+    if (mem.eql(u8, zig_tool, "c++") and shouldReturnEarly(args.items))
+        return .early_ok;
+
+    return ParseResults{
+        .exec = .{
+            .args = args,
+            .env = env,
+        },
+    };
+}
+
+fn fatal(
+    arena: Allocator,
+    comptime fmt: []const u8,
+    args: anytype,
+) error{OutOfMemory}!ParseResults {
+    const msg = try std.fmt.allocPrint(arena, fmt ++ "\n", args);
+    return ParseResults{ .early_err = msg };
+}
+
+// Golang probing for a particular linker flag causes many unneeded stubs to be
+// built, e.g. glibc, musl, libc++. The hackery can probably be deleted after
+// Go 1.20 is released. In particular,
+// https://go-review.googlesource.com/c/go/+/436884
+fn shouldReturnEarly(args: []const []const u8) bool {
+    const prelude = &[_][]const u8{
+        "-Wl,--no-gc-sections",
+        "-x",
+        "c",
+        "-",
+        "-o",
+        "/dev/null",
+    };
+    if (args.len < prelude.len)
+        return false;
+    for (prelude) |arg, i|
+        if (!mem.eql(u8, arg, args[args.len - prelude.len + i]))
+            return false;
+    return true;
+}
+
+fn getTarget(self_exe: []const u8) error{BadParent}!?[]const u8 {
+    const here = fs.path.dirname(self_exe) orelse return error.BadParent;
+    const triple = fs.path.basename(here);
+
+    // Validating the triple now will help users catch errors even if they
+    // don't yet need the target. yes yes the validation will miss things
+    // strings `is.it.x86_64?-stallinux,macos-`; we are trying to aid users
+    // that run things from the wrong directory, not trying to punish the ones
+    // having fun.
+    {
+        var it = mem.split(u8, triple, "-");
+        if (it.next()) |arch| {
+            if (mem.indexOf(u8, "aarch64,x86_64", arch) == null)
+                return error.BadParent;
+        } else return error.BadParent;
+
+        if (it.next()) |got_os| {
+            if (mem.indexOf(u8, "linux,macos,windows", got_os) == null)
+                return error.BadParent;
+        } else return error.BadParent;
+
+        // ABI triple is too much of a moving target
+        if (it.next() == null) return error.BadParent;
+
+        // but the target needs to have 3 dashes.
+        if (it.next() != null) return error.BadParent;
+    }
+
+    if (mem.eql(u8, "c++" ++ EXE, fs.path.basename(self_exe))) {
+        return triple;
+    } else return null;
+}
+
+const testing = std.testing;
+
+fn split(comptime str: []const u8) []const []const u8 {
+    var arr: [str.len][]const u8 = undefined;
+    var i: usize = 0;
+    var it = mem.split(u8, str, " ");
+    while (it.next()) |arg| : (i += 1)
+        arr[i] = arg;
+    return arr[0..i];
+}
+
+test "launcher:shouldReturnEarly" {
+    const pos = .{
+        "-Wl,--no-gc-sections -x c - -o /dev/null",
+        "foo.c -o main -Wl,--no-gc-sections -x c - -o /dev/null",
+    };
+    const neg = .{
+        "cc -Wl,--no-gc-sections -x c - -o /dev/null x",
+        "-Wl,--no-gc-sections -x c - -o",
+        "incorrect-value -x c - -o /dev/null",
+    };
+
+    inline for (pos) |tt|
+        try testing.expect(shouldReturnEarly(split(tt)));
+
+    inline for (neg) |tt|
+        try testing.expect(!shouldReturnEarly(split(tt)));
+}
+
+pub const TestArgIterator = struct {
+    index: usize = 0,
+    argv: []const [:0]const u8,
+
+    pub fn next(self: *TestArgIterator) ?[:0]const u8 {
+        if (self.index == self.argv.len) return null;
+
+        defer self.index += 1;
+        return self.argv[self.index];
+    }
+
+    pub fn skip(self: *TestArgIterator) bool {
+        if (self.index == self.len) return false;
+
+        self.index += 1;
+        return true;
+    }
+};
+
+fn compareExec(
+    res: ParseResults,
+    want_args: []const [:0]const u8,
+    want_env_zig_lib_dir: []const u8,
+) !void {
+    try testing.expectEqual(want_args.len, res.exec.args.items.len);
+    for (want_args) |want_arg, i| {
+        try testing.expectEqualStrings(
+            want_arg,
+            res.exec.args.items[i],
+        );
+    }
+    try testing.expectEqualStrings(
+        want_env_zig_lib_dir,
+        res.exec.env.get("ZIG_LIB_DIR").?,
+    );
+}
+
+test "launcher:parseArgs" {
+    // not using testing.allocator, because parseArgs is designed to be used
+    // with an arena.
+    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
+    var allocator = gpa.allocator();
+
+    const tests = [_]struct {
+        args: []const [:0]const u8,
+        precreate_dir: ?[]const u8 = null,
+        want_result: union(Action) {
+            early_ok,
+            early_err: []const u8,
+            exec: struct {
+                args: []const [:0]const u8,
+                env_zig_lib_dir: []const u8,
+            },
+        },
+    }{
+        .{
+            .args = &[_][:0]const u8{"ar" ++ EXE},
+            .want_result = .{
+                .early_err = std.fmt.comptimePrint(usage_other ++ "\n", .{
+                    .zig_tool = "ar",
+                    .exe = EXE,
+                }),
+            },
+        },
+        .{
+            .args = &[_][:0]const u8{"c++" ++ EXE},
+            .want_result = .{
+                .early_err = std.fmt.comptimePrint(usage_cpp ++ "\n", .{
+                    .zig_tool = "c++",
+                    .exe = EXE,
+                }),
+            },
+        },
+        .{
+            .args = &[_][:0]const u8{
+                "external" ++ sep ++ "zig_sdk" ++ "tools" ++ sep ++
+                    "x86_64-linux-musl" ++ sep ++ "c++" ++ EXE,
+                "-Wl,--no-gc-sections",
+                "-x",
+                "c",
+                "-",
+                "-o",
+                "/dev/null",
+            },
+            .want_result = .early_ok,
+        },
+        .{
+            .args = &[_][:0]const u8{
+                "tools" ++ sep ++ "x86_64-linux-musl" ++ sep ++ "c++" ++ EXE,
+                "main.c",
+                "-o",
+                "/dev/null",
+            },
+            .want_result = .{
+                .exec = .{
+                    .args = &[_][:0]const u8{
+                        "tools" ++ sep ++ "x86_64-linux-musl" ++ sep ++
+                            ".." ++ sep ++ ".." ++ sep ++ "zig" ++ EXE,
+                        "c++",
+                        "-target",
+                        "x86_64-linux-musl",
+                        "main.c",
+                        "-o",
+                        "/dev/null",
+                    },
+                    .env_zig_lib_dir = "tools" ++ sep ++ "x86_64-linux-musl" ++
+                        sep ++ ".." ++ sep ++ ".." ++ sep ++ "lib",
+                },
+            },
+        },
+        .{
+            .args = &[_][:0]const u8{
+                "tools" ++ sep ++ "x86_64-linux-musl" ++ sep ++ "ar" ++ EXE,
+                "-rcs",
+                "all.a",
+                "main.o",
+                "foo.o",
+            },
+            .want_result = .{
+                .exec = .{
+                    .args = &[_][:0]const u8{
+                        "tools" ++ sep ++ "x86_64-linux-musl" ++ sep ++ ".." ++
+                            sep ++ ".." ++ sep ++ "zig" ++ EXE,
+                        "ar",
+                        "-rcs",
+                        "all.a",
+                        "main.o",
+                        "foo.o",
+                    },
+                    .env_zig_lib_dir = "tools" ++ sep ++ "x86_64-linux-musl" ++
+                        sep ++ ".." ++ sep ++ ".." ++ sep ++ "lib",
+                },
+            },
+        },
+        .{
+            .args = &[_][:0]const u8{
+                "external_zig_sdk" ++ sep ++ "tools" ++ sep ++
+                    "x86_64-linux-gnu.2.28" ++ sep ++ "c++" ++ EXE,
+                "main.c",
+                "-o",
+                "/dev/null",
+            },
+            .precreate_dir = "external" ++ sep ++ "zig_sdk" ++ sep ++ "lib",
+            .want_result = .{
+                .exec = .{
+                    .args = &[_][:0]const u8{
+                        "external" ++ sep ++ "zig_sdk" ++ sep ++ "zig" ++ EXE,
+                        "c++",
+                        "-target",
+                        "x86_64-linux-gnu.2.28",
+                        "main.c",
+                        "-o",
+                        "/dev/null",
+                    },
+                    .env_zig_lib_dir = "external" ++ sep ++ "zig_sdk" ++
+                        sep ++ "lib",
+                },
+            },
+        },
+    };
+
+    for (tests) |tt| {
+        var tmp = testing.tmpDir(.{});
+        defer tmp.cleanup();
+
+        if (tt.precreate_dir) |dir|
+            try tmp.dir.makePath(dir);
+
+        var res = try parseArgs(allocator, tmp.dir, &TestArgIterator{
+            .argv = tt.args,
+        });
+
+        switch (tt.want_result) {
+            .early_ok => try testing.expectEqual(res, .early_ok),
+            .early_err => |want_msg| try testing.expectEqualStrings(
+                want_msg,
+                res.early_err,
+            ),
+            .exec => |want| {
+                try compareExec(res, want.args, want.env_zig_lib_dir);
+            },
+        }
+    }
+}
diff --git a/toolchain/private/defs.bzl b/toolchain/private/defs.bzl
index 9409f5e..635dde9 100644
--- a/toolchain/private/defs.bzl
+++ b/toolchain/private/defs.bzl
@@ -32,7 +32,7 @@ LIBCS = ["musl"] + ["gnu.{}".format(glibc) for glibc in _GLIBCS]
 
 def zig_tool_path(os):
     if os == "windows":
-        return _ZIG_TOOL_PATH + ".bat"
+        return _ZIG_TOOL_PATH + ".exe"
     else:
         return _ZIG_TOOL_PATH
 
-- 
2.34.5

---

From: "builds.sr.ht" <[email-masked]>
Date: Fri, 23 Dec 2022 11:24:15 +0000

bazel-zig-cc/patches/.build.yml: SUCCESS in 19m41s

[zig launcher: replace shell wrappers with a binary][0] from [~motiejus][1]

[0]: https://lists.sr.ht/~motiejus/bazel-zig-cc/patches/37721
[1]: mailto:[email-masked]

✓ #910438 SUCCESS bazel-zig-cc/patches/.build.yml https://builds.sr.ht/~motiejus/job/910438

---

From: Fabian Hahn <[email-masked]>
Date: Mon, 2 Jan 2023 21:53:48 +0000

Hi Motiejus,

Apologies for the delay in responding, I was traveling over the 
Christmas and new year break and just got back home.

I've tested your patch on Windows and there are currently two issues 
with it, one is trivial to fix and the other one I'm not sure about:

First, cache_prefix in toolchain/defs.bzl needs to have its backslashes 
double escaped: once for bazel, and once for zig itself, otherwise the 
zig compiler will complain about invalid escape characters. So it should 
be "C:\\\\Temp\\\\bazel-zig-cc"

Second, zig further fails to compile launcher.zig with the following error:

launcher.zig:70:73: error: expected ';' after declaration

     \\Usage: <...>/tools/<target-triple>/{[zig_tool]s}{[exe]s} <args>...

                                                                         ^~~~

launcher.zig:71:3: note: invalid byte: ' '

     \\

   ^

The reason for this is that Git on Windows inserts carriage return 
characters before newlines when checking out local files so they show up 
fine in Windows text editors where \r\n is the default for line breaks. 
This then breaks the zig parser's support for multiline constants as 
detailed in https://github.com/ziglang/zig/issues/9257 which is marked 
as closed even though it doesn't look resolved to me at all. Converting 
newlines is the default behavior for Git unless you explicitly disable 
it by setting core.autocrlf in your Git config. While that would be a 
workaround, I would find this an uncommon thing to do and an odd thing 
to require of Windows users, which is something also pointed out in the 
above issue.

I did confirm that either of the following two hacks to launcher.zig do 
make your patch work on Windows:

1. Replace usage_cpp and usage_other by regular strings without 
multiline constant syntax.

2. Setting core.autocrlf to false in Git and recloning the repository.

3. Running dos2unix on launcher.zig before compiling it.

None of these are ideal, so I'm not quite sure how to go about solving 
this. What do you think?

Best,

Fabian

---

From: Motiejus Jakštys <[email-masked]>
Date: Tue, 3 Jan 2023 00:34:13 +0200

> Christmas and new year break and just got back home.

Happy new year! :)

> First, cache_prefix in toolchain/defs.bzl needs to <...>

Fixed, thanks.

<...>

> multiline constant syntax.

This.

> None of these are ideal, so I'm not quite sure how to go about solving

I am OK with using ugly strings with ++ until
https://github.com/ziglang/zig/issues/9257 is fixed one way or
another. Can you do a final test on
https://git.sr.ht/~motiejus/bazel-zig-cc/commit/motiejus_launcher_zig
before I merge it?

Thanks,
Motiejus

---

From: [email-masked]
Date: Tue, 03 Jan 2023 08:43:02 +0000

> https://git.sr.ht/~motiejus/bazel-zig-cc/commit/motiejus_launcher_zig

Sounds good, thanks for making the change. I've tested your latest 
branch and Windows host support is working fine :)

Best,
Fabian