Subject: bazel test under qemu-aarch64: help needed

From: Motiejus Jakštys <[email-masked]>
Date: Fri, 4 Feb 2022 13:55:57 +0200

Hi folks,

I spent a good day trying to figure this out, I need help enabling tests
for aarch64.

Goal: run `bazel test //test/...` for 2 architectures: amd64 and arm64.
Arm64 would be executed under qemu-aarch64.

This works on my laptop (amd64/buster):

    git clone https://git.sr.ht/~motiejus/bazel-zig-cc -b qemu
    cd bazel-zig-cc; direnv allow .
    bazel test --run_under=qemu-aarch64 \
        --platforms @zig_sdk//:linux_arm64_platform \
        --extra_toolchains @zig_sdk//:linux_arm64_musl_toolchain \
        //test/go:go_test

Fails[1] on builds.sr.ht, which is also buster/amd64.

    2022/02/04 09:21:03 fork/exec /home/<...>/go_test: exec format error

However, when executed directly via qemu-aarch64 on the given CI
machine, it works:

build@build:~/bazel-zig-cc$ qemu-aarch64 /<...>/go_test
PASS
build@build:~/bazel-zig-cc$ 

strace, investigating the bazel's sandboxing (I suspected this may be
caused by linux-sandbox-pid1.cc) didn't reveal anything. Any pointers?

Thanks,
Motiejus

[1]: https://builds.sr.ht/~motiejus/job/687819

---

From: Motiejus Jakštys <[email-masked]>
Date: Sat, 5 Feb 2022 06:55:29 +0200

> I spent a good day trying to figure this out, I need help enabling tests

Hi all,

Piotr Sikora replied out-of-band[1]. Two things needed to be changed:

1. Install binfmt-support. That registers qemu handlers to binfmt.
2. use qemu-aarch64-static instead of the shared version.

... and now we run all unit tests on both amd64 and arm64, and the test
runner[2] is less ugly.

Motiejus

[1]:
https://github.com/PiotrSikora/bazel-zig-cc/commit/10101bf7f11ff404e66f582bbb64a3a875f8e45f#commitcomment-66048081
[2]: https://git.sr.ht/~motiejus/bazel-zig-cc/tree/main/item/ci/test