[INFO] cloning repository https://github.com/Rayato159/rust-basic-ann-in-candle [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/Rayato159/rust-basic-ann-in-candle" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FRayato159%2Frust-basic-ann-in-candle", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FRayato159%2Frust-basic-ann-in-candle'... [INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }` [INFO] [stdout] 56918327dbd925a4600b3198a84cf67b225a50b9 [INFO] testing Rayato159/rust-basic-ann-in-candle against master#160e7623e8cbbf1feab2b6e2a24733a98c7bde9c for pr-145628-retry-1 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FRayato159%2Frust-basic-ann-in-candle" "/workspace/builds/worker-1-tc1/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-1-tc1/source'... [INFO] [stderr] done. [INFO] started tweaking git repo https://github.com/Rayato159/rust-basic-ann-in-candle [INFO] finished tweaking git repo https://github.com/Rayato159/rust-basic-ann-in-candle [INFO] tweaked toml for git repo https://github.com/Rayato159/rust-basic-ann-in-candle written to /workspace/builds/worker-1-tc1/source/Cargo.toml [INFO] validating manifest of git repo https://github.com/Rayato159/rust-basic-ann-in-candle on toolchain 160e7623e8cbbf1feab2b6e2a24733a98c7bde9c [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+160e7623e8cbbf1feab2b6e2a24733a98c7bde9c" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] crate git repo https://github.com/Rayato159/rust-basic-ann-in-candle already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+160e7623e8cbbf1feab2b6e2a24733a98c7bde9c" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] [stderr] Updating git repository `https://github.com/huggingface/candle.git` [INFO] [stderr] Updating git submodule `https://github.com/NVIDIA/cutlass.git` [INFO] [stderr] Updating crates.io index [INFO] [stderr] Downloading crates ... [INFO] [stderr] Downloaded gemm-c32 v0.17.1 [INFO] [stderr] Downloaded zerofrom v0.1.3 [INFO] [stderr] Downloaded memmap2 v0.9.4 [INFO] [stderr] Downloaded bindgen_cuda v0.1.5 [INFO] [stderr] Downloaded derive_arbitrary v1.3.2 [INFO] [stderr] Downloaded thiserror-impl v1.0.59 [INFO] [stderr] Downloaded half v2.4.1 [INFO] [stderr] Downloaded gemm-f16 v0.17.1 [INFO] [stderr] Downloaded cudarc v0.10.0 [INFO] [stderr] Downloaded gemm-c64 v0.17.1 [INFO] [stderr] Downloaded dyn-stack v0.10.0 [INFO] [stderr] Downloaded serde_derive v1.0.199 [INFO] [stderr] Downloaded raw-cpuid v10.7.0 [INFO] [stderr] Downloaded pulp v0.18.10 [INFO] [stderr] Downloaded serde_json v1.0.116 [INFO] [stderr] Downloaded serde v1.0.199 [INFO] [stderr] Downloaded zip v1.1.1 [INFO] [stderr] Downloaded bytemuck v1.15.0 [INFO] [stderr] Downloaded sysctl v0.5.5 [INFO] [stderr] Downloaded gemm-common v0.17.1 [INFO] [stderr] Downloaded num-complex v0.4.5 [INFO] [stderr] Downloaded seq-macro v0.3.5 [INFO] [stderr] Downloaded zerofrom-derive v0.1.3 [INFO] [stderr] Downloaded crc32fast v1.4.0 [INFO] [stderr] Downloaded bytemuck_derive v1.6.0 [INFO] [stderr] Downloaded rand_distr v0.4.3 [INFO] [stderr] Downloaded thiserror v1.0.59 [INFO] [stderr] Downloaded gemm-f32 v0.17.1 [INFO] [stderr] Downloaded gemm v0.17.1 [INFO] [stderr] Downloaded yoke v0.7.3 [INFO] [stderr] Downloaded safetensors v0.4.3 [INFO] [stderr] Downloaded reborrow v0.5.5 [INFO] [stderr] Downloaded gemm-f64 v0.17.1 [INFO] [stderr] Downloaded yoke-derive v0.7.3 [INFO] [stderr] Downloaded arbitrary v1.3.2 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+160e7623e8cbbf1feab2b6e2a24733a98c7bde9c" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }` [INFO] [stdout] bf1d2ea2ddce80cc3efbbc70fee73a936c722c50701b5829aee50e39d192d1e5 [INFO] running `Command { std: "docker" "start" "-a" "bf1d2ea2ddce80cc3efbbc70fee73a936c722c50701b5829aee50e39d192d1e5", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "bf1d2ea2ddce80cc3efbbc70fee73a936c722c50701b5829aee50e39d192d1e5", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "bf1d2ea2ddce80cc3efbbc70fee73a936c722c50701b5829aee50e39d192d1e5", kill_on_drop: false }` [INFO] [stdout] bf1d2ea2ddce80cc3efbbc70fee73a936c722c50701b5829aee50e39d192d1e5 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+160e7623e8cbbf1feab2b6e2a24733a98c7bde9c" "build" "--frozen" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] eb17c93d00d762f85c8b50c255aa967fcb3da85422d5388448cadb73d804a412 [INFO] running `Command { std: "docker" "start" "-a" "eb17c93d00d762f85c8b50c255aa967fcb3da85422d5388448cadb73d804a412", kill_on_drop: false }` [INFO] [stderr] Compiling proc-macro2 v1.0.81 [INFO] [stderr] Compiling libc v0.2.153 [INFO] [stderr] Compiling crossbeam-utils v0.8.19 [INFO] [stderr] Compiling autocfg v1.2.0 [INFO] [stderr] Compiling either v1.11.0 [INFO] [stderr] Compiling paste v1.0.14 [INFO] [stderr] Compiling reborrow v0.5.5 [INFO] [stderr] Compiling once_cell v1.19.0 [INFO] [stderr] Compiling seq-macro v0.3.5 [INFO] [stderr] Compiling serde v1.0.199 [INFO] [stderr] Compiling raw-cpuid v10.7.0 [INFO] [stderr] Compiling glob v0.3.1 [INFO] [stderr] Compiling crc32fast v1.4.0 [INFO] [stderr] Compiling serde_json v1.0.116 [INFO] [stderr] Compiling thiserror v1.0.59 [INFO] [stderr] Compiling ryu v1.0.17 [INFO] [stderr] Compiling stable_deref_trait v1.2.0 [INFO] [stderr] Compiling cudarc v0.10.0 [INFO] [stderr] Compiling num-traits v0.2.18 [INFO] [stderr] Compiling zip v1.1.1 [INFO] [stderr] error: failed to run custom build command for `cudarc v0.10.0` [INFO] [stderr] note: To improve backtraces for build dependencies, set the CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation. [INFO] [stderr] [INFO] [stderr] Caused by: [INFO] [stderr] process didn't exit successfully: `/opt/rustwide/target/debug/build/cudarc-9c5d26f5357d9061/build-script-build` (exit status: 101) [INFO] [stderr] --- stdout [INFO] [stderr] cargo:rerun-if-changed=build.rs [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_ROOT [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_PATH [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_TOOLKIT_ROOT_DIR [INFO] [stderr] [INFO] [stderr] --- stderr [INFO] [stderr] [INFO] [stderr] thread 'main' (529) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/cudarc-0.10.0/build.rs:21:13: [INFO] [stderr] Unable to find `include/cuda.h` under any of: ["/usr", "/usr/local/cuda", "/opt/cuda", "/usr/lib/cuda", "C:/Program Files/NVIDIA GPU Computing Toolkit", "C:/CUDA"]. Set the `CUDA_ROOT` environment variable to `$CUDA_ROOT/include/cuda.h` to override path. [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x627438ae04f2 - std::backtrace_rs::backtrace::libunwind::trace::hf3086ff3c8ddfec7 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stderr] 1: 0x627438ae04f2 - std::backtrace_rs::backtrace::trace_unsynchronized::h6caf11542219ebc3 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stderr] 2: 0x627438ae04f2 - std::sys::backtrace::_print_fmt::hfd9f352edf0ef63b [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/sys/backtrace.rs:66:9 [INFO] [stderr] 3: 0x627438ae04f2 - ::fmt::h746cb25d4c2f20bf [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/sys/backtrace.rs:39:26 [INFO] [stderr] 4: 0x627438aef31f - core::fmt::rt::Argument::fmt::h7e560d1a4260d0bd [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/core/src/fmt/rt.rs:173:76 [INFO] [stderr] 5: 0x627438aef31f - core::fmt::write::h51c104753c8435b2 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/core/src/fmt/mod.rs:1468:25 [INFO] [stderr] 6: 0x627438abe253 - std::io::default_write_fmt::h3139829dd1dc7ae7 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/io/mod.rs:639:11 [INFO] [stderr] 7: 0x627438abe253 - std::io::Write::write_fmt::h485181afa18655f5 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/io/mod.rs:1954:13 [INFO] [stderr] 8: 0x627438ac3542 - std::sys::backtrace::BacktraceLock::print::heee726a1c5b4a2c3 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/sys/backtrace.rs:42:9 [INFO] [stderr] 9: 0x627438ac500c - std::panicking::default_hook::{{closure}}::h938489fbb79d30ee [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:301:27 [INFO] [stderr] 10: 0x627438ac4e66 - std::panicking::default_hook::hf7e155db98319366 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:328:9 [INFO] [stderr] 11: 0x627438ac55e5 - std::panicking::panic_with_hook::h78bb7329978b77c8 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:834:13 [INFO] [stderr] 12: 0x627438ac547a - std::panicking::panic_handler::{{closure}}::h306ab6080b8e304c [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:707:13 [INFO] [stderr] 13: 0x627438ac3679 - std::sys::backtrace::__rust_end_short_backtrace::hab10906eec9c5584 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/sys/backtrace.rs:174:18 [INFO] [stderr] 14: 0x627438ab2afd - __rustc[31cdaa5515be9d95]::rust_begin_unwind [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:698:5 [INFO] [stderr] 15: 0x627438af2390 - core::panicking::panic_fmt::h8cb2938d8e2b0e7b [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/core/src/panicking.rs:75:14 [INFO] [stderr] 16: 0x627438aaeb6a - build_script_build::link_cuda::{{closure}}::h903b49604f20e616 [INFO] [stderr] 17: 0x627438ab11e5 - core::option::Option::unwrap_or_else::h1dd62f6fcef3506f [INFO] [stderr] 18: 0x627438aae6d5 - build_script_build::link_cuda::h9e49aa785b65f929 [INFO] [stderr] 19: 0x627438aae595 - build_script_build::main::h992455000fcafbf2 [INFO] [stderr] 20: 0x627438aaf203 - core::ops::function::FnOnce::call_once::h6a0ee9fff79c6414 [INFO] [stderr] 21: 0x627438aaec46 - std::sys::backtrace::__rust_begin_short_backtrace::h8ce8fa7f7bd5d93e [INFO] [stderr] 22: 0x627438aaeba9 - std::rt::lang_start::{{closure}}::h12f1472a703f9e15 [INFO] [stderr] 23: 0x627438abf530 - core::ops::function::impls:: for &F>::call_once::hafb92390cd746d32 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/core/src/ops/function.rs:290:21 [INFO] [stderr] 24: 0x627438abf530 - std::panicking::catch_unwind::do_call::hf9ca7f72b5cfc12d [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:590:40 [INFO] [stderr] 25: 0x627438abf530 - std::panicking::catch_unwind::h2633a0b3bb54fb62 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:553:19 [INFO] [stderr] 26: 0x627438abf530 - std::panic::catch_unwind::h7e09e435c07a712b [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panic.rs:359:14 [INFO] [stderr] 27: 0x627438abf530 - std::rt::lang_start_internal::{{closure}}::hca9746c45fe5e4fc [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/rt.rs:175:24 [INFO] [stderr] 28: 0x627438abf530 - std::panicking::catch_unwind::do_call::h2226e3146fd4fcbb [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:590:40 [INFO] [stderr] 29: 0x627438abf530 - std::panicking::catch_unwind::h95d7cf2cf54eaae4 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panicking.rs:553:19 [INFO] [stderr] 30: 0x627438abf530 - std::panic::catch_unwind::hc9da26b0dab9ae52 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/panic.rs:359:14 [INFO] [stderr] 31: 0x627438abf530 - std::rt::lang_start_internal::h31e79a1e3680eaa2 [INFO] [stderr] at /rustc/160e7623e8cbbf1feab2b6e2a24733a98c7bde9c/library/std/src/rt.rs:171:5 [INFO] [stderr] 32: 0x627438aaeb91 - std::rt::lang_start::he59047106037cf8e [INFO] [stderr] 33: 0x627438ab27f5 - main [INFO] [stderr] 34: 0x74e9a3a3d1ca - [INFO] [stderr] 35: 0x74e9a3a3d28b - __libc_start_main [INFO] [stderr] 36: 0x627438aacf05 - _start [INFO] [stderr] 37: 0x0 - [INFO] [stderr] warning: build failed, waiting for other jobs to finish... [INFO] running `Command { std: "docker" "inspect" "eb17c93d00d762f85c8b50c255aa967fcb3da85422d5388448cadb73d804a412", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "eb17c93d00d762f85c8b50c255aa967fcb3da85422d5388448cadb73d804a412", kill_on_drop: false }` [INFO] [stdout] eb17c93d00d762f85c8b50c255aa967fcb3da85422d5388448cadb73d804a412