[INFO] cloning repository https://github.com/Jake-Purton/candle_nn [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/Jake-Purton/candle_nn" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FJake-Purton%2Fcandle_nn", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FJake-Purton%2Fcandle_nn'... [INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }` [INFO] [stdout] 3254073c26fa6c5ba56e670e3a74bde8bf7743e2 [INFO] checking Jake-Purton/candle_nn/3254073c26fa6c5ba56e670e3a74bde8bf7743e2 against master#90b65889799733f21ebdf59d96411aa531c5900a for pr-148477 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FJake-Purton%2Fcandle_nn" "/workspace/builds/worker-3-tc1/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-3-tc1/source'... [INFO] [stderr] done. [INFO] started tweaking git repo https://github.com/Jake-Purton/candle_nn [INFO] finished tweaking git repo https://github.com/Jake-Purton/candle_nn [INFO] tweaked toml for git repo https://github.com/Jake-Purton/candle_nn written to /workspace/builds/worker-3-tc1/source/Cargo.toml [INFO] validating manifest of git repo https://github.com/Jake-Purton/candle_nn on toolchain 90b65889799733f21ebdf59d96411aa531c5900a [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+90b65889799733f21ebdf59d96411aa531c5900a" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] crate git repo https://github.com/Jake-Purton/candle_nn already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+90b65889799733f21ebdf59d96411aa531c5900a" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] [stderr] Updating crates.io index [INFO] [stderr] Downloading crates ... [INFO] [stderr] Downloaded rust-mnist v0.2.0 [INFO] [stderr] Downloaded candle-nn v0.2.2 [INFO] [stderr] Downloaded dyn-stack v0.9.0 [INFO] [stderr] Downloaded clap v4.4.6 [INFO] [stderr] Downloaded const-random-macro v0.1.15 [INFO] [stderr] Downloaded arrow-buffer v45.0.0 [INFO] [stderr] Downloaded indicatif v0.17.7 [INFO] [stderr] Downloaded rustls-webpki v0.100.3 [INFO] [stderr] Downloaded esaxx-rs v0.1.8 [INFO] [stderr] Downloaded arrow-array v45.0.0 [INFO] [stderr] Downloaded unicode-normalization-alignments v0.1.12 [INFO] [stderr] Downloaded webpki-roots v0.23.1 [INFO] [stderr] Downloaded arrow-ipc v45.0.0 [INFO] [stderr] Downloaded ureq v2.7.1 [INFO] [stderr] Downloaded parquet v45.0.0 [INFO] [stderr] Downloaded spm_precompiled v0.1.4 [INFO] [stderr] Downloaded tokenizers v0.13.4 [INFO] [stderr] Downloaded arrow-cast v45.0.0 [INFO] [stderr] Downloaded arrow-data v45.0.0 [INFO] [stderr] Downloaded arrow-schema v45.0.0 [INFO] [stderr] Downloaded arrow-select v45.0.0 [INFO] [stderr] Downloaded rayon-cond v0.1.0 [INFO] [stderr] Downloaded candle-gemm-f16 v0.16.0 [INFO] [stderr] Downloaded snap v1.1.0 [INFO] [stderr] Downloaded macro_rules_attribute-proc_macro v0.1.3 [INFO] [stderr] Downloaded monostate v0.1.9 [INFO] [stderr] Downloaded candle-gemm-c32 v0.16.0 [INFO] [stderr] Downloaded candle-gemm-c64 v0.16.0 [INFO] [stderr] Downloaded candle-gemm-f32 v0.16.0 [INFO] [stderr] Downloaded candle-gemm-common v0.16.0 [INFO] [stderr] Downloaded candle-core v0.2.2 [INFO] [stderr] Downloaded clap_builder v4.4.6 [INFO] [stderr] Downloaded const-random v0.1.15 [INFO] [stderr] Downloaded macro_rules_attribute v0.1.3 [INFO] [stderr] Downloaded monostate-impl v0.1.9 [INFO] [stderr] Downloaded candle-gemm-f64 v0.16.0 [INFO] [stderr] Downloaded candle-gemm v0.16.0 [INFO] [stderr] Downloaded candle-kernels v0.2.2 [INFO] [stderr] Downloaded hf-hub v0.3.1 [INFO] [stderr] Downloaded candle-datasets v0.2.2 [INFO] [stderr] Downloaded cudarc v0.9.14 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-3-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-3-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+90b65889799733f21ebdf59d96411aa531c5900a" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }` [INFO] [stdout] 822638f6e18bf4ae4153d04e786385a0c94735175b7ba8ae388deab4004bf07f [INFO] running `Command { std: "docker" "start" "-a" "822638f6e18bf4ae4153d04e786385a0c94735175b7ba8ae388deab4004bf07f", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "822638f6e18bf4ae4153d04e786385a0c94735175b7ba8ae388deab4004bf07f", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "822638f6e18bf4ae4153d04e786385a0c94735175b7ba8ae388deab4004bf07f", kill_on_drop: false }` [INFO] [stdout] 822638f6e18bf4ae4153d04e786385a0c94735175b7ba8ae388deab4004bf07f [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-3-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-3-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+90b65889799733f21ebdf59d96411aa531c5900a" "check" "--frozen" "--all" "--all-targets" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] 1412e0e736707e8dca88566acfa0ebced22d6d2d431e604d5ccaab0fd6a409d6 [INFO] running `Command { std: "docker" "start" "-a" "1412e0e736707e8dca88566acfa0ebced22d6d2d431e604d5ccaab0fd6a409d6", kill_on_drop: false }` [INFO] [stderr] Compiling libc v0.2.148 [INFO] [stderr] Compiling libm v0.2.7 [INFO] [stderr] Compiling proc-macro2 v1.0.67 [INFO] [stderr] Compiling crossbeam-utils v0.8.16 [INFO] [stderr] Compiling rayon-core v1.12.0 [INFO] [stderr] Checking either v1.9.0 [INFO] [stderr] Compiling num-traits v0.2.16 [INFO] [stderr] Compiling memoffset v0.9.0 [INFO] [stderr] Compiling crossbeam-epoch v0.9.15 [INFO] [stderr] Compiling num-bigint v0.4.4 [INFO] [stderr] Compiling num-rational v0.4.1 [INFO] [stderr] Compiling serde v1.0.188 [INFO] [stderr] Checking bytemuck v1.14.0 [INFO] [stderr] Compiling num-iter v0.1.43 [INFO] [stderr] Compiling seq-macro v0.3.5 [INFO] [stderr] Checking reborrow v0.5.4 [INFO] [stderr] Checking raw-cpuid v10.7.0 [INFO] [stderr] Compiling syn v1.0.109 [INFO] [stderr] Checking lexical-util v0.8.5 [INFO] [stderr] Compiling serde_json v1.0.107 [INFO] [stderr] Compiling semver v1.0.19 [INFO] [stderr] Compiling ahash v0.8.3 [INFO] [stderr] Checking dyn-stack v0.9.0 [INFO] [stderr] Checking memchr v2.6.3 [INFO] [stderr] Checking arrow-schema v45.0.0 [INFO] [stderr] Checking hashbrown v0.14.0 [INFO] [stderr] Compiling openssl v0.10.57 [INFO] [stderr] Compiling zstd-safe v6.0.6 [INFO] [stderr] Compiling portable-atomic v1.4.3 [INFO] [stderr] Compiling rustls v0.21.7 [INFO] [stderr] Compiling cudarc v0.9.14 [INFO] [stderr] Checking regex-syntax v0.7.5 [INFO] [stderr] Checking lexical-write-integer v0.8.5 [INFO] [stderr] Compiling quote v1.0.33 [INFO] [stderr] Checking lexical-parse-integer v0.8.6 [INFO] [stderr] Checking lexical-write-float v0.8.5 [INFO] [stderr] Checking aho-corasick v1.1.1 [INFO] [stderr] Compiling jobserver v0.1.26 [INFO] [stderr] Checking crossbeam-deque v0.8.3 [INFO] [stderr] Compiling rustc_version v0.4.0 [INFO] [stderr] Compiling syn v2.0.37 [INFO] [stderr] Compiling esaxx-rs v0.1.8 [INFO] [stderr] Checking base64 v0.21.4 [INFO] [stderr] error: failed to run custom build command for `cudarc v0.9.14` [INFO] [stderr] note: To improve backtraces for build dependencies, set the CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation. [INFO] [stderr] [INFO] [stderr] Caused by: [INFO] [stderr] process didn't exit successfully: `/opt/rustwide/target/debug/build/cudarc-8bdd98512bfb498c/build-script-build` (exit status: 101) [INFO] [stderr] --- stdout [INFO] [stderr] cargo:rerun-if-changed=build.rs [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_ROOT [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_PATH [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_TOOLKIT_ROOT_DIR [INFO] [stderr] [INFO] [stderr] --- stderr [INFO] [stderr] [INFO] [stderr] thread 'main' (1018) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/cudarc-0.9.14/build.rs:21:13: [INFO] [stderr] Unable to find `include/cuda.h` under any of: ["/usr", "/usr/local/cuda", "/opt/cuda", "/usr/lib/cuda", "C:/Program Files/NVIDIA GPU Computing Toolkit", "C:/CUDA"]. Set the `CUDA_ROOT` environment variable to `$CUDA_ROOT/include/cuda.h` to override path. [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x6408bcb5e3f2 - std::backtrace_rs::backtrace::libunwind::trace::h3dfece87d38e678e [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stderr] 1: 0x6408bcb5e3f2 - std::backtrace_rs::backtrace::trace_unsynchronized::h598b3d1a4cc142bd [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stderr] 2: 0x6408bcb5e3f2 - std::sys::backtrace::_print_fmt::h8a45e1e0a8207652 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/sys/backtrace.rs:66:9 [INFO] [stderr] 3: 0x6408bcb5e3f2 - ::fmt::h308c0d880f760a32 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/sys/backtrace.rs:39:26 [INFO] [stderr] 4: 0x6408bcb6d0cf - core::fmt::rt::Argument::fmt::h3481dbf5753231bc [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/core/src/fmt/rt.rs:173:76 [INFO] [stderr] 5: 0x6408bcb6d0cf - core::fmt::write::haa945362a56e47a4 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/core/src/fmt/mod.rs:1469:25 [INFO] [stderr] 6: 0x6408bcb3c433 - std::io::default_write_fmt::h4f465fb81c168d76 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/io/mod.rs:639:11 [INFO] [stderr] 7: 0x6408bcb3c433 - std::io::Write::write_fmt::h73e277fd36a2f897 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/io/mod.rs:1954:13 [INFO] [stderr] 8: 0x6408bcb41892 - std::sys::backtrace::BacktraceLock::print::h23fdb9296ed52a08 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/sys/backtrace.rs:42:9 [INFO] [stderr] 9: 0x6408bcb4330c - std::panicking::default_hook::{{closure}}::h874fba906a1eead0 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:301:27 [INFO] [stderr] 10: 0x6408bcb43166 - std::panicking::default_hook::h4a2d336b415f97ea [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:328:9 [INFO] [stderr] 11: 0x6408bcb438e5 - std::panicking::panic_with_hook::h1312efcfed45defb [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:834:13 [INFO] [stderr] 12: 0x6408bcb4377a - std::panicking::panic_handler::{{closure}}::h5f7c2e03b7897253 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:707:13 [INFO] [stderr] 13: 0x6408bcb419c9 - std::sys::backtrace::__rust_end_short_backtrace::h5999a8d4ddd56aac [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/sys/backtrace.rs:174:18 [INFO] [stderr] 14: 0x6408bcb30fcd - __rustc[5252cc545ffd5943]::rust_begin_unwind [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:698:5 [INFO] [stderr] 15: 0x6408bcb701a0 - core::panicking::panic_fmt::h08c11d25cce8cb30 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/core/src/panicking.rs:80:14 [INFO] [stderr] 16: 0x6408bcb2d13a - build_script_build::link_cuda::{{closure}}::hb8d9cd0098a69d06 [INFO] [stderr] 17: 0x6408bcb2f6f9 - core::option::Option::unwrap_or_else::h547d6ca401403329 [INFO] [stderr] 18: 0x6408bcb2cdb5 - build_script_build::link_cuda::hf135d590f0c71953 [INFO] [stderr] 19: 0x6408bcb2cc75 - build_script_build::main::h86f1482472fb56c8 [INFO] [stderr] 20: 0x6408bcb2d913 - core::ops::function::FnOnce::call_once::hf9c009f069a4b42f [INFO] [stderr] 21: 0x6408bcb2d326 - std::sys::backtrace::__rust_begin_short_backtrace::h5f0d67c3c4dc866d [INFO] [stderr] 22: 0x6408bcb2d289 - std::rt::lang_start::{{closure}}::h44ed7381d8164c91 [INFO] [stderr] 23: 0x6408bcb3d870 - core::ops::function::impls:: for &F>::call_once::h4f6436b78d6afdc7 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/core/src/ops/function.rs:287:21 [INFO] [stderr] 24: 0x6408bcb3d870 - std::panicking::catch_unwind::do_call::h11545474392ae041 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:590:40 [INFO] [stderr] 25: 0x6408bcb3d870 - std::panicking::catch_unwind::h73c184249c61faac [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:553:19 [INFO] [stderr] 26: 0x6408bcb3d870 - std::panic::catch_unwind::h87b5477170146628 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panic.rs:359:14 [INFO] [stderr] 27: 0x6408bcb3d870 - std::rt::lang_start_internal::{{closure}}::h70bc8394d41bdbd5 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/rt.rs:175:24 [INFO] [stderr] 28: 0x6408bcb3d870 - std::panicking::catch_unwind::do_call::h1357cf1f57518799 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:590:40 [INFO] [stderr] 29: 0x6408bcb3d870 - std::panicking::catch_unwind::h51fa10671853189c [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panicking.rs:553:19 [INFO] [stderr] 30: 0x6408bcb3d870 - std::panic::catch_unwind::h3d96451d4b7c4f47 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/panic.rs:359:14 [INFO] [stderr] 31: 0x6408bcb3d870 - std::rt::lang_start_internal::hfb2d56f56ca71999 [INFO] [stderr] at /rustc/90b65889799733f21ebdf59d96411aa531c5900a/library/std/src/rt.rs:171:5 [INFO] [stderr] 32: 0x6408bcb2d271 - std::rt::lang_start::h763fae8b1a24d4fe [INFO] [stderr] 33: 0x6408bcb30ce5 - main [INFO] [stderr] 34: 0x7fdcd63ed1ca - [INFO] [stderr] 35: 0x7fdcd63ed28b - __libc_start_main [INFO] [stderr] 36: 0x6408bcb2b7c5 - _start [INFO] [stderr] 37: 0x0 - [INFO] [stderr] warning: build failed, waiting for other jobs to finish... [INFO] running `Command { std: "docker" "inspect" "1412e0e736707e8dca88566acfa0ebced22d6d2d431e604d5ccaab0fd6a409d6", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "1412e0e736707e8dca88566acfa0ebced22d6d2d431e604d5ccaab0fd6a409d6", kill_on_drop: false }` [INFO] [stdout] 1412e0e736707e8dca88566acfa0ebced22d6d2d431e604d5ccaab0fd6a409d6