[INFO] cloning repository https://github.com/huggingface/candle-extensions [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/huggingface/candle-extensions" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fhuggingface%2Fcandle-extensions", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fhuggingface%2Fcandle-extensions'... [INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }` [INFO] [stdout] e287dd09535ab5d7631c0f2ae0a93d7ec168cb86 [INFO] testing huggingface/candle-extensions against 1.95.0 for beta-1.96-2 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fhuggingface%2Fcandle-extensions" "/workspace/builds/worker-7-tc1/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-7-tc1/source'... [INFO] [stderr] done. [INFO] started tweaking git repo https://github.com/huggingface/candle-extensions [INFO] finished tweaking git repo https://github.com/huggingface/candle-extensions [INFO] tweaked toml for git repo https://github.com/huggingface/candle-extensions written to /workspace/builds/worker-7-tc1/source/Cargo.toml [INFO] validating manifest of git repo https://github.com/huggingface/candle-extensions on toolchain 1.95.0 [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.95.0" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] crate git repo https://github.com/huggingface/candle-extensions already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.95.0" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] [stderr] Updating crates.io index [INFO] [stderr] Downloading crates ... [INFO] [stderr] Downloaded num v0.4.3 [INFO] [stderr] Downloaded libloading v0.8.6 [INFO] [stderr] Downloaded gemm-f32 v0.18.2 [INFO] [stderr] Downloaded bytemuck_derive v1.9.3 [INFO] [stderr] Downloaded ug-cuda v0.1.0 [INFO] [stderr] Downloaded bindgen_cuda v0.1.5 [INFO] [stderr] Downloaded gemm-c32 v0.17.1 [INFO] [stderr] Downloaded gemm-c64 v0.17.1 [INFO] [stderr] Downloaded gemm v0.18.2 [INFO] [stderr] Downloaded glob v0.3.2 [INFO] [stderr] Downloaded raw-cpuid v10.7.0 [INFO] [stderr] Downloaded winnow v0.7.6 [INFO] [stderr] Downloaded raw-cpuid v11.5.0 [INFO] [stderr] Downloaded zip v1.1.4 [INFO] [stderr] Downloaded candle-kernels v0.8.4 [INFO] [stderr] Downloaded rand_distr v0.5.1 [INFO] [stderr] Downloaded candle-core v0.8.4 [INFO] [stderr] Downloaded pulp v0.21.4 [INFO] [stderr] Downloaded dyn-stack v0.13.0 [INFO] [stderr] Downloaded sysctl v0.5.5 [INFO] [stderr] Downloaded crunchy v0.2.3 [INFO] [stderr] Downloaded gemm-common v0.17.1 [INFO] [stderr] Downloaded gemm-f64 v0.17.1 [INFO] [stderr] Downloaded gemm-f32 v0.17.1 [INFO] [stderr] Downloaded anyhow v1.0.98 [INFO] [stderr] Downloaded num-complex v0.4.6 [INFO] [stderr] Downloaded arbitrary v1.4.1 [INFO] [stderr] Downloaded pulp v0.18.22 [INFO] [stderr] Downloaded toml_edit v0.22.24 [INFO] [stderr] Downloaded candle-nn v0.8.4 [INFO] [stderr] Downloaded safetensors v0.4.5 [INFO] [stderr] Downloaded sysctl v0.6.0 [INFO] [stderr] Downloaded num_enum v0.7.3 [INFO] [stderr] Downloaded reborrow v0.5.5 [INFO] [stderr] Downloaded gemm-common v0.18.2 [INFO] [stderr] Downloaded memmap2 v0.9.5 [INFO] [stderr] Downloaded ug v0.1.0 [INFO] [stderr] Downloaded indexmap v2.9.0 [INFO] [stderr] Downloaded num_enum_derive v0.7.3 [INFO] [stderr] Downloaded half v2.6.0 [INFO] [stderr] Downloaded dyn-stack v0.10.0 [INFO] [stderr] Downloaded gemm-f16 v0.18.2 [INFO] [stderr] Downloaded derive_arbitrary v1.4.1 [INFO] [stderr] Downloaded proc-macro-crate v3.3.0 [INFO] [stderr] Downloaded gemm v0.17.1 [INFO] [stderr] Downloaded gemm-c64 v0.18.2 [INFO] [stderr] Downloaded gemm-f64 v0.18.2 [INFO] [stderr] Downloaded libm v0.2.11 [INFO] [stderr] Downloaded gemm-f16 v0.17.1 [INFO] [stderr] Downloaded gemm-c32 v0.18.2 [INFO] [stderr] Downloaded seq-macro v0.3.6 [INFO] [stderr] Downloaded bytemuck v1.22.0 [INFO] [stderr] Downloaded cudarc v0.13.9 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+1.95.0" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }` [INFO] [stdout] 25b523bbf15707ac707438beaf76f923761b7bffccc786d8c06285798c045cbf [INFO] running `Command { std: "docker" "start" "-a" "25b523bbf15707ac707438beaf76f923761b7bffccc786d8c06285798c045cbf", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "25b523bbf15707ac707438beaf76f923761b7bffccc786d8c06285798c045cbf", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "25b523bbf15707ac707438beaf76f923761b7bffccc786d8c06285798c045cbf", kill_on_drop: false }` [INFO] [stdout] 25b523bbf15707ac707438beaf76f923761b7bffccc786d8c06285798c045cbf [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+1.95.0" "build" "--frozen" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] 5a359fc54415a9b1d8a299a959e9405bd011f50d5c53c45bb2ac476d2ac9f6ca [INFO] running `Command { std: "docker" "start" "-a" "5a359fc54415a9b1d8a299a959e9405bd011f50d5c53c45bb2ac476d2ac9f6ca", kill_on_drop: false }` [INFO] [stderr] Compiling proc-macro2 v1.0.94 [INFO] [stderr] Compiling unicode-ident v1.0.18 [INFO] [stderr] Compiling libc v0.2.171 [INFO] [stderr] Compiling crossbeam-utils v0.8.21 [INFO] [stderr] Compiling autocfg v1.4.0 [INFO] [stderr] Compiling libm v0.2.11 [INFO] [stderr] Compiling either v1.15.0 [INFO] [stderr] Compiling getrandom v0.3.2 [INFO] [stderr] Compiling zerocopy v0.8.24 [INFO] [stderr] Compiling reborrow v0.5.5 [INFO] [stderr] Compiling once_cell v1.21.3 [INFO] [stderr] Compiling seq-macro v0.3.6 [INFO] [stderr] Compiling pulp v0.21.4 [INFO] [stderr] Compiling bitflags v2.9.0 [INFO] [stderr] Compiling object v0.36.7 [INFO] [stderr] Compiling raw-cpuid v10.7.0 [INFO] [stderr] Compiling memchr v2.7.4 [INFO] [stderr] Compiling gimli v0.31.1 [INFO] [stderr] Compiling adler2 v2.0.0 [INFO] [stderr] Compiling miniz_oxide v0.8.8 [INFO] [stderr] Compiling raw-cpuid v11.5.0 [INFO] [stderr] Compiling anyhow v1.0.98 [INFO] [stderr] Compiling num-traits v0.2.19 [INFO] [stderr] Compiling hashbrown v0.15.2 [INFO] [stderr] Compiling rustc-demangle v0.1.24 [INFO] [stderr] Compiling glob v0.3.2 [INFO] [stderr] Compiling serde v1.0.219 [INFO] [stderr] Compiling equivalent v1.0.2 [INFO] [stderr] Compiling thiserror v1.0.69 [INFO] [stderr] Compiling serde_json v1.0.140 [INFO] [stderr] Compiling winnow v0.7.6 [INFO] [stderr] Compiling toml_datetime v0.6.8 [INFO] [stderr] Compiling itoa v1.0.15 [INFO] [stderr] Compiling quote v1.0.40 [INFO] [stderr] Compiling crossbeam-epoch v0.9.18 [INFO] [stderr] Compiling num-integer v0.1.46 [INFO] [stderr] Compiling syn v2.0.100 [INFO] [stderr] Compiling crossbeam-deque v0.8.6 [INFO] [stderr] Compiling stable_deref_trait v1.2.0 [INFO] [stderr] Compiling ryu v1.0.20 [INFO] [stderr] Compiling tracing-core v0.1.33 [INFO] [stderr] Compiling libloading v0.8.6 [INFO] [stderr] Compiling indexmap v2.9.0 [INFO] [stderr] Compiling cudarc v0.13.9 [INFO] [stderr] Compiling rayon-core v1.12.1 [INFO] [stderr] Compiling num-bigint v0.4.6 [INFO] [stderr] Compiling num-iter v0.1.45 [INFO] [stderr] Compiling pin-project-lite v0.2.16 [INFO] [stderr] Compiling zip v1.1.4 [INFO] [stderr] Compiling byteorder v1.5.0 [INFO] [stderr] error: failed to run custom build command for `cudarc v0.13.9` [INFO] [stderr] note: To improve backtraces for build dependencies, set the CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation. [INFO] [stderr] [INFO] [stderr] Caused by: [INFO] [stderr] process didn't exit successfully: `/opt/rustwide/target/debug/build/cudarc-a40c836595d3549f/build-script-build` (exit status: 101) [INFO] [stderr] --- stdout [INFO] [stderr] cargo:rerun-if-changed=build.rs [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_ROOT [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_PATH [INFO] [stderr] cargo:rerun-if-env-changed=CUDA_TOOLKIT_ROOT_DIR [INFO] [stderr] [INFO] [stderr] --- stderr [INFO] [stderr] [INFO] [stderr] thread 'main' (872) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/cudarc-0.13.9/build.rs:63:10: [INFO] [stderr] Failed to execute `nvcc`: Os { code: 2, kind: NotFound, message: "No such file or directory" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x61b99bad6dda - std[e28293b1aa0f68bd]::backtrace_rs::backtrace::libunwind::trace [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stderr] 1: 0x61b99bad6dda - std[e28293b1aa0f68bd]::backtrace_rs::backtrace::trace_unsynchronized:: [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stderr] 2: 0x61b99bad6dda - std[e28293b1aa0f68bd]::sys::backtrace::_print_fmt [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/sys/backtrace.rs:74:9 [INFO] [stderr] 3: 0x61b99bad6dda - <::print::DisplayBacktrace as core[c1f1a4ba060b9bfa]::fmt::Display>::fmt [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/sys/backtrace.rs:44:26 [INFO] [stderr] 4: 0x61b99bae808a - ::fmt [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/core/src/fmt/rt.rs:152:76 [INFO] [stderr] 5: 0x61b99bae808a - core[c1f1a4ba060b9bfa]::fmt::write [INFO] [stderr] 6: 0x61b99badaea2 - std[e28293b1aa0f68bd]::io::default_write_fmt:: [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/io/mod.rs:639:11 [INFO] [stderr] 7: 0x61b99badaea2 - ::write_fmt [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/io/mod.rs:1994:13 [INFO] [stderr] 8: 0x61b99babc65f - ::print [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/sys/backtrace.rs:47:9 [INFO] [stderr] 9: 0x61b99babc65f - std[e28293b1aa0f68bd]::panicking::default_hook::{closure#0} [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:292:27 [INFO] [stderr] 10: 0x61b99bad2a41 - std[e28293b1aa0f68bd]::panicking::default_hook [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:319:9 [INFO] [stderr] 11: 0x61b99bad2bfb - std[e28293b1aa0f68bd]::panicking::panic_with_hook [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:825:13 [INFO] [stderr] 12: 0x61b99babc718 - std[e28293b1aa0f68bd]::panicking::panic_handler::{closure#0} [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:698:13 [INFO] [stderr] 13: 0x61b99bab4219 - std[e28293b1aa0f68bd]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/sys/backtrace.rs:182:18 [INFO] [stderr] 14: 0x61b99babcf7d - __rustc[b7974e8690430dd9]::rust_begin_unwind [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:689:5 [INFO] [stderr] 15: 0x61b99bae889c - core[c1f1a4ba060b9bfa]::panicking::panic_fmt [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/core/src/panicking.rs:80:14 [INFO] [stderr] 16: 0x61b99bae8672 - core[c1f1a4ba060b9bfa]::result::unwrap_failed [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/core/src/result.rs:1867:5 [INFO] [stderr] 17: 0x61b99baa53f2 - core::result::Result::expect::h356adcd28edf53fd [INFO] [stderr] 18: 0x61b99baa0659 - build_script_build::cuda_version_from_build_system::ha99bb3f08a1a34f1 [INFO] [stderr] 19: 0x61b99baa0fb0 - build_script_build::main::hd73b34a6797c67ce [INFO] [stderr] 20: 0x61b99baa3923 - core::ops::function::FnOnce::call_once::h65c39527593fb849 [INFO] [stderr] 21: 0x61b99ba9a226 - std::sys::backtrace::__rust_begin_short_backtrace::h8d6b12c405a1802f [INFO] [stderr] 22: 0x61b99baa33c9 - std::rt::lang_start::{{closure}}::h7fd6f8ab13f56bbd [INFO] [stderr] 23: 0x61b99bad1c84 - <&dyn core[c1f1a4ba060b9bfa]::ops::function::Fn<(), Output = i32> + core[c1f1a4ba060b9bfa]::marker::Sync + core[c1f1a4ba060b9bfa]::panic::unwind_safe::RefUnwindSafe as core[c1f1a4ba060b9bfa]::ops::function::FnOnce<()>>::call_once [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/core/src/ops/function.rs:287:21 [INFO] [stderr] 24: 0x61b99bad1c84 - std[e28293b1aa0f68bd]::panicking::catch_unwind::do_call::<&dyn core[c1f1a4ba060b9bfa]::ops::function::Fn<(), Output = i32> + core[c1f1a4ba060b9bfa]::marker::Sync + core[c1f1a4ba060b9bfa]::panic::unwind_safe::RefUnwindSafe, i32> [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:581:40 [INFO] [stderr] 25: 0x61b99bad1c84 - std[e28293b1aa0f68bd]::panicking::catch_unwind:: + core[c1f1a4ba060b9bfa]::marker::Sync + core[c1f1a4ba060b9bfa]::panic::unwind_safe::RefUnwindSafe> [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:544:19 [INFO] [stderr] 26: 0x61b99bad1c84 - std[e28293b1aa0f68bd]::panic::catch_unwind::<&dyn core[c1f1a4ba060b9bfa]::ops::function::Fn<(), Output = i32> + core[c1f1a4ba060b9bfa]::marker::Sync + core[c1f1a4ba060b9bfa]::panic::unwind_safe::RefUnwindSafe, i32> [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panic.rs:359:14 [INFO] [stderr] 27: 0x61b99bad1c84 - std[e28293b1aa0f68bd]::rt::lang_start_internal::{closure#0} [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/rt.rs:175:24 [INFO] [stderr] 28: 0x61b99bad1c84 - std[e28293b1aa0f68bd]::panicking::catch_unwind::do_call:: [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:581:40 [INFO] [stderr] 29: 0x61b99bad1c84 - std[e28293b1aa0f68bd]::panicking::catch_unwind:: [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:544:19 [INFO] [stderr] 30: 0x61b99bad1c84 - std[e28293b1aa0f68bd]::panic::catch_unwind:: [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panic.rs:359:14 [INFO] [stderr] 31: 0x61b99bad1c84 - std[e28293b1aa0f68bd]::rt::lang_start_internal [INFO] [stderr] at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/rt.rs:171:5 [INFO] [stderr] 32: 0x61b99baa33b1 - std::rt::lang_start::hccff11c563895e0a [INFO] [stderr] 33: 0x61b99baa2cf5 - main [INFO] [stderr] 34: 0x75fcf103e1ca - [INFO] [stderr] 35: 0x75fcf103e28b - __libc_start_main [INFO] [stderr] 36: 0x61b99ba99eb5 - _start [INFO] [stderr] 37: 0x0 - [INFO] [stderr] warning: build failed, waiting for other jobs to finish... [INFO] running `Command { std: "docker" "inspect" "5a359fc54415a9b1d8a299a959e9405bd011f50d5c53c45bb2ac476d2ac9f6ca", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "5a359fc54415a9b1d8a299a959e9405bd011f50d5c53c45bb2ac476d2ac9f6ca", kill_on_drop: false }` [INFO] [stdout] 5a359fc54415a9b1d8a299a959e9405bd011f50d5c53c45bb2ac476d2ac9f6ca