[INFO] cloning repository https://github.com/mixlayer/marlin-moe-rs
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/mixlayer/marlin-moe-rs" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fmixlayer%2Fmarlin-moe-rs", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fmixlayer%2Fmarlin-moe-rs'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] f10b07f4eb6479b78714eade05a1bcf786e9c41b
[INFO] checking mixlayer/marlin-moe-rs against 3154cc1a3e6424e44f438a95007d043ed5ceaaf6+rustflags=-Znext-solver=coherence for pr-133502-30
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fmixlayer%2Fmarlin-moe-rs" "/workspace/builds/worker-6-tc1/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-6-tc1/source'...
[INFO] [stderr] done.
[INFO] removed /workspace/builds/worker-6-tc1/source/.cargo/config.toml
[INFO] started tweaking git repo https://github.com/mixlayer/marlin-moe-rs
[INFO] finished tweaking git repo https://github.com/mixlayer/marlin-moe-rs
[INFO] tweaked toml for git repo https://github.com/mixlayer/marlin-moe-rs written to /workspace/builds/worker-6-tc1/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/mixlayer/marlin-moe-rs on toolchain 3154cc1a3e6424e44f438a95007d043ed5ceaaf6
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+3154cc1a3e6424e44f438a95007d043ed5ceaaf6" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/mixlayer/marlin-moe-rs already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+3154cc1a3e6424e44f438a95007d043ed5ceaaf6" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Updating git repository `https://github.com/mixlayer/candle`
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded float8 v0.6.1
[INFO] [stderr]   Downloaded gemm v0.19.0
[INFO] [stderr]   Downloaded float8 v0.7.0
[INFO] [stderr]   Downloaded gemm-f64 v0.19.0
[INFO] [stderr]   Downloaded gemm-f32 v0.19.0
[INFO] [stderr]   Downloaded gemm-c64 v0.19.0
[INFO] [stderr]   Downloaded gemm-common v0.19.0
[INFO] [stderr]   Downloaded gemm-c32 v0.19.0
[INFO] [stderr]   Downloaded dyn-stack-macros v0.1.3
[INFO] [stderr]   Downloaded gemm-f16 v0.19.0
[INFO] [stderr]   Downloaded bindgen_cuda v0.1.6
[INFO] [stderr]   Downloaded sysctl v0.6.0
[INFO] [stderr]   Downloaded dyn-stack v0.13.2
[INFO] [stderr]   Downloaded cudarc v0.19.8
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-6-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-6-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-m" "1610612736" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:77db811e55d90add9212f6832c23229e3de5f1c1f9905e8cb07319ea15df73ac" "sleep" "infinity", kill_on_drop: false }`
[INFO] [stdout] 1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074
[INFO] running `Command { std: "docker" "start" "1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-w" "/opt/rustwide/workdir" "--user" "0:0" "1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074" "/opt/rustwide/cargo-home/bin/cargo" "+3154cc1a3e6424e44f438a95007d043ed5ceaaf6" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid -Znext-solver=coherence" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-w" "/opt/rustwide/workdir" "--user" "0:0" "1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074" "/opt/rustwide/cargo-home/bin/cargo" "+3154cc1a3e6424e44f438a95007d043ed5ceaaf6" "check" "--frozen" "--all" "--all-targets" "--message-format=json", kill_on_drop: false }`
[INFO] [stderr]    Compiling crossbeam-utils v0.8.22
[INFO] [stderr]    Compiling crossbeam-epoch v0.9.20
[INFO] [stderr]    Compiling crossbeam-deque v0.8.7
[INFO] [stderr]    Compiling zerocopy-derive v0.8.55
[INFO] [stderr]    Compiling zerocopy v0.8.55
[INFO] [stderr]    Compiling bytemuck_derive v1.11.0
[INFO] [stderr]    Compiling pulp v0.22.3
[INFO] [stderr]     Checking raw-cpuid v11.6.0
[INFO] [stderr]     Checking reborrow v0.5.5
[INFO] [stderr]    Compiling dyn-stack-macros v0.1.3
[INFO] [stderr]     Checking either v1.17.0
[INFO] [stderr]     Checking pulp-wasm-simd-flag v0.1.1
[INFO] [stderr]    Compiling seq-macro v0.3.6
[INFO] [stderr]    Compiling serde_core v1.0.229
[INFO] [stderr]    Compiling syn v3.0.3
[INFO] [stderr]    Compiling num_cpus v1.17.0
[INFO] [stderr]     Checking rayon-core v1.13.0
[INFO] [stderr]    Compiling marlin-moe-sys v0.1.0 (/opt/rustwide/workdir/crates/marlin-moe-sys)
[INFO] [stderr]     Checking rayon v1.12.0
[INFO] [stderr] error: failed to run custom build command for `marlin-moe-sys v0.1.0 (/opt/rustwide/workdir/crates/marlin-moe-sys)`
[INFO] [stderr] note: To improve backtraces for build dependencies, set the CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   process didn't exit successfully: `/opt/rustwide/target/debug/build/marlin-moe-sys/a9e8f13ec8bb4fff/out/build_script_build` (exit status: 101)
[INFO] [stderr]   --- stdout
[INFO] [stderr]   cargo:rerun-if-env-changed=CUDA_HOME
[INFO] [stderr]   cargo:rerun-if-env-changed=CUDA_ROOT
[INFO] [stderr]   cargo:rerun-if-env-changed=CUDA_PATH
[INFO] [stderr]   cargo:rerun-if-env-changed=CUDA_COMPUTE_CAP
[INFO] [stderr]   cargo:rerun-if-env-changed=MARLIN_MOE_ARCHS
[INFO] [stderr]   cargo:rerun-if-env-changed=MARLIN_MOE_BUILD_DIR
[INFO] [stderr]   cargo:rerun-if-env-changed=CANDLE_NVCC_CCBIN
[INFO] [stderr]   cargo:rerun-if-env-changed=NVCC_THREADS
[INFO] [stderr]   cargo:rerun-if-env-changed=MARLIN_MOE_NO_CCACHE
[INFO] [stderr]   cargo:rerun-if-env-changed=MARLIN_MOE_PTXAS_VERBOSE
[INFO] [stderr]   cargo:rerun-if-changed=build.rs
[INFO] [stderr]   cargo:rerun-if-changed=csrc/marlin_moe_c_api.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/moe/marlin_moe_wna16/ops.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/moe/marlin_moe_wna16/sm80_kernel_bfloat16_u4_bfloat16.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/moe/marlin_moe_wna16/sm80_kernel_bfloat16_u4b8_bfloat16.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/moe/marlin_moe_wna16/sm80_kernel_float16_u4_float16.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/moe/marlin_moe_wna16/sm80_kernel_float16_u4b8_float16.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/quantization/marlin/gptq_marlin_repack.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/moe/moe_support.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/moe/moe_sum_reduce.cu
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin/moe/marlin_moe_wna16/sm80_kernel_bfloat16_fe4m3fn_bfloat16.cu
[INFO] [stderr]   cargo:rerun-if-changed=csrc/marlin_moe_c_api.h
[INFO] [stderr]   cargo:rerun-if-changed=vendor/marlin
[INFO] [stderr]   cargo:rustc-env=MARLIN_MOE_SOURCE_ROOT=/opt/rustwide/workdir/crates/marlin-moe-sys/vendor/marlin
[INFO] [stderr] 
[INFO] [stderr]   --- stderr
[INFO] [stderr] 
[INFO] [stderr]   thread 'main' (238) panicked at crates/marlin-moe-sys/build.rs:154:28:
[INFO] [stderr]   nvcc not found; set CUDA_HOME, CUDA_ROOT, or CUDA_PATH
[INFO] [stderr]   stack backtrace:
[INFO] [stderr]      0:     0x5814a286cd21 - std[3164a59e47d3bde]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stderr]      1:     0x5814a286cd21 - std[3164a59e47d3bde]::backtrace_rs::backtrace::trace_unsynchronized::<std[3164a59e47d3bde]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stderr]      2:     0x5814a286cd21 - std[3164a59e47d3bde]::sys::backtrace::_print_fmt
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/sys/backtrace.rs:74:9
[INFO] [stderr]      3:     0x5814a286cd21 - <<std[3164a59e47d3bde]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[934024a1e6126fb4]::fmt::Display>::fmt
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/sys/backtrace.rs:44:26
[INFO] [stderr]      4:     0x5814a287f67a - <core[934024a1e6126fb4]::fmt::rt::Argument>::fmt
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/core/src/fmt/rt.rs:152:76
[INFO] [stderr]      5:     0x5814a287f67a - core[934024a1e6126fb4]::fmt::write
[INFO] [stderr]      6:     0x5814a2871c1c - core[934024a1e6126fb4]::io::write::default_write_fmt::<std[3164a59e47d3bde]::sys::stdio::unix::Stderr>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/core/src/io/write.rs:402:11
[INFO] [stderr]      7:     0x5814a2871c1c - <std[3164a59e47d3bde]::sys::stdio::unix::Stderr as core[934024a1e6126fb4]::io::write::Write>::write_fmt
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/core/src/io/write.rs:335:13
[INFO] [stderr]      8:     0x5814a284f556 - <std[3164a59e47d3bde]::sys::backtrace::BacktraceLock>::print
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/sys/backtrace.rs:47:9
[INFO] [stderr]      9:     0x5814a284f556 - std[3164a59e47d3bde]::panicking::default_hook::{closure#0}
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:289:27
[INFO] [stderr]     10:     0x5814a2868a31 - std[3164a59e47d3bde]::panicking::default_hook
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:316:9
[INFO] [stderr]     11:     0x5814a2868beb - std[3164a59e47d3bde]::panicking::panic_with_hook
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:811:13
[INFO] [stderr]     12:     0x5814a284f634 - std[3164a59e47d3bde]::panicking::panic_handler::{closure#0}
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:678:13
[INFO] [stderr]     13:     0x5814a28483f9 - std[3164a59e47d3bde]::sys::backtrace::__rust_end_short_backtrace::<std[3164a59e47d3bde]::panicking::panic_handler::{closure#0}, !>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/sys/backtrace.rs:182:18
[INFO] [stderr]     14:     0x5814a284fdcd - __rustc[2e7b49d4226056d3]::rust_begin_unwind
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:676:5
[INFO] [stderr]     15:     0x5814a287fdec - core[934024a1e6126fb4]::panicking::panic_fmt
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/core/src/panicking.rs:80:14
[INFO] [stderr]     16:     0x5814a2830525 - build_script_build[9c6f17f44a092c18]::discover_cuda_root::{closure#0}
[INFO] [stderr]     17:     0x5814a282e8a4 - <core[934024a1e6126fb4]::option::Option<std[3164a59e47d3bde]::path::PathBuf>>::unwrap_or_else::<build_script_build[9c6f17f44a092c18]::discover_cuda_root::{closure#0}>
[INFO] [stderr]     18:     0x5814a28313a3 - build_script_build[9c6f17f44a092c18]::discover_cuda_root
[INFO] [stderr]     19:     0x5814a28323e3 - build_script_build[9c6f17f44a092c18]::main
[INFO] [stderr]     20:     0x5814a283cbd3 - <fn() as core[934024a1e6126fb4]::ops::function::FnOnce<()>>::call_once
[INFO] [stderr]     21:     0x5814a283976e - std[3164a59e47d3bde]::sys::backtrace::__rust_begin_short_backtrace::<fn(), ()>
[INFO] [stderr]     22:     0x5814a2839e19 - std[3164a59e47d3bde]::rt::lang_start::<()>::{closure#0}
[INFO] [stderr]     23:     0x5814a2867ea8 - <&dyn core[934024a1e6126fb4]::ops::function::Fn<(), Output = i32> + core[934024a1e6126fb4]::marker::Sync + core[934024a1e6126fb4]::panic::unwind_safe::RefUnwindSafe as core[934024a1e6126fb4]::ops::function::FnOnce<()>>::call_once
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/core/src/ops/function.rs:287:21
[INFO] [stderr]     24:     0x5814a2867ea8 - std[3164a59e47d3bde]::panicking::catch_unwind::do_call::<&dyn core[934024a1e6126fb4]::ops::function::Fn<(), Output = i32> + core[934024a1e6126fb4]::marker::Sync + core[934024a1e6126fb4]::panic::unwind_safe::RefUnwindSafe, i32>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:573:43
[INFO] [stderr]     25:     0x5814a2867ea8 - std[3164a59e47d3bde]::panicking::catch_unwind::<i32, &dyn core[934024a1e6126fb4]::ops::function::Fn<(), Output = i32> + core[934024a1e6126fb4]::marker::Sync + core[934024a1e6126fb4]::panic::unwind_safe::RefUnwindSafe>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:541:19
[INFO] [stderr]     26:     0x5814a2867ea8 - std[3164a59e47d3bde]::panic::catch_unwind::<&dyn core[934024a1e6126fb4]::ops::function::Fn<(), Output = i32> + core[934024a1e6126fb4]::marker::Sync + core[934024a1e6126fb4]::panic::unwind_safe::RefUnwindSafe, i32>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panic.rs:359:14
[INFO] [stderr]     27:     0x5814a2867ea8 - std[3164a59e47d3bde]::rt::lang_start_internal::{closure#0}
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/rt.rs:175:24
[INFO] [stderr]     28:     0x5814a2867ea8 - std[3164a59e47d3bde]::panicking::catch_unwind::do_call::<std[3164a59e47d3bde]::rt::lang_start_internal::{closure#0}, isize>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:573:43
[INFO] [stderr]     29:     0x5814a2867ea8 - std[3164a59e47d3bde]::panicking::catch_unwind::<isize, std[3164a59e47d3bde]::rt::lang_start_internal::{closure#0}>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panicking.rs:541:19
[INFO] [stderr]     30:     0x5814a2867ea8 - std[3164a59e47d3bde]::panic::catch_unwind::<std[3164a59e47d3bde]::rt::lang_start_internal::{closure#0}, isize>
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/panic.rs:359:14
[INFO] [stderr]     31:     0x5814a2867ea8 - std[3164a59e47d3bde]::rt::lang_start_internal
[INFO] [stderr]                                  at /rustc/3154cc1a3e6424e44f438a95007d043ed5ceaaf6/library/std/src/rt.rs:171:5
[INFO] [stderr]     32:     0x5814a2839081 - std[3164a59e47d3bde]::rt::lang_start::<()>
[INFO] [stderr]     33:     0x5814a28370f5 - main
[INFO] [stderr]     34:     0x74a2ad31a601 - <unknown>
[INFO] [stderr]     35:     0x74a2ad31a718 - __libc_start_main
[INFO] [stderr]     36:     0x5814a282e675 - _start
[INFO] [stderr]     37:                0x0 - <unknown>
[INFO] [stderr] warning: build failed, waiting for other jobs to finish...
[INFO] running `Command { std: "docker" "inspect" "1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074", kill_on_drop: false }`
[INFO] [stdout] 1e0c03e2f87ee4b08b22352fc66fba034e1c0dceb82833fd4bb8f35329115074
