[INFO] cloning repository https://github.com/eisenzopf/llm-engine
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/eisenzopf/llm-engine" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Feisenzopf%2Fllm-engine", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Feisenzopf%2Fllm-engine'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 35fb4dba488d6c7c5b29f8123808d7fb748fccd8
[INFO] checking eisenzopf/llm-engine against master#82310651b93a594a3fd69015e1562186a080d94c for pr-145770
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Feisenzopf%2Fllm-engine" "/workspace/builds/worker-4-tc1/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-4-tc1/source'...
[INFO] [stderr] done.
[INFO] started tweaking git repo https://github.com/eisenzopf/llm-engine
[INFO] removed 0 missing examples
[INFO] finished tweaking git repo https://github.com/eisenzopf/llm-engine
[INFO] tweaked toml for git repo https://github.com/eisenzopf/llm-engine written to /workspace/builds/worker-4-tc1/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/eisenzopf/llm-engine on toolchain 82310651b93a594a3fd69015e1562186a080d94c
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+82310651b93a594a3fd69015e1562186a080d94c" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/eisenzopf/llm-engine already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+82310651b93a594a3fd69015e1562186a080d94c" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Blocking waiting for file lock on package cache
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded tokio-test v0.4.4
[INFO] [stderr]   Downloaded thiserror v1.0.68
[INFO] [stderr]   Downloaded thiserror-impl v1.0.68
[INFO] [stderr]   Downloaded oorandom v11.1.4
[INFO] [stderr]   Downloaded serde_derive v1.0.214
[INFO] [stderr]   Downloaded gemm-f64 v0.17.1
[INFO] [stderr]   Downloaded bindgen_cuda v0.1.5
[INFO] [stderr]   Downloaded gemm-c64 v0.17.1
[INFO] [stderr]   Downloaded monostate-impl v0.1.13
[INFO] [stderr]   Downloaded cuda-runtime-sys v0.3.0-alpha.1
[INFO] [stderr]   Downloaded dyn-stack v0.10.0
[INFO] [stderr]   Downloaded gemm-c32 v0.17.1
[INFO] [stderr]   Downloaded sysctl v0.5.5
[INFO] [stderr]   Downloaded gemm-common v0.17.1
[INFO] [stderr]   Downloaded derive_builder_core v0.20.2
[INFO] [stderr]   Downloaded monostate v0.1.13
[INFO] [stderr]   Downloaded safetensors v0.4.5
[INFO] [stderr]   Downloaded candle-core v0.7.2
[INFO] [stderr]   Downloaded cc v1.1.36
[INFO] [stderr]   Downloaded candle-nn v0.7.2
[INFO] [stderr]   Downloaded rustix v0.38.39
[INFO] [stderr]   Downloaded spm_precompiled v0.1.4
[INFO] [stderr]   Downloaded onig_sys v69.8.1
[INFO] [stderr]   Downloaded pulp v0.18.22
[INFO] [stderr]   Downloaded esaxx-rs v0.1.10
[INFO] [stderr]   Downloaded fancy-regex v0.13.0
[INFO] [stderr]   Downloaded raw-cpuid v10.7.0
[INFO] [stderr]   Downloaded tokio v1.41.0
[INFO] [stderr]   Downloaded candle-transformers v0.7.2
[INFO] [stderr]   Downloaded zip v1.1.4
[INFO] [stderr]   Downloaded serde_plain v1.0.2
[INFO] [stderr]   Downloaded derive_builder_macro v0.20.2
[INFO] [stderr]   Downloaded derive_builder v0.20.2
[INFO] [stderr]   Downloaded macro_rules_attribute-proc_macro v0.2.0
[INFO] [stderr]   Downloaded macro_rules_attribute v0.2.0
[INFO] [stderr]   Downloaded onig v6.4.0
[INFO] [stderr]   Downloaded tokenizers v0.20.3
[INFO] [stderr]   Downloaded gemm-f16 v0.17.1
[INFO] [stderr]   Downloaded gemm-f32 v0.17.1
[INFO] [stderr]   Downloaded reborrow v0.5.5
[INFO] [stderr]   Downloaded enum-as-inner v0.6.1
[INFO] [stderr]   Downloaded seq-macro v0.3.5
[INFO] [stderr]   Downloaded gemm v0.17.1
[INFO] [stderr]   Downloaded rayon-cond v0.3.0
[INFO] [stderr]   Downloaded unicode-normalization-alignments v0.1.12
[INFO] [stderr]   Downloaded serde v1.0.214
[INFO] [stderr]   Downloaded cuda-config v0.1.0
[INFO] [stderr]   Downloaded cudarc v0.12.1
[INFO] [stderr]   Downloaded candle-kernels v0.7.2
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+82310651b93a594a3fd69015e1562186a080d94c" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 13e16dc60741d39e9196b58243a95c5078f65338de14ba046c10f6de3f81200d
[INFO] running `Command { std: "docker" "start" "-a" "13e16dc60741d39e9196b58243a95c5078f65338de14ba046c10f6de3f81200d", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "13e16dc60741d39e9196b58243a95c5078f65338de14ba046c10f6de3f81200d", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "13e16dc60741d39e9196b58243a95c5078f65338de14ba046c10f6de3f81200d", kill_on_drop: false }`
[INFO] [stdout] 13e16dc60741d39e9196b58243a95c5078f65338de14ba046c10f6de3f81200d
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+82310651b93a594a3fd69015e1562186a080d94c" "check" "--frozen" "--all" "--all-targets" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 8952bc4e3bc74ee6777cc1ed2495abe0b37e94c31f989f4a68d1f33b1459e077
[INFO] running `Command { std: "docker" "start" "-a" "8952bc4e3bc74ee6777cc1ed2495abe0b37e94c31f989f4a68d1f33b1459e077", kill_on_drop: false }`
[INFO] [stderr]    Compiling proc-macro2 v1.0.89
[INFO] [stderr]    Compiling unicode-ident v1.0.13
[INFO] [stderr]    Compiling libc v0.2.161
[INFO] [stderr]    Compiling autocfg v1.4.0
[INFO] [stderr]     Checking cfg-if v1.0.0
[INFO] [stderr]    Compiling crossbeam-utils v0.8.20
[INFO] [stderr]    Compiling libm v0.2.11
[INFO] [stderr]    Compiling rayon-core v1.12.1
[INFO] [stderr]     Checking byteorder v1.5.0
[INFO] [stderr]     Checking once_cell v1.20.2
[INFO] [stderr]     Checking memchr v2.7.4
[INFO] [stderr]     Checking either v1.13.0
[INFO] [stderr]    Compiling paste v1.0.15
[INFO] [stderr]    Compiling serde v1.0.214
[INFO] [stderr]     Checking bitflags v1.3.2
[INFO] [stderr]     Checking reborrow v0.5.5
[INFO] [stderr]    Compiling glob v0.3.1
[INFO] [stderr]     Checking raw-cpuid v10.7.0
[INFO] [stderr]     Checking itoa v1.0.11
[INFO] [stderr]    Compiling seq-macro v0.3.5
[INFO] [stderr]     Checking pin-project-lite v0.2.15
[INFO] [stderr]     Checking smallvec v1.13.2
[INFO] [stderr]    Compiling hashbrown v0.15.1
[INFO] [stderr]    Compiling serde_json v1.0.132
[INFO] [stderr]    Compiling equivalent v1.0.1
[INFO] [stderr]    Compiling shlex v1.3.0
[INFO] [stderr]    Compiling cc v1.1.36
[INFO] [stderr]    Compiling toml_datetime v0.6.8
[INFO] [stderr]     Checking ryu v1.0.18
[INFO] [stderr]    Compiling winnow v0.6.20
[INFO] [stderr]     Checking regex-syntax v0.8.5
[INFO] [stderr]    Compiling ident_case v1.0.1
[INFO] [stderr]     Checking aho-corasick v1.1.3
[INFO] [stderr]    Compiling quote v1.0.37
[INFO] [stderr]    Compiling num-traits v0.2.19
[INFO] [stderr]    Compiling lock_api v0.4.12
[INFO] [stderr]    Compiling thiserror v1.0.68
[INFO] [stderr]    Compiling syn v2.0.87
[INFO] [stderr]    Compiling fnv v1.0.7
[INFO] [stderr]     Checking crossbeam-epoch v0.9.18
[INFO] [stderr]    Compiling strsim v0.11.1
[INFO] [stderr]     Checking futures-core v0.3.31
[INFO] [stderr]    Compiling parking_lot_core v0.9.10
[INFO] [stderr]     Checking scopeguard v1.2.0
[INFO] [stderr]    Compiling slab v0.4.9
[INFO] [stderr]    Compiling cudarc v0.12.1
[INFO] [stderr]    Compiling zip v1.1.4
[INFO] [stderr]     Checking lazy_static v1.5.0
[INFO] [stderr]     Checking stable_deref_trait v1.2.0
[INFO] [stderr]    Compiling crossbeam-deque v0.8.5
[INFO] [stderr]    Compiling indexmap v2.6.0
[INFO] [stderr]    Compiling pkg-config v0.3.31
[INFO] [stderr] error: failed to run custom build command for `cudarc v0.12.1`
[INFO] [stderr] note: To improve backtraces for build dependencies, set the CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   process didn't exit successfully: `/opt/rustwide/target/debug/build/cudarc-ec32e410df717af5/build-script-build` (exit status: 101)
[INFO] [stderr]   --- stdout
[INFO] [stderr]   cargo:rerun-if-changed=build.rs
[INFO] [stderr]   cargo:rerun-if-env-changed=CUDA_ROOT
[INFO] [stderr]   cargo:rerun-if-env-changed=CUDA_PATH
[INFO] [stderr]   cargo:rerun-if-env-changed=CUDA_TOOLKIT_ROOT_DIR
[INFO] [stderr] 
[INFO] [stderr]   --- stderr
[INFO] [stderr] 
[INFO] [stderr]   thread 'main' panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/cudarc-0.12.1/build.rs:57:10:
[INFO] [stderr]   Failed to execute `nvcc`: Os { code: 2, kind: NotFound, message: "No such file or directory" }
[INFO] [stderr]   stack backtrace:
[INFO] [stderr]      0:     0x607c5cef6012 - std::backtrace_rs::backtrace::libunwind::trace::h76ea176a64426f0f
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stderr]      1:     0x607c5cef6012 - std::backtrace_rs::backtrace::trace_unsynchronized::h0830675862e8e766
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stderr]      2:     0x607c5cef6012 - std::sys::backtrace::_print_fmt::hba1ccef63faaa048
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/sys/backtrace.rs:66:9
[INFO] [stderr]      3:     0x607c5cef6012 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::hca516e2c7056a217
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/sys/backtrace.rs:39:26
[INFO] [stderr]      4:     0x607c5cf15853 - core::fmt::rt::Argument::fmt::ha510764ac9e77b66
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/core/src/fmt/rt.rs:173:76
[INFO] [stderr]      5:     0x607c5cf15853 - core::fmt::write::h97a802838d45fe7e
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/core/src/fmt/mod.rs:1469:25
[INFO] [stderr]      6:     0x607c5cef3603 - std::io::default_write_fmt::hb63ad55b2aa53928
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/io/mod.rs:639:11
[INFO] [stderr]      7:     0x607c5cef3603 - std::io::Write::write_fmt::h6b2df461464d9a05
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/io/mod.rs:1954:13
[INFO] [stderr]      8:     0x607c5cef5e62 - std::sys::backtrace::BacktraceLock::print::h81c102a9ac0a1bc7
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/sys/backtrace.rs:42:9
[INFO] [stderr]      9:     0x607c5cef6ee3 - std::panicking::default_hook::{{closure}}::h9934714bf98b056d
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:300:27
[INFO] [stderr]     10:     0x607c5cef6cc5 - std::panicking::default_hook::hb31a4fc725ffcf0a
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:327:9
[INFO] [stderr]     11:     0x607c5cef77b5 - std::panicking::rust_panic_with_hook::h8df5393cb811c708
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:833:13
[INFO] [stderr]     12:     0x607c5cef764a - std::panicking::begin_panic_handler::{{closure}}::h73dc0a59c2d3b9c7
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:706:13
[INFO] [stderr]     13:     0x607c5cef6509 - std::sys::backtrace::__rust_end_short_backtrace::h9eb84a109f69f8ca
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/sys/backtrace.rs:174:18
[INFO] [stderr]     14:     0x607c5cef72dd - __rustc[98c2592f34d7b78b]::rust_begin_unwind
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:697:5
[INFO] [stderr]     15:     0x607c5cf14b20 - core::panicking::panic_fmt::hb0df011123225e8b
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/core/src/panicking.rs:75:14
[INFO] [stderr]     16:     0x607c5cf14eb6 - core::result::unwrap_failed::h15fa24b59c94aa17
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/core/src/result.rs:1761:5
[INFO] [stderr]     17:     0x607c5cecef62 - core::result::Result<T,E>::expect::h9ba0da716a782cfb
[INFO] [stderr]     18:     0x607c5cec6d56 - build_script_build::cuda_version_from_build_system::h02cb25cdb7c43598
[INFO] [stderr]     19:     0x607c5cec6ad2 - build_script_build::main::h69b8aa410a80920a
[INFO] [stderr]     20:     0x607c5cecb263 - core::ops::function::FnOnce::call_once::h3bc25999d8cdf548
[INFO] [stderr]     21:     0x607c5cece016 - std::sys::backtrace::__rust_begin_short_backtrace::ha643797961e65116
[INFO] [stderr]     22:     0x607c5cecdf89 - std::rt::lang_start::{{closure}}::h91d2db01943cb868
[INFO] [stderr]     23:     0x607c5cef0290 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h0bcac900efc89129
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/core/src/ops/function.rs:290:21
[INFO] [stderr]     24:     0x607c5cef0290 - std::panicking::catch_unwind::do_call::hfadc8e58bd88a07a
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:589:40
[INFO] [stderr]     25:     0x607c5cef0290 - std::panicking::catch_unwind::h44c20dc1edb63a1e
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:552:19
[INFO] [stderr]     26:     0x607c5cef0290 - std::panic::catch_unwind::hed574ee6107b8e15
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panic.rs:359:14
[INFO] [stderr]     27:     0x607c5cef0290 - std::rt::lang_start_internal::{{closure}}::h7f49669407df0483
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/rt.rs:175:24
[INFO] [stderr]     28:     0x607c5cef0290 - std::panicking::catch_unwind::do_call::hdc3b107dc7611175
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:589:40
[INFO] [stderr]     29:     0x607c5cef0290 - std::panicking::catch_unwind::h50fbac1cdf73dd69
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panicking.rs:552:19
[INFO] [stderr]     30:     0x607c5cef0290 - std::panic::catch_unwind::h061b64ee85b76832
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/panic.rs:359:14
[INFO] [stderr]     31:     0x607c5cef0290 - std::rt::lang_start_internal::h935ec80c2dc417b9
[INFO] [stderr]                                  at /rustc/82310651b93a594a3fd69015e1562186a080d94c/library/std/src/rt.rs:171:5
[INFO] [stderr]     32:     0x607c5cecdf71 - std::rt::lang_start::h71b8c5ea3c1a0570
[INFO] [stderr]     33:     0x607c5cec85b5 - main
[INFO] [stderr]     34:     0x711e24a951ca - <unknown>
[INFO] [stderr]     35:     0x711e24a9528b - __libc_start_main
[INFO] [stderr]     36:     0x607c5cec3935 - _start
[INFO] [stderr]     37:                0x0 - <unknown>
[INFO] [stderr] warning: build failed, waiting for other jobs to finish...
[INFO] running `Command { std: "docker" "inspect" "8952bc4e3bc74ee6777cc1ed2495abe0b37e94c31f989f4a68d1f33b1459e077", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "8952bc4e3bc74ee6777cc1ed2495abe0b37e94c31f989f4a68d1f33b1459e077", kill_on_drop: false }`
[INFO] [stdout] 8952bc4e3bc74ee6777cc1ed2495abe0b37e94c31f989f4a68d1f33b1459e077